Transparent by design
Evidence methodology
Dated facts, explicit source permissions, exact attribution, uncertainty and review gates.
Methodology version 1.0.0 — 2026-10-10
Official facts, developer reports, third-party benchmarks and editorial interpretation are different information classes. Providers, coding agents, models and subscription plans are distinct entities. A developer report establishes what one author reported, not universal product behavior.
Official facts and freshness
The current catalog contains manually inspected US/USD monthly offers. Every offer links to its official source and records the actual observation date. Seven days after observation, a price requires rechecking. Rechecking unchanged content is not a price change. Tax, geography, annual commitments and API costs cannot be silently converted into comparable monthly subscriptions.
Frozen taxonomy
task_completion: exclude generic praise without task detail; code_quality: exclude the mere existence of generated code; bug_fixing: exclude GitHub issue describing a bug in some unrelated library; refactoring: exclude generic mention of “refactor”; context_handling: exclude context window marketing spec alone; speed_latency: exclude monthly quota exhaustion; usage_limits: exclude price alone or one isolated network outage; reliability: exclude inaccurate code, which belongs to quality/task dimensions; developer_experience: exclude benchmark performance not experienced by author; pricing_value: exclude price announcement without user's value judgment; security_privacy: exclude speculative safety rhetoric unsupported by actual usage. Workflow tags are filters, not sentiment dimensions.
Developer evidence qualification
Candidates must identify an exact canonical entity and a unique unchanged quote from the main author text. Parent comments can disambiguate attribution but cannot supply a purported direct quote. Model confidence does not establish correctness. Valid candidates remain pending semantic human review. Questions, promotions, announcements and hearsay do not become firsthand experience votes.
Current measurement and ranking policy
The new held-out quality gates are unmeasured. No automatic claim publication, numerical rankings, satisfaction percentages or inferred winners are enabled in public v1. Historical agent-reviewed summaries are legacy unverified material. They are not a two-reviewer gold dataset.
Sampling and independence
A bounded source selection is not representative of all developers. Language, query selection, author concentration and collection failures introduce bias. Mirrors and duplicate text must not count as independent experiences. No ratio is presented as product satisfaction. Missing or unavailable source coverage is distinct from zero negative reports.
Sources, removal and corrections
HN and GitHub processing/display require reviewed permission records; those v1 permissions are pending. Reddit and Facebook remain disabled. Edited, deleted or restricted sources invalidate dependent claims and current aggregates. Submit a factual correction at /corrections/. Submissions are reviewed by an editor; they do not change published facts or trigger model processing.
Independence and limitations
No affiliate payout, sponsorship or advertisement influences comparisons. There are no qualified benchmark results in the current catalog. Official vendor offers are advertised terms, not measured output quality.