Methods
How the data table in every article is built, format by format, and what it does not measure.
Every table on this site is built by software. This page explains how, page type by page type, and what the tables do not measure.
Best tools, with the leaderboard’s verdict
The page names the model that a public leaderboard puts first on the day it was published. The leaderboard is Arena’s public leaderboard dataset, published under the CC BY 4.0 licence; Arena ranks models by the votes of people who compare the outputs of two anonymous models side by side. Our software writes the verdict sentence from the dataset: the model, its organisation, its score and number of votes, the date of the leaderboard and the day we read it. When the first model’s 95% interval reaches the second’s, the sentence says that its lead is within the leaderboard’s confidence interval. The table under the verdict lists the platforms whose own pages name a version of that model family, in alphabetical order, without a ranking, with their prices as their pages list them.
The verdict rule: a page says “the best” only in this sentence, only when the leaderboard gives it, never for a product with the same owner as this site, and never from a test of ours. A page whose topic has no public leaderboard has no verdict.
Weekly rankings
One leaderboard, one category, its models in the leaderboard’s own order, with the score, the 95% interval and the number of votes as the dataset publishes them, and the leaderboard’s publication date.
Prices
One tool’s plans, as its own pricing page lists them in each billing state, with the credits or limits of each plan, what each plan’s card lists, and the costs per clip or image when the page states them. Prices marked as time-limited offers are shown as such.
What the tables do not measure
The tables say what a page or a leaderboard states. They do not measure the quality of the output for your own prompts, the speed or the support, and none of it was tested by us.
Comparison table
Compares the model in the story with up to four recent models in our tracker, on attributes such as maximum clip length, maximum resolution, frame rate, native audio, price, access (API, app or open weights), licence and release date. Only figures quoted in primary sources and verified word for word are used, so a cell stays empty when no verified figure is on file. The tracker grows with every story we read; nothing in it comes from memory or from unverified sources.
Spec tracker
A variant of the comparison table focused on published specifications and limits. The same rules apply: each cell comes from a quote verified in a primary source, with the date it was read, and a cell stays empty when the figure is not published.
Repository metrics
Stars, forks, open issues, licence and date of the last push of the code repositories cited in the story, read from the public GitHub REST API at the time shown with the table. These figures change every day: the table is a snapshot, not a ranking of quality.
Paper trend
Weekly counts of new arXiv papers matching a query set for this publication, for the last eight complete weeks, Monday to Sunday, read from the public arXiv API (metadata under CC0). The last column is the difference from the previous week, with a plus sign when the count rose. The arXiv API selects papers by submission date in GMT, so a paper submitted close to midnight can fall in the neighbouring week. A rise or fall in the count does not show its cause.
Timeline
Release dates taken only from quotes verified in primary sources stored by our pipeline, in date order. It is used only when at least five such dates are on file.
Prices and specifications
Prices and specifications of the tool in the story and of its direct competitors, read from their official pricing and documentation pages. The competitors come from a fixed list per category, not from the story. Each cell keeps the page address and the line it was read from; a cell stays empty when the page does not state the figure. Units can differ between vendors (dollars, credits): the table shows them as published.
Release history
Dated entries the project published in its own release notes, news list or changelog, newest first, with the page they were read from. The text of each entry is the project's own wording: it says what the project announced and when, not whether it works as described.
Published results
A results table published by the project on its own page (benchmarks, hardware needs, timings), copied cell for cell with the page address. These are the project's own figures, not an independent test.
Guide table
A comparison table for a guide: every cell was read on the official page named in its row (pricing pages, documentation, papers, reports), with the line that holds it. Each value is taken with its quote, and our code keeps a cell only when that line is on the page and holds the value word for word; an empty cell means the page did not state it in a form our code could match. No competitor's article is ever used. The verification date is shown on the guide.