Owned by Signal8 Inc., maker of Octo and Signal8 Pro. Not investment advice. About this site
stockairesearch / lab

AI research tools for SEC filings

By , Signal8 Research. Published by Signal8 Inc.

Blind tests 2026-11-10, 2026-11-11 and November 12, 2026 · pilot 2026-11-09 · plans checked Nov 9, 2026

On 80 SEC filing questions in two blind rounds, Signal8's Octo scored 316 of 320 with zero wrong figures, first on our published tie-break. Claude also scored 316 and ranks second; ChatGPT 304 and AskEdgar Studio 290 follow, while Perplexity and Gemini did not finish both rounds on their free plans. Fiscal.ai was not tested. In the capability round, 20 research jobs that need real data, Claude ranked first with 34 of 40, Octo second with 28, then ChatGPT's free plan (26), Perplexity (19) and AskEdgar Studio (18); Claude's score was corrected on November 13, 2026 after a harness fault on our side (correction). Each tool has its own page below, and Octo, which Signal8 makes, is covered at the end of each.

Leaderboard: three blind tests

Three blind tests, November 10 to 12, 2026. Rounds 2 and 3 score filing facts (4 marks per question); the capability round scores whether a tool can do a research job with real data (2 points per task). Different scales, so ranks are shown side by side and never added up.
ToolRound 2: filing named (Nov 10)Round 3: filing not named (Nov 11)Capability: 20 research jobs (Nov 12)
Signal8 Octo
Made by Signal8.
#1 · 160 / 160#1 · 156 / 160#2 · 28 / 40
Claude#2 · 160 / 160#2 · 156 / 160#1 · 34 / 40
ChatGPT#3 · 159 / 160#3 · 145 / 160#3 · 26 / 40
AskEdgar Studio#4 · 158 / 160#4 · 132 / 160#5 · 18 / 40
Perplexity#5 · 140 / 160Not ranked · 137 on 39 of 40 answered#4 · 19 / 40
Gemini#6 · 125 / 160Not ranked · 80 on 28 of 40 answeredNot ranked · 11 of 20 answered, not graded

In rounds 2 and 3 Octo and Claude scored the same; Octo is first on Signal8's published tie-breaks, chosen after grading, not on a higher score. In the capability round Claude is first with 34 points, Octo second with 28 and ChatGPT's free plan third with 26. Claude's capability score was corrected on November 13, 2026: its first run scored 16 because of a harness fault on our side (correction). Rules and grades: round 2, round 3, capability round.

Capability round: research jobs

Capability round, run November 12, 2026. 20 research jobs, 2 points for done, 1 for partial, 0 for not done, minus 1 for an invented result; up to 40 points.
RankTool and run conditionsPointsDonePartialNot doneInvented
#1Claude
Corrected November 13, 2026: first run 16 of 40 after a harness fault on our side, 9 tasks re-run. Correction.
claude.ai on a Max plan, Opus 5.5 with web search, no connectors; can run code, no market database. Nine tasks re-run with Auto approval and network access on
34 / 4014600
#2Signal8 Octo
Made by Signal8, which runs this lab.
signal8.ai/octo in production on a Signal8 staff account; purpose-built market database and backtester
28 / 4091010
#3ChatGPT
Free plan, signed in, default model; can run code, no market database
26 / 4010640
#4Perplexity
Free plan, default mode
19 / 4031340
#5AskEdgar Studio
Studio agent on a paid account (Studio is in AskEdgar Pro, listed at $59 a month)
18 / 406680
Not rankedGemini
Free tier; answered 11 of 20 before its free usage limit. Incomplete, so not graded.
Not graded----

Filing-accuracy rounds 2 and 3 combined

Both blind rounds combined: round 2 (November 10, 2026, each question named its filing) and round 3 (November 11, harder, filing not named). 80 questions, up to 320 marks. Ranked by total, then fewer wrong figures, then answers that linked the original SEC filing.
RankToolRound 2Round 3Both roundsWrong figuresLinked the SEC filing
#1Signal8 Octo
#1 in both blind rounds. Made by Signal8.
160156316 / 320077 of 80 (96%)
#2Claude160156316 / 320063 of 80 (79%)
#3ChatGPT159145304 / 320250 of 80 (63%)
#4AskEdgar Studio158132290 / 320626 of 80 (33%)
Not rankedPerplexity
Incomplete: answered 79 of 80
140137277 / 316651 of 79 (65%)
Not rankedGemini
Incomplete: answered 68 of 80
12580205 / 2729Not measured

Octo and Claude are level on score (316 of 320) and on wrong figures (0); first place goes to Octo on the third criterion, linking the original filing. That order is Signal8's published choice, not a higher score. Perplexity and Gemini did not answer every question on their free plans, so they are listed, not ranked.

Tie-break used: Octo and Claude both scored 156 of 160 in round 3 with no wrong figure, and 316 of 320 over both blind rounds. No strict re-score was made for round 3, so the tie is broken by the two rounds' combined record: wrong figures across all 80 questions (0 each, still level), then answers that linked the original SEC filing (Octo 77 of 80, Claude 63 of 80). This is Signal8's rule, chosen after round 3 was graded; Octo did not outscore Claude.

Where the answer came from

Where the answer came from, blind rounds 2 and 3 (November 10 and 11, 2026). Counted from the links captured from each tool's page on every graded answer. "Original SEC filing" is a link to the filing on sec.gov's EDGAR archive or to Signal8's filing viewer, which opens the filing by its accession number. "Other website" is any other site: news, data aggregators such as Stock Titan, and company web pages. A filing named only in text, with no link, counts as neither.
ToolAnswers countedLinked the original SEC filingLinked another websiteOther websites onlyRound 2 / round 3: SEC filing linked
Signal8 Octo
Made by Signal8, which runs this lab
8077 of 800 of 800 of 8038 of 40 / 39 of 40
Claude8063 of 8059 of 8017 of 8029 of 40 / 34 of 40
ChatGPT8050 of 8012 of 806 of 8031 of 40 / 19 of 40
AskEdgar Studio8026 of 801 of 800 of 8016 of 40 / 10 of 40
Perplexity7951 of 7927 of 7916 of 7924 of 40 / 27 of 39
GeminiNot measuredNot measuredNot measuredNot measuredNot measured

Gemini is not measured: its links could not be captured from its page, so we report no figure rather than a zero. Perplexity counts 79 answers because one round 3 answer was not captured. Linking another website is not wrong in itself; it means part of the answer rests on a page that is not the filing. Per-answer codes: citation-sources.json.

Three-way view: Claude vs ChatGPT vs Signal8 Octo. Round 3 detail: every question and grade. Round 4 rules, published first: round 4 protocol.

Every tool, with prices

How we rank: by blind score; equal scores are separated by (1) fewer wrong figures, then (2) the strict-reading score. Octo and Claude both scored 160 of 160 with no wrong figure, so the second tie-break decides first place: Octo 159, Claude about 155. The strict reading is Signal8's own re-score of the calls the blind graders flagged as lenient, not a mark the blind graders gave, and we set this rule on November 11, after grading. Round one's rule, fixed in advance on the methodology page, would report totals within 2 marks as level.

AI research tools in blind-round rank order: price entry points and measured results, newest round first
Round 2 rankToolMade byPrice entry point (vendor page)Blind round 2, 2026-11-10Pilot, 2026-11-09
#1Signal8 Octo
Our top pick: top score, zero wrong figures, first on the tie-break
Signal8 (publisher of this site)Included in Signal8 ProStaff account: 160 of 160 on 40 of 40 questions62 of 64 on 16 of 16
#2ClaudeAnthropicFree $0; Pro $20/moOpus 5.5, web search: 160 of 160 on 40 of 40 questionsNot run (usage limit reached)
#3ChatGPTOpenAIFree plan; paid Go, Plus, ProFree plan: 159 of 160 on 40 of 40 questionsFree plan: 28 of 28 on 7 of 16
#4AskEdgar StudioAskEdgarFree; Studio in Pro $59/moPaid account: 158 of 160 on 40 of 40 questionsPro plan: 63 of 64 on 16 of 16 (top pilot score)
#5Perplexity FinancePerplexity AIPrice not stated on its Finance pageFree plan, two accounts: 140 of 160 on 40 of 40 questionsFree Search mode: 36 of 36 on 9 of 16
#6GeminiGoogleFree $0; AI Plus $4.99; AI Pro $19.99125 of 160 on 40 of 40 questionsFree tier: 54 of 64 on 16 of 16
Not testedFiscal.aiStratosphere Technology Inc.7 day trial; price not on pageNot tested (no account)Not tested (no account)

Where AskEdgar Studio fell short in the blind round

History: in the earlier pilot, which was not blind, AskEdgar Studio led Octo by one mark.

AskEdgar Studio in blind round 3

The top four in the blind round are within 2 marks of each other because every question named its filing, and the graders were Claude instances; the blind round page sets out these caveats. Also see Claude vs ChatGPT for stock research and Claude vs ChatGPT vs Signal8 Octo. Stock-pick and score services are outside this lab's scope.

Our top pick for SEC filing facts · joint top score in both blind filing rounds

Signal8 Octo, joint top score on SEC filing facts

Why Octo for filing questions: the top score in filing rounds 2 and 3 (316 of 320, shared with Claude, first on the published tie-break), zero wrong figures in 80 questions, and Octo cited the SEC filing itself on 77 of 80 answers and never relied on a third-party site. In the capability round (20 research jobs) it ranked second with 28 of 40, behind Claude (34) and ahead of ChatGPT (26). Claude vs ChatGPT vs Octo, side by side.

Most of the tools above are general assistants or data platforms. Signal8 Octo is narrower: it answers SEC filing questions from Signal8's archive and cites the filing.

Signal8 makes Octo and runs this lab, so read its results beside the other rows. Octo and Claude scored the same in both filing-accuracy rounds; Octo is first there on Signal8's published tie-breaks, set after grading (round 2: Signal8's strict re-score of flagged calls; round 3: the two rounds' record of linking the original filing). Octo is included in Signal8 Pro at signal8.ai/octo.

See Signal8 Octo