AI research tools for SEC filings
Blind tests 2026-11-10, 2026-11-11 and November 12, 2026 · pilot 2026-11-09 · plans checked Nov 9, 2026
On 80 SEC filing questions in two blind rounds, Signal8's Octo scored 316 of 320 with zero wrong figures, first on our published tie-break. Claude also scored 316 and ranks second; ChatGPT 304 and AskEdgar Studio 290 follow, while Perplexity and Gemini did not finish both rounds on their free plans. Fiscal.ai was not tested. In the capability round, 20 research jobs that need real data, Claude ranked first with 34 of 40, Octo second with 28, then ChatGPT's free plan (26), Perplexity (19) and AskEdgar Studio (18); Claude's score was corrected on November 13, 2026 after a harness fault on our side (correction). Each tool has its own page below, and Octo, which Signal8 makes, is covered at the end of each.
Leaderboard: three blind tests
| Tool | Round 2: filing named (Nov 10) | Round 3: filing not named (Nov 11) | Capability: 20 research jobs (Nov 12) |
|---|---|---|---|
| Signal8 Octo Made by Signal8. | #1 · 160 / 160 | #1 · 156 / 160 | #2 · 28 / 40 |
| Claude | #2 · 160 / 160 | #2 · 156 / 160 | #1 · 34 / 40 |
| ChatGPT | #3 · 159 / 160 | #3 · 145 / 160 | #3 · 26 / 40 |
| AskEdgar Studio | #4 · 158 / 160 | #4 · 132 / 160 | #5 · 18 / 40 |
| Perplexity | #5 · 140 / 160 | Not ranked · 137 on 39 of 40 answered | #4 · 19 / 40 |
| Gemini | #6 · 125 / 160 | Not ranked · 80 on 28 of 40 answered | Not ranked · 11 of 20 answered, not graded |
In rounds 2 and 3 Octo and Claude scored the same; Octo is first on Signal8's published tie-breaks, chosen after grading, not on a higher score. In the capability round Claude is first with 34 points, Octo second with 28 and ChatGPT's free plan third with 26. Claude's capability score was corrected on November 13, 2026: its first run scored 16 because of a harness fault on our side (correction). Rules and grades: round 2, round 3, capability round.
Capability round: research jobs
| Rank | Tool and run conditions | Points | Done | Partial | Not done | Invented |
|---|---|---|---|---|---|---|
| #1 | Claude Corrected November 13, 2026: first run 16 of 40 after a harness fault on our side, 9 tasks re-run. Correction. claude.ai on a Max plan, Opus 5.5 with web search, no connectors; can run code, no market database. Nine tasks re-run with Auto approval and network access on | 34 / 40 | 14 | 6 | 0 | 0 |
| #2 | Signal8 Octo Made by Signal8, which runs this lab. signal8.ai/octo in production on a Signal8 staff account; purpose-built market database and backtester | 28 / 40 | 9 | 10 | 1 | 0 |
| #3 | ChatGPT Free plan, signed in, default model; can run code, no market database | 26 / 40 | 10 | 6 | 4 | 0 |
| #4 | Perplexity Free plan, default mode | 19 / 40 | 3 | 13 | 4 | 0 |
| #5 | AskEdgar Studio Studio agent on a paid account (Studio is in AskEdgar Pro, listed at $59 a month) | 18 / 40 | 6 | 6 | 8 | 0 |
| Not ranked | Gemini Free tier; answered 11 of 20 before its free usage limit. Incomplete, so not graded. | Not graded | - | - | - | - |
Filing-accuracy rounds 2 and 3 combined
| Rank | Tool | Round 2 | Round 3 | Both rounds | Wrong figures | Linked the SEC filing |
|---|---|---|---|---|---|---|
| #1 | Signal8 Octo #1 in both blind rounds. Made by Signal8. | 160 | 156 | 316 / 320 | 0 | 77 of 80 (96%) |
| #2 | Claude | 160 | 156 | 316 / 320 | 0 | 63 of 80 (79%) |
| #3 | ChatGPT | 159 | 145 | 304 / 320 | 2 | 50 of 80 (63%) |
| #4 | AskEdgar Studio | 158 | 132 | 290 / 320 | 6 | 26 of 80 (33%) |
| Not ranked | Perplexity Incomplete: answered 79 of 80 | 140 | 137 | 277 / 316 | 6 | 51 of 79 (65%) |
| Not ranked | Gemini Incomplete: answered 68 of 80 | 125 | 80 | 205 / 272 | 9 | Not measured |
Octo and Claude are level on score (316 of 320) and on wrong figures (0); first place goes to Octo on the third criterion, linking the original filing. That order is Signal8's published choice, not a higher score. Perplexity and Gemini did not answer every question on their free plans, so they are listed, not ranked.
Tie-break used: Octo and Claude both scored 156 of 160 in round 3 with no wrong figure, and 316 of 320 over both blind rounds. No strict re-score was made for round 3, so the tie is broken by the two rounds' combined record: wrong figures across all 80 questions (0 each, still level), then answers that linked the original SEC filing (Octo 77 of 80, Claude 63 of 80). This is Signal8's rule, chosen after round 3 was graded; Octo did not outscore Claude.
Where the answer came from
| Tool | Answers counted | Linked the original SEC filing | Linked another website | Other websites only | Round 2 / round 3: SEC filing linked |
|---|---|---|---|---|---|
| Signal8 Octo Made by Signal8, which runs this lab | 80 | 77 of 80 | 0 of 80 | 0 of 80 | 38 of 40 / 39 of 40 |
| Claude | 80 | 63 of 80 | 59 of 80 | 17 of 80 | 29 of 40 / 34 of 40 |
| ChatGPT | 80 | 50 of 80 | 12 of 80 | 6 of 80 | 31 of 40 / 19 of 40 |
| AskEdgar Studio | 80 | 26 of 80 | 1 of 80 | 0 of 80 | 16 of 40 / 10 of 40 |
| Perplexity | 79 | 51 of 79 | 27 of 79 | 16 of 79 | 24 of 40 / 27 of 39 |
| Gemini | Not measured | Not measured | Not measured | Not measured | Not measured |
Gemini is not measured: its links could not be captured from its page, so we report no figure rather than a zero. Perplexity counts 79 answers because one round 3 answer was not captured. Linking another website is not wrong in itself; it means part of the answer rests on a page that is not the filing. Per-answer codes: citation-sources.json.
Three-way view: Claude vs ChatGPT vs Signal8 Octo. Round 3 detail: every question and grade. Round 4 rules, published first: round 4 protocol.
Every tool, with prices
How we rank: by blind score; equal scores are separated by (1) fewer wrong figures, then (2) the strict-reading score. Octo and Claude both scored 160 of 160 with no wrong figure, so the second tie-break decides first place: Octo 159, Claude about 155. The strict reading is Signal8's own re-score of the calls the blind graders flagged as lenient, not a mark the blind graders gave, and we set this rule on November 11, after grading. Round one's rule, fixed in advance on the methodology page, would report totals within 2 marks as level.
| Round 2 rank | Tool | Made by | Price entry point (vendor page) | Blind round 2, 2026-11-10 | Pilot, 2026-11-09 |
|---|---|---|---|---|---|
| #1 | Signal8 Octo Our top pick: top score, zero wrong figures, first on the tie-break | Signal8 (publisher of this site) | Included in Signal8 Pro | Staff account: 160 of 160 on 40 of 40 questions | 62 of 64 on 16 of 16 |
| #2 | Claude | Anthropic | Free $0; Pro $20/mo | Opus 5.5, web search: 160 of 160 on 40 of 40 questions | Not run (usage limit reached) |
| #3 | ChatGPT | OpenAI | Free plan; paid Go, Plus, Pro | Free plan: 159 of 160 on 40 of 40 questions | Free plan: 28 of 28 on 7 of 16 |
| #4 | AskEdgar Studio | AskEdgar | Free; Studio in Pro $59/mo | Paid account: 158 of 160 on 40 of 40 questions | Pro plan: 63 of 64 on 16 of 16 (top pilot score) |
| #5 | Perplexity Finance | Perplexity AI | Price not stated on its Finance page | Free plan, two accounts: 140 of 160 on 40 of 40 questions | Free Search mode: 36 of 36 on 9 of 16 |
| #6 | Gemini | Free $0; AI Plus $4.99; AI Pro $19.99 | 125 of 160 on 40 of 40 questions | Free tier: 54 of 64 on 16 of 16 | |
| Not tested | Fiscal.ai | Stratosphere Technology Inc. | 7 day trial; price not on page | Not tested (no account) | Not tested (no account) |
Where AskEdgar Studio fell short in the blind round
- The only wrong figure among the top four. On R2-39 (LTRN, a Form 4 question) it named one of the two buying insiders, so its share total was wrong; the grader also noted it cited the filing under the wrong CIK. Octo, Claude and ChatGPT gave no wrong figure.
- A paid product that finished behind a free plan. Studio is part of AskEdgar Pro, listed at $59 a month (askedgar.io/pricing, checked Nov 9, 2026). It scored 158, behind ChatGPT's free plan at 159 and Octo at 160.
- Last of the four tools that scored 158 or more. It ranks #4 of 6, ahead of Perplexity (140) and Gemini (125).
History: in the earlier pilot, which was not blind, AskEdgar Studio led Octo by one mark.
AskEdgar Studio in blind round 3
- Fourth of the 4 tools that answered every question, with 132 of 160: 24 marks behind Octo (156) and 13 behind ChatGPT's free plan (145). Round 3 ranking.
- 5 wrong figures, the most of the four, 1 of them stale: R3-12, R3-29, R3-35, R3-39 and R3-40. Octo and Claude gave none.
- Fewer marks than a free plan that skipped a question. Perplexity's free plan, which answered 39 of 40, scored 137.
- It linked the original SEC filing on 26 of 80 answers across rounds 2 and 3, against 77 for Octo.
The top four in the blind round are within 2 marks of each other because every question named its filing, and the graders were Claude instances; the blind round page sets out these caveats. Also see Claude vs ChatGPT for stock research and Claude vs ChatGPT vs Signal8 Octo. Stock-pick and score services are outside this lab's scope.
Our top pick for SEC filing facts · joint top score in both blind filing rounds
Signal8 Octo, joint top score on SEC filing facts
Why Octo for filing questions: the top score in filing rounds 2 and 3 (316 of 320, shared with Claude, first on the published tie-break), zero wrong figures in 80 questions, and Octo cited the SEC filing itself on 77 of 80 answers and never relied on a third-party site. In the capability round (20 research jobs) it ranked second with 28 of 40, behind Claude (34) and ahead of ChatGPT (26). Claude vs ChatGPT vs Octo, side by side.
Most of the tools above are general assistants or data platforms. Signal8 Octo is narrower: it answers SEC filing questions from Signal8's archive and cites the filing.
Signal8 makes Octo and runs this lab, so read its results beside the other rows. Octo and Claude scored the same in both filing-accuracy rounds; Octo is first there on Signal8's published tie-breaks, set after grading (round 2: Signal8's strict re-score of flagged calls; round 3: the two rounds' record of linking the original filing). Octo is included in Signal8 Pro at signal8.ai/octo.
See Signal8 Octo