Claude vs ChatGPT vs Signal8 Octo for stock research
Blind tests 2026-11-10, 2026-11-11 and November 12, 2026 · 80 SEC filing questions and 20 research jobs · Octo first on filing facts by tie-break, Claude first on research jobs · Signal8 makes Octo and runs this lab
In the capability round, 20 research jobs that need real data or computation, Claude scored 34 of 40, Signal8 Octo 28 and ChatGPT's free plan 26; Claude's score was corrected on November 13, 2026 after a harness fault on our side (correction). On SEC filing facts, Octo ranked first in both rounds, Claude second and ChatGPT third. Octo and Claude scored the same, 316 of 320, both with zero wrong figures; ChatGPT scored 304 with 2 wrong figures. The difference between Octo and Claude is in the sourcing: Octo cited the SEC filing itself on 77 of 80 answers and never relied on a third-party site. Claude linked the filing on 63 of 80 and other websites on 59. Signal8 makes Octo and runs this lab.
Our top pick for SEC filing facts · joint top score in both blind filing rounds
Signal8 Octo, joint top score on SEC filing facts
Why Octo for filing questions: the top score in filing rounds 2 and 3 (316 of 320, shared with Claude, first on the published tie-break), zero wrong figures in 80 questions, and Octo cited the SEC filing itself on 77 of 80 answers and never relied on a third-party site. In the capability round (20 research jobs) it ranked second with 28 of 40, behind Claude (34) and ahead of ChatGPT (26). Claude vs ChatGPT vs Octo, side by side.
Octo answers from Signal8's archive of SEC filings and links each figure to the filing by its accession number. Claude and ChatGPT are general assistants that search the open web.
Signal8 makes Octo and runs this lab, so read its results beside the other rows. Octo and Claude scored the same in both filing-accuracy rounds; Octo is first there on Signal8's published tie-breaks, set after grading (round 2: Signal8's strict re-score of flagged calls; round 3: the two rounds' record of linking the original filing). Octo is included in Signal8 Pro at signal8.ai/octo.
See Signal8 OctoBoth blind rounds, side by side
| Measure | Signal8 Octo | Claude | ChatGPT |
|---|---|---|---|
| Rank, both rounds | #1 | #2 | #3 |
| Round 2 score | 160 / 160 | 160 / 160 | 159 / 160 |
| Round 3 score | 156 / 160 | 156 / 160 | 145 / 160 |
| Both rounds | 316 / 320 | 316 / 320 | 304 / 320 |
| Wrong figures (80 questions) | 0 | 0 | 2 |
| Answers with 4 of 4 | 76 of 80 | 76 of 80 | 72 of 80 |
| Linked the original SEC filing | 77 of 80 | 63 of 80 | 50 of 80 |
| Linked another website | 0 of 80 | 59 of 80 | 12 of 80 |
| Other websites only | 0 of 80 | 17 of 80 | 6 of 80 |
How the tie is broken. Octo and Claude are level on score and on wrong figures in both rounds, so first place comes from Signal8's published tie-breaks, set after grading: in round 2 Signal8's strict re-score of calls the graders flagged as lenient (Octo 159, Claude about 155); in round 3 and over both rounds, answers that linked the original SEC filing (Octo 77, Claude 63). We do not claim Octo outscored Claude. Rules and every grade: round 2, round 3.
Capability round: can each one do the job?
The capability round on November 12, 2026 asked for results, not facts: run a backtest, screen the market, pull a structured history, run an event study, chain several lookups. Each tool was tested as shipped, with no follow-up. Claude and ChatGPT can write and run code but have no market database; Octo has one, plus a backtester. Graded blind: done 2, partial 1, not done 0, invented -1.
| Measure | Signal8 Octo | Claude | ChatGPT |
|---|---|---|---|
| Rank of 5 | #2 | #1 | #3 |
| Points | 28 / 40 | 34 / 40 | 26 / 40 |
| Done | 9 | 14 | 10 |
| Partial | 10 | 6 | 6 |
| Not done | 1 | 0 | 4 |
| Invented | 0 | 0 | 0 |
| Backtests (5) | 5 | 6 | 5 |
| Market screens (4) | 7 | 7 | 7 |
| Lookups (4) | 7 | 7 | 4 |
| Event studies (3) | 4 | 6 | 4 |
| Multi-step (4) | 5 | 8 | 6 |
Claude's lead over Octo comes from the event studies (6 points to 4) and multi-step jobs (8 to 5), plus a point on backtests; the two are level on screens and lookups. Octo's lead over ChatGPT comes from the per-ticker lookups (7 points to 4); the two are level on backtests, screens and event studies, and ChatGPT is a point ahead on multi-step jobs. They got there differently: ChatGPT matched the reference on the SPY and QQQ backtests (C-01, C-02) but did not run the three market-wide ones, while Octo ran all five on real data, including delisted stocks, and its 50-position cap and missing 200-day filter left every one partial. In its first run, with network access off on our account, Claude wrote and tested backtest code for C-01 and C-03 to C-05 without real data to run it on; those four stayed partial, because only its 9 zero-point tasks were re-run. In the re-run it did the QQQ backtest (C-02) on real prices and matched the reference. Graders were Claude instances. Every task and grade.
Where the answer came from
Each answer's links were captured from the tool's page and sorted: the filing itself on sec.gov (or Signal8's viewer, which opens it by accession number), or another website such as a news story, an aggregator like Stock Titan, or a company page. A right figure that rests on a news page is harder to check than one that links the filing. ChatGPT's page showed no link at all on 24 of its 80 answers.
| Tool | Answers counted | Linked the original SEC filing | Linked another website | Other websites only | Round 2 / round 3: SEC filing linked |
|---|---|---|---|---|---|
| Signal8 Octo Made by Signal8, which runs this lab | 80 | 77 of 80 | 0 of 80 | 0 of 80 | 38 of 40 / 39 of 40 |
| Claude | 80 | 63 of 80 | 59 of 80 | 17 of 80 | 29 of 40 / 34 of 40 |
| ChatGPT | 80 | 50 of 80 | 12 of 80 | 6 of 80 | 31 of 40 / 19 of 40 |
| AskEdgar Studio | 80 | 26 of 80 | 1 of 80 | 0 of 80 | 16 of 40 / 10 of 40 |
| Perplexity | 79 | 51 of 79 | 27 of 79 | 16 of 79 | 24 of 40 / 27 of 39 |
| Gemini | Not measured | Not measured | Not measured | Not measured | Not measured |
Gemini is not measured: its links could not be captured from its page, so we report no figure rather than a zero. Perplexity counts 79 answers because one round 3 answer was not captured. Linking another website is not wrong in itself; it means part of the answer rests on a page that is not the filing. Per-answer codes: citation-sources.json.
Where they lost marks
- Signal8 Octo: no wrong figure in either round. Four source marks in round 3 (R3-02, R3-07, R3-12, R3-27), all on questions that count reverse splits, where it did not cite every split's own 8-K. A fix is in progress at Signal8.
- Claude: no wrong figure in either round. Four source marks in round 3 (R3-12, R3-16, R3-27, R3-30), where part of the answer rested on a page that did not state the figure or could not be opened, often a news or aggregator site.
- ChatGPT (free plan): one source mark in round 2 (R2-40), and in round 3 7 answers lost marks (R3-10, R3-12, R3-27, R3-32, R3-35, R3-36, R3-40), 2 of them with a wrong figure.
What each is built for
| Tool | Built for | Data it works from |
|---|---|---|
| Signal8 Octo | Stock research on US companies: filing questions, backtests in Octo Lab, dilution screens | Signal8's database: SEC filings, dilution data, trading halts, borrow rates and insider trades |
| Claude | A general assistant: writing, analysis and code | Web search and files you upload, in the setup we tested; a market database has to be connected or supplied by you |
| ChatGPT | A general assistant: writing, analysis, deep research and code | Web search and files you upload, in the setup we tested; a market database has to be connected or supplied by you |
So the choice depends on the job. For anything outside markets, Claude and ChatGPT are the wider tools. For a filing fact about a US small cap, or a screen or backtest on market data, Octo starts from the data rather than from a web search.
Plans and run conditions
Octo ran on signal8.ai/octo in production on a Signal8 staff account. Claude ran on claude.ai with web search on; in round 2 it ran as Opus 5.5 with Signal8's own connectors switched off, and none of its round 3 answers used Signal8 data. ChatGPT ran on its free plan. Claude's plans run from Free at $0 to Pro at $20 a month and up; ChatGPT's from Free to Go, Plus and Pro (prices on the Claude vs ChatGPT page). Each question went verbatim to a fresh chat, and answers were graded blind by Claude instances, the same model family as one contestant, against keys sealed before the run.
How to check any of the three yourself
- Ask for the filing's form, date and accession number, then open it on EDGAR.
- If the answer links a news or aggregator page, find the same figure in the filing before relying on it.
- Check for a later filing that changes the figure, and for any reverse split since.
Related: Claude vs ChatGPT for stock research, all AI research tools, and the round 4 protocol, which adds a mark for citing the original filing.