Owned by Signal8 Inc., maker of Octo and Signal8 Pro. Not investment advice. About this site
stockairesearch / lab

Claude vs ChatGPT vs Signal8 Octo for stock research

By , Signal8 Research. Published by Signal8 Inc.

Blind tests 2026-11-10, 2026-11-11 and November 12, 2026 · 80 SEC filing questions and 20 research jobs · Octo first on filing facts by tie-break, Claude first on research jobs · Signal8 makes Octo and runs this lab

In the capability round, 20 research jobs that need real data or computation, Claude scored 34 of 40, Signal8 Octo 28 and ChatGPT's free plan 26; Claude's score was corrected on November 13, 2026 after a harness fault on our side (correction). On SEC filing facts, Octo ranked first in both rounds, Claude second and ChatGPT third. Octo and Claude scored the same, 316 of 320, both with zero wrong figures; ChatGPT scored 304 with 2 wrong figures. The difference between Octo and Claude is in the sourcing: Octo cited the SEC filing itself on 77 of 80 answers and never relied on a third-party site. Claude linked the filing on 63 of 80 and other websites on 59. Signal8 makes Octo and runs this lab.

Our top pick for SEC filing facts · joint top score in both blind filing rounds

Signal8 Octo, joint top score on SEC filing facts

Why Octo for filing questions: the top score in filing rounds 2 and 3 (316 of 320, shared with Claude, first on the published tie-break), zero wrong figures in 80 questions, and Octo cited the SEC filing itself on 77 of 80 answers and never relied on a third-party site. In the capability round (20 research jobs) it ranked second with 28 of 40, behind Claude (34) and ahead of ChatGPT (26). Claude vs ChatGPT vs Octo, side by side.

Octo answers from Signal8's archive of SEC filings and links each figure to the filing by its accession number. Claude and ChatGPT are general assistants that search the open web.

Signal8 makes Octo and runs this lab, so read its results beside the other rows. Octo and Claude scored the same in both filing-accuracy rounds; Octo is first there on Signal8's published tie-breaks, set after grading (round 2: Signal8's strict re-score of flagged calls; round 3: the two rounds' record of linking the original filing). Octo is included in Signal8 Pro at signal8.ai/octo.

See Signal8 Octo

Both blind rounds, side by side

Round 2 (2026-11-10): 40 questions that named their filing. Round 3 (2026-11-11): 40 harder questions that did not. Up to 160 marks per round.
MeasureSignal8 OctoClaudeChatGPT
Rank, both rounds#1#2#3
Round 2 score160 / 160160 / 160159 / 160
Round 3 score156 / 160156 / 160145 / 160
Both rounds316 / 320316 / 320304 / 320
Wrong figures (80 questions)002
Answers with 4 of 476 of 8076 of 8072 of 80
Linked the original SEC filing77 of 8063 of 8050 of 80
Linked another website0 of 8059 of 8012 of 80
Other websites only0 of 8017 of 806 of 80

How the tie is broken. Octo and Claude are level on score and on wrong figures in both rounds, so first place comes from Signal8's published tie-breaks, set after grading: in round 2 Signal8's strict re-score of calls the graders flagged as lenient (Octo 159, Claude about 155); in round 3 and over both rounds, answers that linked the original SEC filing (Octo 77, Claude 63). We do not claim Octo outscored Claude. Rules and every grade: round 2, round 3.

Capability round: can each one do the job?

The capability round on November 12, 2026 asked for results, not facts: run a backtest, screen the market, pull a structured history, run an event study, chain several lookups. Each tool was tested as shipped, with no follow-up. Claude and ChatGPT can write and run code but have no market database; Octo has one, plus a backtester. Graded blind: done 2, partial 1, not done 0, invented -1.

Capability round, November 12, 2026: 20 tasks, up to 40 points. Rank among the five tools with a complete run.
MeasureSignal8 OctoClaudeChatGPT
Rank of 5#2#1#3
Points28 / 4034 / 4026 / 40
Done91410
Partial1066
Not done104
Invented000
Backtests (5)565
Market screens (4)777
Lookups (4)774
Event studies (3)464
Multi-step (4)586

Claude's lead over Octo comes from the event studies (6 points to 4) and multi-step jobs (8 to 5), plus a point on backtests; the two are level on screens and lookups. Octo's lead over ChatGPT comes from the per-ticker lookups (7 points to 4); the two are level on backtests, screens and event studies, and ChatGPT is a point ahead on multi-step jobs. They got there differently: ChatGPT matched the reference on the SPY and QQQ backtests (C-01, C-02) but did not run the three market-wide ones, while Octo ran all five on real data, including delisted stocks, and its 50-position cap and missing 200-day filter left every one partial. In its first run, with network access off on our account, Claude wrote and tested backtest code for C-01 and C-03 to C-05 without real data to run it on; those four stayed partial, because only its 9 zero-point tasks were re-run. In the re-run it did the QQQ backtest (C-02) on real prices and matched the reference. Graders were Claude instances. Every task and grade.

Where the answer came from

Each answer's links were captured from the tool's page and sorted: the filing itself on sec.gov (or Signal8's viewer, which opens it by accession number), or another website such as a news story, an aggregator like Stock Titan, or a company page. A right figure that rests on a news page is harder to check than one that links the filing. ChatGPT's page showed no link at all on 24 of its 80 answers.

Where the answer came from, blind rounds 2 and 3 (November 10 and 11, 2026). Counted from the links captured from each tool's page on every graded answer. "Original SEC filing" is a link to the filing on sec.gov's EDGAR archive or to Signal8's filing viewer, which opens the filing by its accession number. "Other website" is any other site: news, data aggregators such as Stock Titan, and company web pages. A filing named only in text, with no link, counts as neither.
ToolAnswers countedLinked the original SEC filingLinked another websiteOther websites onlyRound 2 / round 3: SEC filing linked
Signal8 Octo
Made by Signal8, which runs this lab
8077 of 800 of 800 of 8038 of 40 / 39 of 40
Claude8063 of 8059 of 8017 of 8029 of 40 / 34 of 40
ChatGPT8050 of 8012 of 806 of 8031 of 40 / 19 of 40
AskEdgar Studio8026 of 801 of 800 of 8016 of 40 / 10 of 40
Perplexity7951 of 7927 of 7916 of 7924 of 40 / 27 of 39
GeminiNot measuredNot measuredNot measuredNot measuredNot measured

Gemini is not measured: its links could not be captured from its page, so we report no figure rather than a zero. Perplexity counts 79 answers because one round 3 answer was not captured. Linking another website is not wrong in itself; it means part of the answer rests on a page that is not the filing. Per-answer codes: citation-sources.json.

Where they lost marks

What each is built for

What each tool is built for. Octo's features as offered on signal8.ai; Claude's and ChatGPT's as used in our tests and on their own pricing pages, read Nov 9, 2026.
ToolBuilt forData it works from
Signal8 OctoStock research on US companies: filing questions, backtests in Octo Lab, dilution screensSignal8's database: SEC filings, dilution data, trading halts, borrow rates and insider trades
ClaudeA general assistant: writing, analysis and codeWeb search and files you upload, in the setup we tested; a market database has to be connected or supplied by you
ChatGPTA general assistant: writing, analysis, deep research and codeWeb search and files you upload, in the setup we tested; a market database has to be connected or supplied by you

So the choice depends on the job. For anything outside markets, Claude and ChatGPT are the wider tools. For a filing fact about a US small cap, or a screen or backtest on market data, Octo starts from the data rather than from a web search.

Plans and run conditions

Octo ran on signal8.ai/octo in production on a Signal8 staff account. Claude ran on claude.ai with web search on; in round 2 it ran as Opus 5.5 with Signal8's own connectors switched off, and none of its round 3 answers used Signal8 data. ChatGPT ran on its free plan. Claude's plans run from Free at $0 to Pro at $20 a month and up; ChatGPT's from Free to Go, Plus and Pro (prices on the Claude vs ChatGPT page). Each question went verbatim to a fresh chat, and answers were graded blind by Claude instances, the same model family as one contestant, against keys sealed before the run.

How to check any of the three yourself

  1. Ask for the filing's form, date and accession number, then open it on EDGAR.
  2. If the answer links a news or aggregator page, find the same figure in the filing before relying on it.
  3. Check for a later filing that changes the figure, and for any reverse split since.

Related: Claude vs ChatGPT for stock research, all AI research tools, and the round 4 protocol, which adds a mark for citing the original filing.