Owned by Signal8 Inc., maker of Octo and Signal8 Pro. Not investment advice. About this site
stockairesearch / lab

Signal8 Octo for stock research: blind test results

By , Signal8 Research. Published by Signal8 Inc.

Blind tests 2026-11-10, 2026-11-11 and November 12, 2026 · joint top score on filing facts, first on the tie-break · second in the capability round · Signal8 makes Octo and runs this lab

Signal8 Octo has the joint top score, with Claude, in both of our blind rounds on SEC filing facts, and ranks first there on our published tie-break. In the capability round, 20 research jobs that need real data or computation, it ranked second with 28 of 40, behind Claude (34) and ahead of ChatGPT (26). On SEC filing facts, in round 2 (40 questions that named their filing) it scored 160 of 160; in the harder round 3 (40 questions that did not) 156 of 160. That is 316 of 320 with zero wrong figures in 80 questions. Claude scored the same in both rounds and ranks second on our published tie-breaks. Octo cited the SEC filing itself on 77 of 80 answers and never relied on a third-party site. Signal8 makes Octo and publishes this site, so we set out below exactly how the tests were run and what they do not show.

Our top pick for SEC filing facts · joint top score in both blind filing rounds

Why we point filing questions to Octo

Try Signal8 Octo

What it is

Octo is Signal8's research assistant for US stocks. You ask a question in plain English, such as how many warrants a company has outstanding or when a reverse split takes effect, and it reads Signal8's archive of SEC filings, answers, and cites the filing behind each figure with a link to the document. It is aimed at small and micro-cap companies, where the answer usually sits in a 424B5, an 8-K, a 10-Q footnote or a Form 4 rather than in a data feed. Octo is included in Signal8 Pro at signal8.ai/octo.

Three blind tests: the overall leaderboard

Three blind tests, November 10 to 12, 2026. Rounds 2 and 3 score filing facts (4 marks per question); the capability round scores whether a tool can do a research job with real data (2 points per task). Different scales, so ranks are shown side by side and never added up.
ToolRound 2: filing named (Nov 10)Round 3: filing not named (Nov 11)Capability: 20 research jobs (Nov 12)
Signal8 Octo
Made by Signal8.
#1 · 160 / 160#1 · 156 / 160#2 · 28 / 40
Claude#2 · 160 / 160#2 · 156 / 160#1 · 34 / 40
ChatGPT#3 · 159 / 160#3 · 145 / 160#3 · 26 / 40
AskEdgar Studio#4 · 158 / 160#4 · 132 / 160#5 · 18 / 40
Perplexity#5 · 140 / 160Not ranked · 137 on 39 of 40 answered#4 · 19 / 40
Gemini#6 · 125 / 160Not ranked · 80 on 28 of 40 answeredNot ranked · 11 of 20 answered, not graded

In rounds 2 and 3 Octo and Claude scored the same; Octo is first on Signal8's published tie-breaks, chosen after grading, not on a higher score. In the capability round Claude is first with 34 points, Octo second with 28 and ChatGPT's free plan third with 26. Claude's capability score was corrected on November 13, 2026: its first run scored 16 because of a harness fault on our side (correction). Rules and grades: round 2, round 3, capability round.

Capability round (November 12, 2026): can it do the job?

The capability round gave every tool 20 research jobs, sent verbatim with no follow-up: 5 backtests, 4 market screens, 4 per-ticker lookups, 3 event studies and 4 multi-step jobs. Octo answers from Signal8's market database and has a backtester; Claude and ChatGPT can run code but ship with no market database. Octo scored 28 of 40 and ranked second: 9 done, 10 partial, 1 not done, none invented. Claude ranked first with 34 after a corrected run (correction).

Capability round, run November 12, 2026. 20 research jobs, 2 points for done, 1 for partial, 0 for not done, minus 1 for an invented result; up to 40 points.
RankTool and run conditionsPointsDonePartialNot doneInvented
#1Claude
Corrected November 13, 2026: first run 16 of 40 after a harness fault on our side, 9 tasks re-run. Correction.
claude.ai on a Max plan, Opus 5.5 with web search, no connectors; can run code, no market database. Nine tasks re-run with Auto approval and network access on
34 / 4014600
#2Signal8 Octo
Made by Signal8, which runs this lab.
signal8.ai/octo in production on a Signal8 staff account; purpose-built market database and backtester
28 / 4091010
#3ChatGPT
Free plan, signed in, default model; can run code, no market database
26 / 4010640
#4Perplexity
Free plan, default mode
19 / 4031340
#5AskEdgar Studio
Studio agent on a paid account (Studio is in AskEdgar Pro, listed at $59 a month)
18 / 406680
Not rankedGemini
Free tier; answered 11 of 20 before its free usage limit. Incomplete, so not graded.
Not graded----

Where Octo fell short. Its backtester caps a run at 50 open positions, so the three market-wide backtests that asked for every signal as a separate trade (C-03 to C-05) scored partial. It has no 200-day moving average filter (C-02, partial). Its borrow tool does not return a value for an arbitrary past date (C-11, partial). It has no market-wide halt ranking, so C-17 scored zero; only Claude's re-run did C-17. On C-01 its average return per trade (+0.6%) was outside the reference's tolerance (+0.21%), which one other tool matched exactly; we reviewed the reference and kept it. Every task and grade.

What we measured in the blind round

We asked Octo on signal8.ai each question verbatim in a fresh chat, using the production version that any admitted account uses. Answers from every tool were stripped of tool names and graded by eight fresh Claude instances against answer keys built from the primary filings and sealed, with a published hash, before any tool ran. Figure correct 40 of 40, cites a source 40, source contains it 40, as-of date right 40. 40 of 40 answers scored 4 of 4.

Every answer it gave in the blind round scored 4 of 4, so there is no lost mark to list.

Blind round 3: harder questions, 2026-11-11

Round 3's questions did not name the filing: late filers, superseded agreements, several reverse splits to count, amended filings and Form 4s to add up. Octo scored 156 of 160 with every figure right. Its four lost marks were all "source has it" on split-count questions, where the filings it cited did not cover every split; a fix so that it cites each split's own 8-K is in progress at Signal8.

Tie-break used: Octo and Claude both scored 156 of 160 in round 3 with no wrong figure, and 316 of 320 over both blind rounds. No strict re-score was made for round 3, so the tie is broken by the two rounds' combined record: wrong figures across all 80 questions (0 each, still level), then answers that linked the original SEC filing (Octo 77 of 80, Claude 63 of 80). This is Signal8's rule, chosen after round 3 was graded; Octo did not outscore Claude.

Blind round 3, run 2026-11-11. 40 questions, up to 4 marks each. Tools that answered all 40 are ranked by score, then fewer wrong figures, then the combined two-round record.
RankTool and run conditionsAnsweredFigure correctCites a sourceSource has itAs-of rightScore4 of 4Wrong figures
#1Signal8 Octo
Ranked first: top score, zero wrong figures, first on the tie-break. Made by Signal8.
signal8.ai/octo in production, on a Signal8 staff account
40 of 4040403640156 / 160360
#2Claude
claude.ai with web search on; none of its answers used Signal8, DilutionScan or SplitScanner data
40 of 4040403640156 / 160360
#3ChatGPT
Free plan, signed in, default model, web search
40 of 4038353438145 / 160332
#4AskEdgar Studio
Studio agent on a paid account (Studio is in AskEdgar Pro, listed at $59 a month)
40 of 4035312739132 / 160275 (1 stale)
Not rankedPerplexity
Free plan on a Signal8 team account; one answer (R3-14) was not captured
39 of 4034372838137 / 156285 (1 stale)
Not rankedGemini
Free tier; 12 questions were blocked by its free usage limit
28 of 402317132780 / 112135
Signal8 Octo, blind round 3 2026-11-11: answers that lost a mark
QuestionCompany and topicMarks of 4Blind grader's note
R3-02XXII, Reverse splits35 splits, product 16x135x23x15x20 = 1-for-14,904,000, excludes Nov 2026 1-for-58; cites only 0001493152-26-049743 (trap 8-K, states only 1-for-58) and 0001493152-26-036144 (WebFetch: Aug 2026 8-K approving a future split range, none of the five ratios), so cited sources do not state the figures.
R3-07JAGX, Reverse splits34 splits, 1-for-60/25/35/15 and 787,500, all correct. Cites key 8-K 0001193125-26-391415 (1-for-15), 8-K 0001193125-26-292677 (checked: mentions only 1-for-35) and 8-K 0001193125-26-451933 (checked: no 60/25 ratios). No cited document states the 1-for-60 or 1-for-25 splits.
R3-12IVF, Reverse splits34 splits, 1-for-1,440, correctly excludes the Oct 30, 2026 split; cites key 8-Ks for splits 3 and 4 only. Splits 1 and 2 rest on an uncited tool record, so the cited documents do not support the full count or factor.
R3-27SLXN, Reverse splits3Gave 3 splits 1-for-9/15/10, cumulative 1,350, separated the out-of-window 1-for-24; cited accessions -002393 (May 5, 2026 meeting 8-K, mentions only 1-for-10), -004849 (Oct 26, 2026, 1-for-24 only) and the -005089 trap; I fetched both non-key docs and neither states the prior splits or 1,350.

Where Octo's answers came from

Octo cited the SEC filing itself on 77 of 80 answers and never relied on a third-party site. The three answers without a filing link had no link captured from the page.

Where the answer came from, blind rounds 2 and 3 (November 10 and 11, 2026). Counted from the links captured from each tool's page on every graded answer. "Original SEC filing" is a link to the filing on sec.gov's EDGAR archive or to Signal8's filing viewer, which opens the filing by its accession number. "Other website" is any other site: news, data aggregators such as Stock Titan, and company web pages. A filing named only in text, with no link, counts as neither.
ToolAnswers countedLinked the original SEC filingLinked another websiteOther websites onlyRound 2 / round 3: SEC filing linked
Signal8 Octo
Made by Signal8, which runs this lab
8077 of 800 of 800 of 8038 of 40 / 39 of 40
Claude8063 of 8059 of 8017 of 8029 of 40 / 34 of 40
ChatGPT8050 of 8012 of 806 of 8031 of 40 / 19 of 40
AskEdgar Studio8026 of 801 of 800 of 8016 of 40 / 10 of 40
Perplexity7951 of 7927 of 7916 of 7924 of 40 / 27 of 39
GeminiNot measuredNot measuredNot measuredNot measuredNot measured

Gemini is not measured: its links could not be captured from its page, so we report no figure rather than a zero. Perplexity counts 79 answers because one round 3 answer was not captured. Linking another website is not wrong in itself; it means part of the answer rests on a page that is not the filing. Per-answer codes: citation-sources.json.

Both rounds combined

Both blind rounds combined: round 2 (November 10, 2026, each question named its filing) and round 3 (November 11, harder, filing not named). 80 questions, up to 320 marks. Ranked by total, then fewer wrong figures, then answers that linked the original SEC filing.
RankToolRound 2Round 3Both roundsWrong figuresLinked the SEC filing
#1Signal8 Octo
#1 in both blind rounds. Made by Signal8.
160156316 / 320077 of 80 (96%)
#2Claude160156316 / 320063 of 80 (79%)
#3ChatGPT159145304 / 320250 of 80 (63%)
#4AskEdgar Studio158132290 / 320626 of 80 (33%)
Not rankedPerplexity
Incomplete: answered 79 of 80
140137277 / 316651 of 79 (65%)
Not rankedGemini
Incomplete: answered 68 of 80
12580205 / 2729Not measured

Octo and Claude are level on score (316 of 320) and on wrong figures (0); first place goes to Octo on the third criterion, linking the original filing. That order is Signal8's published choice, not a higher score. Perplexity and Gemini did not answer every question on their free plans, so they are listed, not ranked.

Side by side with the two general assistants: Claude vs ChatGPT vs Signal8 Octo. Next round's rules, published first: round 4 protocol.

Round 2: how the six tools rank

How we rank: by blind score; equal scores are separated by (1) fewer wrong figures, then (2) the strict-reading score. Octo and Claude both scored 160 of 160 with no wrong figure, so the second tie-break decides first place: Octo 159, Claude about 155. The strict reading is Signal8's own re-score of the calls the blind graders flagged as lenient, not a mark the blind graders gave, and we set this rule on November 11, after grading. Round one's rule, fixed in advance on the methodology page, would report totals within 2 marks as level.

Blind round, run 2026-11-10. 40 questions, up to 4 marks each. Ranked by score, then fewest wrong figures, then the strict reading.
RankTool and run conditionsAnsweredFigure correctCites a sourceSource has itAs-of rightScore4 of 4Wrong figureNo figure given
#1Signal8 Octo
Ranked first: top score, zero wrong figures, first on the tie-break. Made by Signal8.
signal8.ai/octo in production, on a Signal8 staff account
40 of 4040404040160 / 1604000
#2Claude
Claude Opus 5.5 with web search switched on; Signal8, DilutionScan and SplitScanner connectors switched off
40 of 4040404040160 / 1604000
#3ChatGPT
Free plan, default model; it hit its file and analysis allowance at question 6 and carried on text-only with web search
40 of 4040403940159 / 1603900
#4AskEdgar Studio
Studio agent on a paid account (Studio is in AskEdgar Pro, listed at $59 a month)
40 of 4039403940158 / 1603910
#5Perplexity
Free plan on two accounts: the first reached its free search limit, so R2-21 to R2-40 were asked on a second free account and graded blind one answer per packet
40 of 4039333038140 / 1602910
#6Gemini
Default model, in the same signed-in browser as the pilot (free tier there)
40 of 4033302636125 / 1602543

On the strict reading Octo has 159, ChatGPT 158, Claude about 155 and AskEdgar Studio about 154; it was not computed for Perplexity or Gemini, whose ranks do not depend on it.

Blind round caveats: every question named its filing, so the top four tools finished within 2 marks; the graders were Claude instances, the same model family as one contestant; plans differed by tool. Details on the blind round page.

What these tests do not show

The capability round measures whether a tool delivers a computed result as shipped, with no follow-up. Claude scored 6 points more than Octo there, after we corrected a harness fault in Claude's first run. Round 2 named the filing in every question; round 3 did not, and the gap between the top two and the rest widened (ChatGPT 145, AskEdgar Studio 132). Octo's first place over Claude rests on our tie-breaks, not on a higher blind score, so we do not call Octo more accurate than Claude. What the data does show is how each answer was sourced: Octo's links went to the filing, Claude's often to other websites.

Earlier results

In the pilot on 2026-11-09, which was not blind, Octo scored 62 of 64, one mark behind AskEdgar Studio at 63. In Signal8's own 33-question internal test, scored by Signal8 with the scorer knowing the tool, Octo won 30 of 33 questions against AskEdgar Studio (details). We list both because a lab run by Octo's maker should show the rounds Octo did not win.

How to check an Octo answer against the filing

  1. Open the filing link under the figure; it goes to the exact SEC filing by accession number.
  2. Find the sentence that states the figure and check the date it is "as of".
  3. Check the company's EDGAR page for any later filing that changes the figure.