Owned by Signal8 Inc., maker of Octo and Signal8 Pro. Not investment advice. About this site
stockairesearch / lab

Signal8 Octo vs AskEdgar Studio: Octo ranked #1 at 160 of 160, AskEdgar Studio #4 at 158 in a blind test

By , Signal8 Research. Published by Signal8 Inc.

Blind round 2026-11-10 · pilot Nov 9 · Signal8 internal benchmark Nov 2 to 4

In our blind round of 40 SEC filing questions, Signal8's Octo ranked #1 of 6 with 160 of 160 and no wrong figure; AskEdgar Studio ranked #4 with 158, behind ChatGPT's free plan (159), and was the only one of the top four to give a wrong figure. Signal8 runs this lab and makes Octo.

Our top pick · ranked #1

Octo, Signal8's AI research assistant

Why Octo ranks #1: the top blind score, 160 of 160; zero wrong figures; first on the published tie-break; and it cited the SEC filing for every answer, with the grader confirming the filing states the figure (40 of 40).

Ask Octo on Signal8

Where AskEdgar Studio fell short in the blind round

AskEdgar Studio in blind round 3

Both blind rounds together: Octo 316 of 320 with no wrong figure, AskEdgar Studio 290 with 6. Round 3, every question and grade.

Three tests, newest first

Octo against AskEdgar Studio. Different rubrics, graders and questions; shown side by side, never combined.
TestDateQuestionsGraded byOctoAskEdgar Studio
Blind round2026-11-1040Eight AI graders (Claude instances), no tool names shown, keys sealed first160 of 160 (#1 of 6)158 of 160 (#4 of 6)
PilotNov 9, 202616One Signal8 grader who knew the tools62 of 6463 of 64
Internal benchmark (below)Nov 2 to 4, 202633Signal8, scorer knew the tool, scorecard out of 30Mean 28.4Mean 23.5

The blind round's caveats apply here too. Every question named its filing, so the test rewards finding and citing a named document, and the top four tools were within 2 marks. The graders were Claude instances. AskEdgar Studio ran on a paid account and Octo on Signal8's production service with a staff account. Every grade is in the blind round CSV.

History, and where AskEdgar Studio did better: it led the earlier, non-blind pilot by one mark (63 to 62), where Octo declined to total one company's warrants, and in the internal test below its first-run answers were faster and easier to skim.

Signal8's internal benchmark, 33 questions

In Signal8's internal benchmark of 33 SEC filing questions, Octo scored a mean of 28.4 out of 30 on production data on November 4, 2026, against 23.5 for AskEdgar Studio, winning 30 of 33 questions, losing 1 and tying 2 on the six-axis scorecard. Signal8 ran the benchmark and makes Octo, so read the limits below before the numbers.

Source: Signal8's three benchmark reports of November 2, 3 and 4, 2026, which are kept in Signal8's research-chat documentation. Figures on this page were recomputed from the per-question totals; the per-question table and a CSV are below. Checked Nov 9, 2026.

What was compared

Results by run

Mean score out of 30 over 33 questions, by run
RunDateScorecardResponse rubricNote
AskEdgar StudioNov 223.524.2Pro plan; not re-run. Zero broken answers; median 17.5 s to completion.
Octo, first runNov 2 (dev)24.122.7Run beside Studio the same afternoon. Three answers were broken, median 36.9 s. Studio led on the response rubric (24.2 against 22.7) and on speed.
Octo, re-runNov 2 (dev)26.225.8After writing, speed and tool changes. Median 24.5 s.
Octo, re-runNov 3 (dev)27.727.7Median 18.8 s.
Octo, productionNov 4 (prod data)28.428.3Median 17.3 s, no answer over 60 s, 33 of 33 answers delivered.

The 28.4 against 23.5 comparison is the November 4 Octo run against the November 2 Studio answers. Against those, Octo won 30 questions on the scorecard (1 loss, 2 ties) and 32 on the response rubric (0 losses, 1 tie). The same-afternoon first run on November 2 is the only like-for-like timing, and in it AskEdgar Studio was faster and tidier.

Where AskEdgar Studio did better

Errors found

Octo's, as reported on November 4 (production). The report lists five factual errors in Octo's answers, two of them from one data defect, plus a data gap and a display bug:

AskEdgar Studio's, as reported on November 2. Four wrong figures: IPW shares outstanding (532,917 against 596,808), SINT ATM remaining ($5.62M against $5.1M), "62 halts and none currently active" when four stocks were halted, and an MDCX equity line shown as live after it ended on September 11. Two omissions: a $12.0M upfront payment missing from an MDCX runway estimate, and ONCO's net proceeds of about $0.25M left as "not stated". Octo's first run had two errors of its own of that kind: the same ended MDCX equity line, and development-environment reverse-split rows that disagreed with production.

Method, as documented

Limits of this benchmark

The lab's blind round and pilot use the lab's own four-mark rubric on different question sets; the full protocol is on the round one page.

Every question

Totals out of 30 per question: scorecard and response rubric, Octo on production data (Nov 4) and AskEdgar Studio (Nov 2)
QCategoryQuestionOcto scorecardStudio scorecardOcto responseStudio responseOcto seconds
Q01dilutionWhat is the dilution risk for MDCX? Cover warrants, convertibles, ATM and shelf capacity, with figures, as-of dates and the filings they come from.2824292622.9
Q02dilutionHow much capacity does VNRX have left on its ATM and its shelf, is it limited by the baby shelf rule, and how much has it sold through the ATM this year?2922282222.7
Q03dilutionList every outstanding warrant and convertible note for IPW on the current post-split basis: exercise or conversion price, shares issuable, expiry or maturity, and any floor or reset terms.2525262644.5
Q04dilutionDoes KALA have an active ATM or equity line? How much has it sold under it this year and what is the remaining capacity?2924282517.7
Q05judgmentIs ONCO's most recent financing toxic? Explain the terms that make it so or not.2822282532.0
Q06will they diluteWill SINT need to raise money in the next 6 months? Base it on cash, burn and what financing vehicles are already in place.2924282622.2
Q07overhangWhat registration statement overhang does AIIO have: pending S-1, F-1 or resale registrations, how many shares, and for whom?2728292728.0
Q08cashHow many months of cash runway does VNRX have, as of what date, and what events since the quarter end (ATM sales, raises, payments) change that number?2824272414.5
Q09going concernDoes GNS have a going concern warning in its latest annual report? Quote the exact language and give the filing.3027302417.3
Q10cash compareCompare cash runway for VNRX and INO: cash, quarterly burn, months left and any ATM or equity line capacity.2726282621.5
Q11reverse splitsWhich reverse splits take effect in the next two weeks? Give ticker, ratio, effective date, market cap and float.2622272330.2
Q12reverse splitsWhat is TOPS's reverse split history, and how did the stock perform in the 30 days after each split?3024292614.6
Q13reverse splits (trap: JOB is NYSE American)Will JOB's upcoming 1-for-30 reverse split be enough to regain Nasdaq $1 bid price compliance, and what is its deadline?3024282516.7
Q14compliance screenWhich Nasdaq companies are under the $1 minimum bid price rule with the least time left before delisting?2822292411.9
Q15delistingIs VNRX at risk of NYSE American delisting? Cite the deficiency notices, the deadlines, and where the stock stands against each listing standard today.3025292717.3
Q16haltsWhich stocks were halted today and on what halt code?3014301813.1
Q17haltsWhy is SUPN halted, what is the halt code, and since when?3018302113.7
Q18deficiencyWhich companies received exchange deficiency notices this week? Give the exchange, the rule, and the filing.3026292814.4
Q19insidersFind insider cluster buys in small caps from the last 30 days where 3 or more insiders bought. Give the prices paid, total dollars and links to the Form 4s.2923292514.6
Q20insiders (trap: none in window)Summarize ORGO's Form 4 activity over the last 90 days: who bought or sold, how many shares, at what price, and open-market versus plan or award.2824272218.2
Q21institutionsWhich institutions added or cut the most INO shares last quarter according to 13F filings?2817281920.9
Q22institutionsWho are the top 13F holders of MDCX, and did they add or reduce in the latest quarter?2920282013.4
Q23float/shortWhat is MDCX's float, short interest and days to cover, and when was each figure last reported?3024292512.4
Q24short/borrowWhat is the short and borrow situation on BYND: short interest, days to cover, and borrow availability if you have it?3025292417.2
Q258-KSummarize MDCX's latest 8-K: filing date, the items reported, what happened, and a link to the filing.3028302713.2
Q268-K screenWhich companies filed an 8-K with Item 3.01 (notice of delisting or failure to satisfy a listing rule) in the last 7 days? Give the company, date and the rule cited.3025302713.8
Q27offerings (trap: none since 2022)What were the terms of ONCO's most recent registered direct or public offering: price per share, shares, warrants and their exercise price, placement agent and net proceeds?2625272530.4
Q28screenWhich small caps have under 6 months of cash runway and an active ATM or shelf?2822282318.5
Q29screenWhich biotechs under $50M market cap have less than $5M in cash and a shelf registration filed in the last 12 months?2926282615.8
Q30trap: adviceShould I buy VNRX before its next earnings report?2824292318.7
Q31trap: renamed tickerWhat is the dilution risk for MULN?2622262218.1
Q32trap: no coverageWhat is the dilution risk and cash runway for PGPHF?2525262516.5
Q33trap: ambiguous tickerWhat is the cash runway for PC?2825272317.6

The CSV adds Octo's November 3 development scores and tool-call counts: per-question.csv. Signal8, which runs this lab, makes Octo.