AI tools vs SEC filings: AskEdgar 63, Octo 62 of 64 in the pilot; Octo won 30 of 33 in a larger internal test
PILOT ROUND · run 2026-11-09 · 16 questions · superseded by the blind round of Nov 10
In our pilot test of AI tools on SEC filings, AskEdgar Studio scored 63 of 64 marks and Octo, Signal8's AI research assistant, 62, on all 16 questions each: a one-mark gap, which is one question. Gemini scored 54 of 64 with 3 wrong figures; ChatGPT and Perplexity answered 7 and 9 of 16 on their free plans.
Newer result: in the blind round a day later (40 questions, sealed keys, graders who saw no tool names), Octo ranked #1 of 6: Octo and Claude scored 160 of 160 and Octo is first on the published tie-break, then ChatGPT 159, AskEdgar Studio 158, Perplexity 140 and Gemini 125. This page is kept as the earlier, non-blind round.
Each question is answered in a specific SEC filing, and every answer was graded against that filing. On the questions they answered, ChatGPT and Perplexity lost no marks. Octo's one miss was declining to total Arcadia's warrants (Q04); AskEdgar Studio's one miss was giving OBAI's preferred shares without the mezzanine class the answer key requires (Q07). Signal8 runs this lab and makes Octo.
Our pick
Octo, Signal8's AI research assistant
Octo finished one mark behind AskEdgar Studio (62 against 63 of 64), so the ranking table puts AskEdgar first on the stated criteria. Octo's record: 15 of 16 answers perfect, a source cited on 16 of 16, the right as-of date on 16 of 16, and no wrong figure.
Ask Octo on Signal8| Tool | Answers graded | Figure correct | Cites a source | Source contains it | As-of date right | Score | Wrong figure | No figure given |
|---|---|---|---|---|---|---|---|---|
| Octo Signal8 admin account with full access, signal8.ai/octo | 16 of 16 | 15 | 16 | 15 | 16 | 62 / 64 (97%) | 0 | 1 |
| ChatGPT Free plan, logged in, default model | 7 of 16 | 7 | 7 | 7 | 7 | 28 / 28 (100%) | 0 | 0 |
| Perplexity Free plan, logged in, default Search mode | 9 of 16 | 9 | 9 | 9 | 9 | 36 / 36 (100%) | 0 | 0 |
| Gemini Free tier, logged in, default model (Auto) | 16 of 16 | 12 | 16 | 12 | 14 | 54 / 64 (84%) | 3 | 1 |
| AskEdgar Studio agent "Edgar", effort Medium, on a paid Pro account (founder's signed-in session); published list price $59 a month (askedgar.io/pricing, checked Nov 9) | 16 of 16 | 15 | 16 | 16 | 16 | 63 / 64 (98%) | 1 | 0 |
| Claude Not run: the account was at its message limit | 0 of 16 | Not tested in this round | ||||||
ChatGPT and Perplexity did not answer every question, so their percentages are not comparable with the others on their own. On the same questions only:
- On the 7 questions ChatGPT answered: ChatGPT 28 of 28, Octo 26 of 28, Gemini 21 of 28, AskEdgar 27 of 28.
- On the 9 questions Perplexity answered: Perplexity 36 of 36, Octo 34 of 36, Gemini 31 of 36, AskEdgar 36 of 36.
Raw data: every question, every verbatim answer and every grade as a CSV. Scores are counts of yes marks; the four marks are defined in the methodology.
What the pilot found
- Octo: 62 of 64 marks on 16 answers; 15 answers perfect; 0 with a wrong figure; 1 with no figure given.
- ChatGPT: 28 of 28 marks on 7 answers; 7 answers perfect; 0 with a wrong figure; 0 with no figure given. 9 questions not run.
- Perplexity: 36 of 36 marks on 9 answers; 9 answers perfect; 0 with a wrong figure; 0 with no figure given. 7 questions not run.
- Gemini: 54 of 64 marks on 16 answers; 12 answers perfect; 3 with a wrong figure; 1 with no figure given.
- AskEdgar: 63 of 64 marks on 16 answers; 15 answers perfect; 1 with a wrong figure; 0 with no figure given.
Octo's misses. Q04 (RKDA): DECLINED: listed the warrant and option series but said it could not confirm a single total; no total given.
Every miss, by tool.
- Gemini, Q01 (ARAI, Share count): STALE: gave 34,213,387 shares from the 10-Q for the quarter ended September 30, 2025, a year older than the latest 10-Q.
- Octo, Q04 (RKDA, Warrants): DECLINED: listed the warrant and option series but said it could not confirm a single total; no total given.
- Gemini, Q04 (RKDA, Warrants): DECLINED: described warrant activity but gave no outstanding count.
- Gemini, Q07 (OBAI, Preferred stock): WRONG: said 0 preferred shares outstanding.
- AskEdgar, Q07 (OBAI, Preferred stock): Gives 14,651,864 preferred shares in stockholders' deficit but not the 821,796 convertible preferred shares in mezzanine equity that the answer key requires. The 10-Q contains the figure.
- Gemini, Q13 (PTIX, Insider purchases): WRONG: reported two purchases totalling 36,634 shares; missed the August 11 and August 19 purchases.
How this pilot differs from round one
- Question draw. Companies were drawn with a fixed random seed from Signal8's own dilution coverage list (plus EDGAR full-text search for splits, deficiency notices, ATMs and shelves), not from all recent 10-Q filers. Octo answers from Signal8's data, so this draw may favour it.
- Plans. Free or already-logged-in accounts in one browser. No paid plan was bought. Octo ran on a Signal8 staff account with full access. AskEdgar ran in its Studio agent ("Edgar", effort Medium) on a paid Pro account that was already open in the browser (published list price $59 a month, askedgar.io/pricing, checked Nov 9); its answers were collected the same day, after the other tools'.
- Grading. One grader, who knew which tool wrote each answer. The later blind round was graded blind, and so will round one be.
- Coverage. 16 questions, two per category. ChatGPT: 9 not run; Perplexity: 7 not run; Claude: 16 not run. The reasons are in the table below.
Tools not fully tested
- ChatGPT: Not run: the ChatGPT session logged out during the run (after question 7); we do not log in or out of accounts during a test.
- Perplexity: Not run: no usable answer was captured. On Q07 to Q09 the page showed no answer or Perplexity's Computer welcome panel; from Q13 on, Perplexity showed its free search limit.
- Claude: Not run: the account was at its Claude message limit on the run date.
- Fiscal.ai: not tested in the pilot (no account in the test browser). It stays in round one.
Every question and every grade
Q01 · Share count · ARAI
How many shares of common stock did Arrive AI Inc. (ARAI) report as outstanding on the cover page of its most recent Form 10-Q, and as of what date? Cite the filing you used.
Correct answer: 51,876,209 shares. As of: August 5, 2026. Source: 10-Q for the quarter ended June 30, 2026 (filed Aug 13, 2026), accession 0001493152-26-037755.
| Tool | Figure | Cites | Source has it | As-of | Note |
|---|---|---|---|---|---|
| Octo | yes | yes | yes | yes | |
| ChatGPT | yes | yes | yes | yes | |
| Perplexity | yes | yes | yes | yes | |
| Gemini | no | yes | no | no | STALE: gave 34,213,387 shares from the 10-Q for the quarter ended September 30, 2025, a year older than the latest 10-Q. |
| AskEdgar | yes | yes | yes | yes | |
| Claude | Not run | ||||
Q02 · Share count · TRNR
How many shares of common stock did Interactive Strength, Inc. (TRNR) report as outstanding on the cover page of its most recent Form 10-Q, and as of what date? Cite the filing you used.
Correct answer: 1,619,702 shares. As of: August 12, 2026 (cover page text; the XBRL tag says August 11). Source: 10-Q for the quarter ended June 30, 2026 (filed Aug 14, 2026), accession 0001193125-26-352477.
| Tool | Figure | Cites | Source has it | As-of | Note |
|---|---|---|---|---|---|
| Octo | yes | yes | yes | yes | |
| ChatGPT | yes | yes | yes | yes | |
| Perplexity | yes | yes | yes | yes | |
| Gemini | yes | yes | yes | yes | Says the 10-Q was filed August 14 in one place; cites ADVFN for the filing date and the accession number. |
| AskEdgar | yes | yes | yes | yes | |
| Claude | Not run | ||||
Q03 · Warrants · NWBO
How many warrants did Northwest Biotherapeutics (NWBO) report as outstanding in its most recent Form 10-Q, and as of what date? Cite the filing you used.
Correct answer: about 91.2 million warrants (91,242 thousand), weighted average exercise price $0.27. As of: June 30, 2026. Source: 10-Q for the quarter ended June 30, 2026 (filed Aug 14, 2026), accession 0001104659-26-097188.
| Tool | Figure | Cites | Source has it | As-of | Note |
|---|---|---|---|---|---|
| Octo | yes | yes | yes | yes | |
| ChatGPT | yes | yes | yes | yes | |
| Perplexity | yes | yes | yes | yes | |
| Gemini | yes | yes | yes | yes | Primary citation chip is Stock Titan; also gives the accession number. |
| AskEdgar | yes | yes | yes | yes | |
| Claude | Not run | ||||
Q04 · Warrants · RKDA
How many warrants did Arcadia Biosciences (RKDA) report as outstanding in its most recent Form 10-Q, and as of what date? Cite the filing you used.
Correct answer: 3,985,971 warrants. As of: June 30, 2026. Source: 10-Q for the quarter ended June 30, 2026 (filed Aug 13, 2026), accession 0001193125-26-349205.
| Tool | Figure | Cites | Source has it | As-of | Note |
|---|---|---|---|---|---|
| Octo | no | yes | no | yes | DECLINED: listed the warrant and option series but said it could not confirm a single total; no total given. |
| ChatGPT | yes | yes | yes | yes | |
| Perplexity | yes | yes | yes | yes | |
| Gemini | no | yes | no | yes | DECLINED: described warrant activity but gave no outstanding count. |
| AskEdgar | yes | yes | yes | yes | |
| Claude | Not run | ||||
Q05 · ATM programs and shelves · VHC
What is the maximum amount VirnetX Holding Corp (VHC) can sell under its current at-the-market program, and who is the sales agent? Cite the filing you used.
Correct answer: up to $19,135,717 of common stock; sales agent Titan Partners Securities LLC. As of: Sales agreement and 424B5 dated October 1, 2026. Source: 424B5 prospectus supplement filed Oct 1, 2026, accession 0001140361-26-038168.
| Tool | Figure | Cites | Source has it | As-of | Note |
|---|---|---|---|---|---|
| Octo | yes | yes | yes | yes | |
| ChatGPT | yes | yes | yes | yes | Cites the same-day 8-K, which also states the amount. |
| Perplexity | yes | yes | yes | yes | |
| Gemini | yes | yes | yes | yes | |
| AskEdgar | yes | yes | yes | yes | Also reports $18,442,060 remaining from its company profile, noted as unconfirmed against filings. |
| Claude | Not run | ||||
Q06 · ATM programs and shelves · GEMI
In the Form S-3 that Gemini Space Station, Inc. (GEMI) filed in October 2026, what maximum aggregate offering price did it register for the company's own primary offerings? Cite the filing you used.
Correct answer: $100,000,000 primary shelf (the filing also registers 7,142,857 resale shares; total offering amount $138,071,427.81). As of: Filed October 1, 2026. Source: S-3 filed Oct 1, 2026 (filing fee exhibit), accession 0001104659-26-112851.
| Tool | Figure | Cites | Source has it | As-of | Note |
|---|---|---|---|---|---|
| Octo | yes | yes | yes | yes | |
| ChatGPT | yes | yes | yes | yes | |
| Perplexity | yes | yes | yes | yes | |
| Gemini | yes | yes | yes | yes | Citation chips point to third-party sites (SEPT Investing, Stock Titan); also gives the accession number. |
| AskEdgar | yes | yes | yes | yes | States $100,000,000 and says it could not confirm the wording against the filing text, so treats it as unverified. |
| Claude | Not run | ||||
Q07 · Preferred stock · OBAI
How many shares of preferred stock did Our Bond, Inc. (OBAI) report as outstanding in its most recent Form 10-Q, and as of what date? Cite the filing you used.
Correct answer: 14,651,864 series preferred shares in stockholders' deficit, plus 821,796 convertible preferred shares in mezzanine equity (15,473,660 combined). Either the 14,651,864 figure with the mezzanine class noted, or the 15,473,660 total, is accepted.. As of: June 30, 2026. Source: 10-Q for the quarter ended June 30, 2026 (filed Aug 14, 2026), accession 0001493152-26-037989.
| Tool | Figure | Cites | Source has it | As-of | Note |
|---|---|---|---|---|---|
| Octo | yes | yes | yes | yes | Gives both preferred classes and the combined total. |
| ChatGPT | yes | yes | yes | yes | Gives both classes. Its EDGAR link has a malformed accession folder; the named 10-Q contains the figures. |
| Perplexity | Not run | ||||
| Gemini | no | yes | no | yes | WRONG: said 0 preferred shares outstanding. |
| AskEdgar | no | yes | yes | yes | Gives 14,651,864 preferred shares in stockholders' deficit but not the 821,796 convertible preferred shares in mezzanine equity that the answer key requires. The 10-Q contains the figure. |
| Claude | Not run | ||||
Q08 · Preferred stock · BIAF
How many shares of preferred stock did bioAffinity Technologies (BIAF) report as outstanding in its most recent Form 10-Q, and as of what date? Cite the filing you used.
Correct answer: 150 preferred shares. As of: September 30, 2026. Source: 10-Q for the quarter ended September 30, 2026 (filed Nov 6, 2026), accession 0001493152-26-050540.
| Tool | Figure | Cites | Source has it | As-of | Note |
|---|---|---|---|---|---|
| Octo | yes | yes | yes | yes | |
| ChatGPT | Not run | ||||
| Perplexity | Not run | ||||
| Gemini | yes | yes | yes | yes | Cites Quantisnow and Stock Titan; names the 10-Q filed November 6. Adds an unverified prior-quarter figure. |
| AskEdgar | yes | yes | yes | yes | |
| Claude | Not run | ||||
Q09 · Cash · PROK
What were ProKidney Corp.'s (PROK) cash and cash equivalents at the end of its latest reported quarter, and what was the quarter-end date? Cite the filing you used.
Correct answer: $74,868,000 (about $74.9 million) cash and cash equivalents. As of: June 30, 2026. Source: 10-Q for the quarter ended June 30, 2026 (filed Aug 10, 2026), accession 0001193125-26-342373.
| Tool | Figure | Cites | Source has it | As-of | Note |
|---|---|---|---|---|---|
| Octo | yes | yes | yes | yes | Also flags the going-concern language. |
| ChatGPT | Not run | ||||
| Perplexity | Not run | ||||
| Gemini | yes | yes | yes | yes | Cites Seeking Alpha; names the 10-Q filed August 10. |
| AskEdgar | yes | yes | yes | yes | |
| Claude | Not run | ||||
Q10 · Cash · CNTY
What were Century Casinos' (CNTY) cash and cash equivalents at the end of its latest reported quarter, and what was the quarter-end date? Cite the filing you used.
Correct answer: $60,179,000 (about $60.2 million) cash and cash equivalents. As of: June 30, 2026. Source: 10-Q for the quarter ended June 30, 2026 (filed Aug 7, 2026), accession 0000911147-26-000039.
| Tool | Figure | Cites | Source has it | As-of | Note |
|---|---|---|---|---|---|
| Octo | yes | yes | yes | yes | Gives the 10-Q filing date as August 6; EDGAR shows August 7 (not scored). |
| ChatGPT | Not run | ||||
| Perplexity | yes | yes | yes | yes | Cites the company's results release and the 10-Q accession. |
| Gemini | yes | yes | yes | yes | Cites PR Newswire; names the 10-Q. |
| AskEdgar | yes | yes | yes | yes | Gives the 10-Q filing date as August 6; EDGAR shows August 7 (not scored). |
| Claude | Not run | ||||
Q11 · Reverse splits · BYND
What was the ratio and effective date of Beyond Meat's (BYND) most recent reverse stock split? Cite the filing you used.
Correct answer: 1-for-30. As of: Effective 11:59 p.m. ET on August 13, 2026; split-adjusted trading from August 14, 2026 (either date accepted). Source: 8-K filed Aug 14, 2026 (Exhibit 99.1), accession 0001655210-26-000059.
| Tool | Figure | Cites | Source has it | As-of | Note |
|---|---|---|---|---|---|
| Octo | yes | yes | yes | yes | |
| ChatGPT | Not run | ||||
| Perplexity | yes | yes | yes | yes | |
| Gemini | yes | yes | yes | yes | Cites Stock Titan; names the 8-K accession. |
| AskEdgar | yes | yes | yes | yes | |
| Claude | Not run | ||||
Q12 · Reverse splits · BRRN
What was the ratio and effective date of the most recent reverse stock split by Bryn Inc. (BRRN)? Cite the filing you used.
Correct answer: 1-for-100,000 (with a name change to MEDO Technologies, Inc.). As of: Effective 6:00 p.m. PDT on September 23, 2026. Source: 8-K filed Sep 16, 2026, accession 0001213900-26-100354.
| Tool | Figure | Cites | Source has it | As-of | Note |
|---|---|---|---|---|---|
| Octo | yes | yes | yes | yes | Also notes the new ticker MDTK. |
| ChatGPT | Not run | ||||
| Perplexity | yes | yes | yes | yes | Correct split. Adds that Bryn did a 1-for-25 split on April 2, 2026, citing an XBRL page from a different company's filing (Cerenome, CIK 1095981). Not scored and not verified. |
| Gemini | yes | yes | yes | yes | Cites Investing.com and Stock Titan; names the September 16 8-K. |
| AskEdgar | yes | yes | yes | yes | Headline says effective September 24 per an exchange record, and also states the 8-K's 6:00 P.M. PDT September 23 effective time. |
| Claude | Not run | ||||
Q13 · Insider purchases · PTIX
Did any officer or director of Protagenic Therapeutics (PTIX) report an open-market purchase on Form 4 between August 11 and November 8, 2026? If so, who, and how many shares in total? Cite the filings you used.
Correct answer: Yes: Garo H. Armen, Executive Chairman, four open-market purchases totalling 41,554 shares (2,220 + 2,700 + 16,634 + 20,000). As of: Transactions August 11 to August 21, 2026. Source: Form 4s filed Aug 12, Aug 20 (two) and Aug 24, 2026, accession 0001493152-26-039816.
| Tool | Figure | Cites | Source has it | As-of | Note |
|---|---|---|---|---|---|
| Octo | yes | yes | yes | yes | Lists all four Form 4s with accession numbers. |
| ChatGPT | Not run | ||||
| Perplexity | Not run | ||||
| Gemini | no | yes | no | no | WRONG: reported two purchases totalling 36,634 shares; missed the August 11 and August 19 purchases. |
| AskEdgar | yes | yes | yes | yes | |
| Claude | Not run | ||||
Q14 · Insider purchases · AMFN
Did any officer or director of American Fusion, Inc. (AMFN) report an open-market purchase on Form 4 between August 11 and November 8, 2026? If so, who, and how many shares in total? Cite the filings you used.
Correct answer: Yes: Alexander William Rossa (Chief Revenue Officer), 842,775 shares at $0.102. As of: Transaction September 22, 2026. Source: Form 4 filed Sep 29, 2026, accession 0001079973-26-001308.
| Tool | Figure | Cites | Source has it | As-of | Note |
|---|---|---|---|---|---|
| Octo | yes | yes | yes | yes | Notes that the Form 4 ticks no officer box although it gives the title Chief Revenue Officer. |
| ChatGPT | Not run | ||||
| Perplexity | Not run | ||||
| Gemini | yes | yes | yes | yes | Cites Stock Titan and OpenInsider; gives the accession number. |
| AskEdgar | yes | yes | yes | yes | |
| Claude | Not run | ||||
Q15 · Listing compliance · ACH
Has Accendra Health (ACH) disclosed a Nasdaq or NYSE continued listing deficiency notice in the last 12 months? If so, which standard, and by when must it regain compliance? Cite the filing you used.
Correct answer: Yes, two NYSE notices: (1) received October 6, 2026, average closing price below $1.00 over 30 trading days, six months to cure (about April 6, 2027), 8-K filed October 9; (2) received October 27, 2026, Section 802.01B, average market capitalization and stockholders' equity both below $50 million, business plan due in 45 days and up to 18 months to cure, 8-K filed October 30. Either notice, described correctly, is accepted. (Our first answer key listed only notice 1; corrected during grading when two tools cited notice 2, which we then verified on EDGAR.). As of: October 6, 2026 (notice 1) or October 27, 2026 (notice 2). Source: 8-K filed Oct 9, 2026 (Item 3.01), accession 0001104659-26-115043.
| Tool | Figure | Cites | Source has it | As-of | Note |
|---|---|---|---|---|---|
| Octo | yes | yes | yes | yes | Gave both NYSE notices, the only answer that did. |
| ChatGPT | Not run | ||||
| Perplexity | Not run | ||||
| Gemini | yes | yes | yes | yes | Described the October 27 notice (Section 802.01B) only; missed the October 6 price notice. |
| AskEdgar | yes | yes | yes | yes | |
| Claude | Not run | ||||
Q16 · Listing compliance · CLDI
Has Calidi Biotherapeutics (CLDI) disclosed a Nasdaq or NYSE continued listing deficiency notice in the last 12 months? If so, which standard, and by when must it regain compliance? Cite the filing you used.
Correct answer: Yes: NYSE American notice received September 21, 2026 under Company Guide Section 1003(a)(ii) (stockholders' equity); plan due October 21, 2026; regain compliance by March 21, 2028. As of: Notice received September 21, 2026. Source: 8-K filed Sep 25, 2026 (Item 3.01), accession 0001493152-26-044381.
| Tool | Figure | Cites | Source has it | As-of | Note |
|---|---|---|---|---|---|
| Octo | yes | yes | yes | yes | |
| ChatGPT | Not run | ||||
| Perplexity | Not run | ||||
| Gemini | yes | yes | yes | yes | Cites Section 1003(a) without the (ii); citation chips point to Minichart, Stock Titan, Nasdaq and Small Cap Siren; names the September 25 8-K. |
| AskEdgar | yes | yes | yes | yes | |
| Claude | Not run | ||||
Full answer text for every row is in the CSV. Signal8, which runs this lab, makes Octo.