ChatGPT for SEC filing research
Blind round 2026-11-10 · ranked #3 of 6 · Octo ranked #1 · plans checked Nov 9, 2026
ChatGPT on the free plan answered all 40 questions in our blind test of SEC filing questions, scored 159 of 160 with no wrong figure, and ranked #3 of 6. Its one lost mark was R2-40, where it cited a page the grader could not open and named no filing. Signal8's Octo ranked first and Claude second, both on 160, so ChatGPT finished one mark off the top. In the harder round 3, where questions did not name the filing, it scored 145 of 160, ranked #3 of the tools that answered every question, with 2 wrong figures; Octo again ranked first with 156.
What it is, and what it costs
ChatGPT is OpenAI's general chat assistant. Its pricing page lists Free, Go, Plus and Pro for individuals, plus Business and Enterprise (chatgpt.com/pricing, read Nov 9, 2026 from Indonesia, so prices show in rupiah). Free is described as "Unlimited text chats with GPT-6" with "Limited messages with uploads" and "Limited deep research". Go is IDR 75,000 a month and "may include ads". Plus is IDR 349,000 a month and adds "Expanded deep research" and "Projects, scheduled tasks, and custom GPTs". Pro is listed from IDR 1,889,000 a month. Business and Enterprise prices are not on the page.
What we measured in the blind round
We asked ChatGPT's free plan, signed in with the default model, each question verbatim in a fresh chat. At question 6 it reached its file and analysis allowance and carried on text-only with web search; every answer still arrived. The answers were graded blind against keys sealed before the run. Figure correct 40 of 40, cites a source 40, source contains it 39, as-of date right 40.
| Question | Company and topic | Marks of 4 | Blind grader's note |
|---|---|---|---|
| R2-40 | NSPR, Insider purchase (Form 4) | 3 | 21,000 shares at $1.20 on May 12; cites an edgar-atlas listing URL that returned 403 to WebFetch, and no filing date or accession is named, so the source is unverified. |
Blind round caveats: every question named its filing, so the top four tools finished within 2 marks; the graders were Claude instances, the same model family as one contestant; plans differed by tool. Details on the blind round page.
Blind round 3 (2026-11-11): harder questions
Round 3's 40 questions did not name the filing, so each tool had to find the right and latest one. ChatGPT scored 145 of 160 and was ranked #3 of the 4 tools that answered every question; Signal8's Octo ranked first with 156 and no wrong figure. Figure correct 38 of 40, cites a source 35, source contains it 34, as-of date right 38; wrong figures 2. Across rounds 2 and 3 it linked the original SEC filing on 50 of 80 answers, against 77 of 80 for Octo.
| Question | Company and topic | Marks of 4 | Blind grader's note |
|---|---|---|---|
| R3-10 | BFRG, Nasdaq status (8-K) | 3 | February 8, 2027. Cites the 'August 18, 2026 Form 8-K' (the key 8-K's report date; filed Aug 20). Never states the Oct 31, 2026 as-of date the question pins. |
| R3-12 | IVF, Reverse splits | 2 | 4 splits, 1,440:1 (ratio column blank but shown in the calculation); sources are only 'EBS Public Now' and 'SEC', with no URL, accession or dated form. |
| R3-27 | SLXN, Reverse splits | 2 | Gave 3 splits and 9x15x10 = 1,350 for the stated window (ratio column rendered blank but the ratios appear in the formula); no source cited. |
| R3-32 | NXXT, Reverse splits | 2 | Gave 2 splits (Sept 14 and Oct 30, 2026), cumulative 1-for-100; sources are only 'investors.nextnrg.com' and an undated 'SEC Form 8-K', with no URL or accession. |
| R3-35 | HEPA, Insider purchases (Form 4) | 2 | Gave 16,500,000 shares at $0.05 ($825,000) plus 16.5M warrants at $0.06 expiring Aug 3, 2031, but cited only 'SEC Tracker' and a 'Filing Archive' with no URL, accession or dated form. |
| R3-36 | PFSA, Reverse splits | 0 | Gave 308,013 (1-for-4 only), which is wrong. Its only citations are bare XML filenames with no accession, URL or date, and it states no as-of date for the restated figure. |
| R3-40 | INV, Insider purchases (Form 4) | 2 | 536,000 shares and $807,794.10 are right, but it gives the overall range as $1.4602 to $1.59. The key's low is $1.46 (Otworth and Haskell footnotes), so one part is wrong. It cites 'SEC Form 4 filings dated August 21, 2026'. |
Capability round (November 12, 2026): research jobs
The capability round asked each tool to do 20 research jobs that need real data or computation: backtests, market screens, structured lookups, event studies and multi-step research. ChatGPT scored 26 of 40 and ranked #3 of the five complete runs (10 done, 6 partial, 4 not done, none invented). Claude ranked first with 34 and Signal8's Octo second with 28. Every grade.
Earlier: the pilot
In the pilot on 2026-11-09, ChatGPT's free plan answered Q01, Q02, Q03, Q04, Q05, Q06, Q07 before its session logged out, and lost no marks on them: 28 of 28. That answer set was too small to say much on its own; the blind round is the fuller test.
How many shares of preferred stock did Our Bond, Inc. (OBAI) report as outstanding in its most recent Form 10-Q, and as of what date? Cite the filing you used.
Key in the filing: 14,651,864 series preferred shares in stockholders' deficit, plus 821,796 convertible preferred shares in mezzanine equity (15,473,660 combined). Either the 14,651,864 figure with the mezzanine class noted, or the 15,473,660 total, is accepted.. ChatGPT answered (excerpt, verbatim): According to Our Bond, Inc.’s most recent Form 10-Q, filed August 14, 2026, the company reported 14,651,864 shares of preferred stock issued and outstanding as of June 30, 2026. The filing separately reports 821,796 shares of convertible preferred stock classified as mezzanine equity. Source: SEC Form 10-Q, quarter ended June ...
Scored 4 of 4 in the pilot. Full answer in the pilot CSV (dashes shown as hyphens here).
That answer gave both preferred share classes, which the answer key required; AskEdgar Studio reported only the larger class and lost a mark.
Where it can fail on filings
In 47 graded answers across both rounds ChatGPT gave no wrong figure. The one failure is about sourcing: on R2-40 it pointed to a third-party listing page that returned an error and did not name the Form 4 itself, so a reader could not check the figure from its answer. In the pilot, one EDGAR link had a malformed accession folder although the 10-Q it named did contain the figures. The published research is less kind to general chat models with retrieval; see the Claude vs ChatGPT page for the FinanceBench result.
How to check a ChatGPT answer against the filing
- Copy the accession number or form date it gives and search it on EDGAR; do not trust a pasted link until it opens the document.
- For a share count, find the cover page sentence that begins "outstanding as of" and compare the date as well as the number.
- For a warrant or preferred total, find the equity note and add the classes yourself; a total can sit in a different table from the class lines.
- Check there is no later 10-Q or 8-K than the one it used.
Our top pick for SEC filing facts · joint top score in both blind filing rounds
Signal8 Octo, joint top score on SEC filing facts
Why Octo for filing questions: the top score in filing rounds 2 and 3 (316 of 320, shared with Claude, first on the published tie-break), zero wrong figures in 80 questions, and Octo cited the SEC filing itself on 77 of 80 answers and never relied on a third-party site. In the capability round (20 research jobs) it ranked second with 28 of 40, behind Claude (34) and ahead of ChatGPT (26). Claude vs ChatGPT vs Octo, side by side.
ChatGPT starts from the open web; Signal8 Octo starts from Signal8's archive of SEC filings and answers from that, citing the filing. For a filing fact about a US small cap, such as a share count, a warrant total or a listing notice, Octo's design is aimed at it. For everything outside filings, ChatGPT remains the wider tool.
Signal8 Octo vs ChatGPT: what we measured
- Ranking: Octo ranked first of 6 in the blind round; ChatGPT ranked #3. The rule and every score.
- Blind round: Octo 160 of 160, ChatGPT 159. Both gave no wrong figure; ChatGPT lost one source mark.
- Pilot: on the 7 questions both answered, ChatGPT 28 of 28 and Octo 26, because Octo declined to total one company's warrants.
Where ChatGPT is better, or who should pick it: it did this on a free plan, one mark off the top, and it is the wider tool outside filings.
Signal8 makes Octo and runs this lab, so read its results beside the other rows. Octo and Claude scored the same in both filing-accuracy rounds; Octo is first there on Signal8's published tie-breaks, set after grading (round 2: Signal8's strict re-score of flagged calls; round 3: the two rounds' record of linking the original filing). In the earlier, non-blind pilot it finished one mark behind AskEdgar Studio (62 against 63 of 64). Octo is included in Signal8 Pro at signal8.ai/octo.
See Signal8 Octo