Owned by Signal8 Inc., maker of Octo and Signal8 Pro. Not investment advice. About this site
stockairesearch / lab

About this lab

The Stock AI Research Lab is run by Signal8 Inc., which makes Octo, an AI research assistant for SEC filings, and Signal8 Pro. The lab exists to answer a narrow question with evidence: which AI tools answer SEC-filing questions correctly, and which ones invent answers.

The conflict, and how we handle it

We test our own product against its competitors. To keep that honest: the protocol is published before the run; the companies are drawn at random on the day; grading is blind; and every answer is published, so anyone can re-grade it. The pilot round shows Octo's misses on the home page and the results page, including the round where AskEdgar Studio finished ahead. The blind round was graded by AI instances that saw no tool names, against answer keys sealed with a published hash before any tool ran; its graders are the same model family as one contestant (Claude), and the page says so. Both blind rounds rank Octo first, and in both Octo and Claude scored the same. Round 2's tie is broken by a strict reading that is Signal8's own re-score of flagged calls; round 3's by the two rounds' record of linking the original filing. Both rules were set after grading, and the round pages say so. Round 4's rules, including its tie rule, are published before it runs. The capability round needed no tie rule: Claude ranked first and Octo second. Its first published version ranked Octo first and Claude last, because of a fault in how we ran Claude (an approval setting and network access left off on our account); we re-ran those tasks and corrected the result on November 13, 2026 (correction).

The disclosure

Signal8 sells market data and research tools and is paid by some public companies to publish information about them. Nothing on this site is investment advice, and no question in the test asks for an opinion on a stock.

Why publish before results

Most AI comparisons publish only the winner. Publishing the rules first means the result cannot be shaped after the fact.

Contact

Tool makers who want to suggest a fairer setting for their product before round one can reach Signal8 through signal8.ai.