Owned by Signal8 Inc., maker of Octo and Signal8 Pro. Not investment advice. About this site
stockairesearch / lab

Joshua Williams

Joshua Williams works on Signal8 research. stockairesearch.com is published by Signal8 Inc.

What Joshua reviews here

Joshua is the named author of the benchmark pages, the question set, and the guide on checking ChatGPT answers against SEC filings. For each benchmark question the lab records the correct answer from the company's own filing, with its accession number, and then grades every tool's answer against it using the rubric published on the methodology page.

Joshua's concern on this site is that the grading stays the same for every tool, including Octo, which Signal8 makes. The rubric is written down before answers are graded, every answer is published verbatim in the raw CSV, and the losses of Signal8's own tool are reported in the same table as its wins.

What the results do and do not show

A benchmark round shows how a set of tools answered a fixed set of questions on one date, on the plans stated. It does not show how those tools will answer different questions, and AI products change often. Each result carries its run date for that reason.

Who publishes this

Signal8 Inc. publishes this lab and makes Octo. The methodology page sets out the rubric and the deviations from the full protocol.