Test both what AI finds and what the reviewer needs to do with the finding. Include known errors, clean controls, inconsistent files and permission boundaries, then compare the full preparation and review effort with the existing process.
Start with the review your team actually performs
Financial statement review brings together the statement, supporting schedules, prior-period records and explanations from several sources. A plausible answer is difficult to use when the reviewer cannot trace the figure or determine which version it came from.
AI can prepare comparisons, locate supporting evidence and flag apparent differences. Whether that capability is useful depends on the quality of the findings, the cost of checking them and how the unresolved work reaches the responsible reviewer.
A reported test gives a useful starting point
In a September 2026 report, OpenAI described a Legora financial statement review test involving 41 documents and four deliberately planted errors. The system found the four errors, including a £500,000 discrepancy, and linked findings to supporting material.
This was a reported vendor test in a specific workflow. It is not evidence of a Halyard implementation or a result that another firm should expect. Its useful lesson is the test design: known discrepancies, multiple documents and findings a professional can inspect.
Build a test pack with answers you already know
Choose a defined review, such as comparing a reporting pack with its agreed supporting schedules. Use synthetic, redacted or otherwise approved information in an environment the firm has authorised. Document the correct result before running the test.
Include files with a duplicate entry, a mismatch between a total and its schedule, an outdated supporting document and a difference that is legitimate but needs explanation. Include a clean case too. A system that flags every difference has not demonstrated useful judgment.
Keep a record of the errors introduced, their significance and the expected supporting sources. The person scoring the outputs should know which findings are material and which are harmless differences in presentation.
Include the awkward formats and missing information
A clean spreadsheet makes a convenient demonstration. The working review may also involve scanned PDFs, renamed files, uneven reporting periods and several versions of the same schedule. Include representative examples of those conditions.
Test what happens when supporting information is missing or outside the user’s permission boundary. The output should describe the gap and the sources searched. It should not manufacture an explanation, silently rely on an inaccessible record or interpret an unsuccessful search as proof that no record exists.
Measure the whole review, including the corrections
Record how many known material errors were found, how many were missed and how many unsupported warnings the system raised. Track whether each finding points to the correct document, period, figure and comparison.
Then ask a reviewer to work through the output. Count time spent opening sources, correcting findings, resolving differences and preparing the final record. Compare that total with the existing process. Faster preparation is useful only if the finished review is dependable and the extra checking is proportionate.
The test should also examine repeatability. Run comparable cases again after a material change to the model, configuration or source format. A single successful output is too narrow a basis for ongoing reliance.
Give every finding a usable review record
A finding should identify the issue, its supporting sources and the reason it was flagged. Preserve the original figures alongside any proposed correction. A reviewer should be able to accept, reject or request more information without losing the evidence.
Agree which role owns the decision and where unresolved items go. Financial judgments, approval of the record and any release of a report remain with the authorised professionals. Keep a workable fallback for cases the system cannot handle.
Review scope
Which statement, schedule or reconciliation is in scope, and which decisions remain outside it?
Known results
What material errors and clean cases will establish whether the system is useful?
Acceptable evidence
Can each finding be checked against an authorised source, period and figure?
Operating responsibility
Who reviews the finding, resolves an exception and decides whether the workflow can continue?
Make the decision specific to the workflow
An evaluation should end with a practical decision: proceed with the agreed scope, narrow it, fix an identified problem or stop. Set those conditions before examining the outputs.
The evidence may support AI preparing one part of the review while leaving another part unchanged. Expand only when the additional sources, formats and responsibilities have been assessed.
Related Halyard resources
Further reading
Founder of Halyard, an AI implementation company helping financial services firms find and implement practical AI safely.
