Why AI Finance Projects Fail in Service Businesses
Level AI finance thesis
A repeatable output can answer the wrong question
Sam Yang, Level
AI Accounting Reconciliation Controls
Set explicit conditions that require human review for incomplete populations, missing costs, unsupported accounting meaning and unauthorized posting. A fluent answer does not satisfy any of those conditions. The person making a pricing or cash decision should see which evidence remains unresolved.
Define the acceptance test before the agent
An AI agent is software that uses a model to carry out a task. The population means the complete set of records that should be included. An acceptance test states the checks that must pass before someone uses the result.
An agent can extract a number faithfully and still leave the decision unsupported. Three linked job IDs with cost for only two jobs produce perfect identity linkage and incomplete profit. A narrative that says the ledger balances cannot repair that missing cost. Decide what must be true before the result can inform pricing, cash planning or close.
Use the agent to collect evidence, propose candidate matches and organize exceptions. Use explicit arithmetic for money, stable identities for populations and a named reviewer for accounting meaning. Posting changes the books and needs separate approval and a check of the resulting records. A successful extraction does not supply that approval. Keep supported, estimated and source-needed outputs visible rather than hiding every uncertainty inside a fluent summary.
When AI extracts a PDF or spreadsheet, compare important amounts and identifiers with the original page or cell. Keep the source location, extraction method and model version, stated confidence, corrections and reviewer decision. A confidence score is a reason to prioritize review, not proof that the extracted value is correct.
Test a failure the polished answer might hide
Test what the workflow actually checks before trusting its result. Two blank company identifiers do not prove the records belong to the same company. A row count repeated from the same extraction is not an independent check that all records were included. A plausible match between a card payment and a supplier bill still needs supporting records and accounting review.
Build counterexamples around those distinctions. Supply an empty company identity, a missing row, two same-amount transactions and a correct sum with a missing cost source. The intended result is a specific unresolved state, with the evidence needed to resolve it. If the workflow still emits a confident financial answer, the problem is the checks required before using the output, not the model's tone.
Model review helps challenge interpretations, but it remains a judgment. Call a result deterministic only when the same saved inputs, agreed definitions and software checks reproduce it. A second model agreeing with the first is useful review evidence; it is not proof that all required ledger records are included or that the accounting action is authorized.
Financial evidence in practice
Why can an AI finance project fail with repeatable numbers?
A repeatable output can answer the wrong question. AI can organize documents and candidate matches, but complete populations, arithmetic, accounting meaning and approval still need explicit ownership. The failure is often an undefined acceptance test.
Run the check
- Write one business question and its source population.
- Use code for arithmetic and identities; retain the exact input.
- Review uncertain classification and missing evidence separately.
- Require independent native read-back before any authorized accounting action.
Fictional example: Linked records with missing costs
All three fictional IDs link, giving 100% linkage. One has no cost evidence. Perfect linkage does not establish profit completeness.
Invented teaching inputs; downloadable files store these dollar amounts in USD cents.
Inspect the expected output
{
"fictional": true,
"unit": "USD cents",
"id": "report-population",
"linkagePct": 100,
"missingCostIds": [
"C"
],
"profitAvailable": false,
"limitation": "This is teaching arithmetic from supplied invented inputs, not source authenticity, financial assurance or authority to post."
}Reproduce this calculation
Download the code and input into one folder, then run:
node reproduce.mjs report-population.input.jsonThe code checks declared units and identities and refuses unsupported inputs. No browser or customer account is used. Source authenticity and accounting approval require separate evidence.
Reviewed 2026-10-06. Level financial-control guidance, with visibly fictional teaching examples. The example shows a control boundary, not a measured customer outcome or industry benchmark.
Read this content revision history
Continue with the related financial decision →More reproducible financial checks
Paid, operational and productive labor hours
Fictional $4,800 payroll over 160 paid hours is $30 per paid hour; over 120 operational hours it is $40. With $1,200 allocated overhead and 80 productive hours, the estimating basis is $75 per productive hour. These rates answer different questions. A separate fictional operational budget of 100 hours and $6,000 payroll gives 20% more actual operational hours and 20% lower payroll, with a lower payroll rate on the same operational-hour basis.
Free benchmark review
See whether your books are benchmark-ready.
Start with a free conversation about your priorities. With agreed access to the relevant records, we compare your numbers with applicable benchmarks and identify what to review first.
Run the question against the relevant software playbook
Open the related financial check when the next question concerns that workflow. The Accounting Software Integration Guides contain the named-system collection notes, pitfalls and prompts with clear limits on what the AI may do.
Make the expected answer downloadable
An automation demo is easier to assess when the reader can inspect the input, reproduce the arithmetic and see the unresolved decision. A screenshot of a plausible total leaves the population and calculation hidden.
The software workflows include fictional spreadsheet-style CSV inputs, expected results tied to those inputs and standalone code for reproducing the calculations. The Jobber cost example checks whether the same cost was counted twice. The Finaloop customer-group example compares customers at the same recorded age and states which costs are included in their profit. A customer group, also called a cohort, contains people who first bought in the same period.
Successful reproduction establishes that those inputs produce those calculations. It does not establish that a live export is complete, a missing cost is zero or an accounting change should be approved. The operating value comes from naming which further evidence and decision each result needs.
Sources and method
Accounting software integration guides. This guide shows how to check whether the numbers are ready to use. The examples use made-up numbers, clearly labeled and available to download. They do not include private customer documents or measured customer results. Before using the method, check that your account has the required access and that your reports count the same things.
Get the next one
Want next week's benchmark in your inbox?
One email a week. Real numbers from 2,200+ contractors in the research universe. No fluff. Unsubscribe anytime.
Related reads
Operations
Why Your Field Software Numbers Don't Match Your Accounting
Reconcile field hours, invoices, recognized revenue, and job costs. Billing capture: 97.1% median, 66.9% bottom decile (963 companies).
Operations
The AI Inbox: Emailed Reports as Finance Pipelines
Asking for a report by email does not mean you have received and saved it. Track delivery, original attachment, parsing and review separately, preserving changed versions.
Operations
The API Is Not Enough for Finance Automation
Use APIs for supported records. Add native reports or permitted browser collection for verified gaps, then compare identities, dates and costs.

About the author
Sam Yang
Founder & CEO
Founder of Level, the always-on finance team for contractors, service businesses, and ecommerce brands. Ex-CFO across trades, SaaS, and service businesses. 4 years as Director of Growth Product at BuildOps, building financial tooling used by 1,000+ commercial contractors. Four years in PE and investment banking rolling up and acquiring service businesses, $2.5B in total transactions including M&A and IPOs. Stanford MBA, Brown undergrad. The Level founding team's analysis of 2,200+ contractors ($13.25B in job revenue) across operating, private-equity, and CFO roles anchors the Level Index benchmark research.
LinkedIn