Don’t mistake fluency for fact: The reality of AI tax research

The Efficiency Paradox in Modern Tax Workflows

For decades, tax research was defined by the exhaustive manual review of statutes, case law, and administrative rulings. A tax professional might spend several hours, or even days, verifying a single jurisdiction-specific exemption or calculating a narrow threshold for nexus. AI tools have compressed these tasks into minutes, offering structured arguments and comprehensive summaries at the click of a button. However, this efficiency creates a "fluency trap." Because AI models are trained on vast datasets of human language, they have become exceptionally skilled at sounding authoritative, even when the underlying logic is flawed.

In the field of taxation, where the margin for error is non-existent, the cost of a hallucination is significantly higher than in other creative or administrative fields. A minor error in interpreting a definition or a missed update in a local tax rate can lead to substantial audit risks, back taxes, and penalties. The fundamental issue is a widespread misunderstanding of what AI language models are: they are predictive engines designed to determine the most statistically likely next word in a sentence, not truth-verifying machines with a conceptual understanding of tax law.

The Evolution of AI in Tax: A Chronology of Risk

The journey toward the current state of AI in tax began in earnest in late 2022 with the public release of GPT-3.5 and subsequent iterations like GPT-4, Gemini, and Claude.

• 2023: The Year of Experimentation. Tax professionals began using general-purpose LLMs to draft memos and summarize legislative updates. Early warnings about hallucinations were often dismissed as "user error" or "poor prompting."
• 2024: The Rise of Specialized Agents. Companies began developing "Tax AI" layers, often using Retrieval-Augmented Generation (RAG) to ground models in specific tax codes. Despite these improvements, the core issue of "confident incorrectness" persisted.
• 2025: The Regulatory Response. Tax authorities, including the IRS and various European revenue services, began issuing warnings about the use of AI-generated tax advice, emphasizing that the taxpayer—and their human representative—remains solely liable for the accuracy of filings.
• 2026: The Verification Era. The industry has shifted toward a "trust but verify" model, recognizing that while AI is an excellent starting point, it cannot serve as the final word in statutory interpretation.

Technical Realities: The Mirage of Real-Time Search and Confidence

One of the most persistent myths in the industry is that an AI tool with web access is inherently more accurate. Aleksandra Bal, Global Indirect Tax Technology Lead at Stripe, notes that "can search" is not a synonym for "always searches." Different models handle live data retrieval with varying degrees of consistency. If a model’s internal weights suggest it already "knows" the answer based on its training data—which may be several months or years old—it may bypass a live search entirely to save computational resources.

Furthermore, the "confidence scores" that many users request from AI models are fundamentally deceptive. When a user asks an AI to "rate your confidence from 1 to 100," the resulting number is not a measure of accuracy or a reflection of a "truth meter" within the software. Instead, it is another layer of predictive text. The model generates a high confidence score because it has been trained on professional literature where experts speak with certainty. It is mimicking the tone of a confident professional rather than performing a self-audit of its own factual claims.

Why Retrieval-Augmented Generation (RAG) is Not a Cure-All

To mitigate hallucinations, many firms have turned to RAG, a process where the AI is forced to look at specific, uploaded documents (such as a specific state’s tax code) before answering. While RAG significantly narrows the window for error, it introduces new, subtler risks:

1. Context Misinterpretation: The AI may find the correct document but fail to understand the hierarchy of law—for example, prioritizing an outdated administrative guide over a more recent legislative amendment.
2. Fragmented Logic: AI models often process text in "chunks." If a tax rule is defined in one section but the critical exception to that rule is buried 50 pages later, the model may fail to connect the two, leading to a technically "correct" but practically "wrong" answer.
3. Synthesis Errors: When asked to compare rules across multiple jurisdictions, the model may blend elements of different laws into a hybrid rule that does not exist in any single jurisdiction.

The Prompting Fallacy: Why Commands Cannot Force Accuracy

A common trend among tax professionals is the use of "strict" prompting. Users often include phrases like "Be 100% accurate," "Act as a world-class tax attorney," or "Only use verified facts." While these prompts can improve the stylistic quality of the output, they do not change the underlying architecture of the model.

In many cases, these prompts actually increase risk. By demanding a "100% accurate" answer, the user encourages the model to remove hedging language—phrases like "this may vary" or "consult local statutes"—which are actually essential indicators of the limits of the AI’s knowledge. The result is a more certain-sounding response that is no more factual than a hedged one, making it harder for the human reviewer to spot potential errors.

Global Implications and Professional Liability

The stakes of AI hallucinations in tax research extend beyond individual filing errors. There are broader implications for the global economy and the legal profession:

• Professional Indemnity: Insurance providers for tax and accounting firms are beginning to scrutinize the use of AI. If a firm relies on an AI-generated argument that leads to a massive tax deficiency, will their malpractice insurance cover it?
• The "Black Box" Problem: In an audit, a taxpayer must be able to explain the rationale for their positions. If that rationale was generated by an AI that cannot provide a clear, traceable logic path back to the original statute, the taxpayer may lose the benefit of the doubt with tax authorities.
• Jurisdictional Complexity: In the United States alone, there are over 13,000 sales tax jurisdictions. Globally, the complexity of VAT, GST, and the emerging OECD Pillar Two rules creates a landscape too volatile for general-purpose AI to manage without specialized, purpose-built guardrails.

Official Responses and Industry Best Practices

Industry leaders and regulatory bodies are beginning to converge on a set of "responsible AI" principles for tax. The consensus is that AI should be used as a "co-pilot" rather than an "autopilot."

1. Mandatory Source Verification: Every AI-generated claim must be mapped back to a primary source (the actual text of the law or a formal ruling). If the AI cannot provide a valid link or citation that a human can verify, the information must be discarded.
2. Specialized Software over General LLMs: Experts like Aleksandra Bal advocate for the use of purpose-built tax technology, such as TaxJar or Stripe Tax, which utilize hard-coded logic and verified databases rather than purely probabilistic language models. These tools are designed to handle the binary nature of tax (where a rule is either applicable or it is not).
3. Human-in-the-Loop (HITL): No AI output should be sent to a client or a tax authority without being reviewed by a qualified professional. The role of the tax professional is shifting from "researcher" to "editor and validator."

Conclusion: Building a Reliable Guardrail

The efficiency gains of AI in tax research are real and permanent. The ability to synthesize vast amounts of data in seconds is a competitive advantage that no firm can afford to ignore. However, the "mirage of accuracy" remains a significant threat to the stability of tax compliance.

As the technology continues to evolve, the most successful tax departments will be those that implement rigorous internal controls. This involves recognizing that fluency is not accuracy and that the most authoritative-sounding AI models are often the most dangerous. For businesses operating at scale, the solution lies in a hybrid approach: leveraging the speed of AI for preliminary research while relying on established, verified tax software to handle the final, high-stakes calculations and filings. In the world of taxation, where precision is the only currency that matters, the human element remains the ultimate and most necessary guardrail.