AI in eDiscovery: The Defensibility Question
The use of artificial intelligence in document review has been judicially accepted since at least 2012, when Magistrate Judge Andrew Peck endorsed Technology Assisted Review (TAR) in Da Silva Moore v. Publicis Groupe. In the years since, TAR and predictive coding have become mainstream tools, and courts have repeatedly affirmed that computer-assisted review is not only acceptable but may be superior to manual review in terms of consistency and recall. The legal foundation for AI-assisted review is well established.
However, the emergence of Large Language Models (LLMs) introduces new concerns that the existing TAR case law does not fully address. Unlike traditional TAR systems, which use supervised machine learning to classify documents based on attorney-coded training sets, LLMs generate natural language responses, can "hallucinate" confident but incorrect analysis, and operate through mechanisms that are difficult to explain to a skeptical judge. The transparency that courts expect — the ability to describe your methodology, explain how the technology works, and demonstrate that the results are reliable — requires a more deliberate approach with LLMs than with traditional TAR.
FRCP Rule 26(g) requires that every discovery response be signed by an attorney who certifies that it is "complete and correct as of the time it is made" and that the discovery effort is "consistent with these rules and warranted by existing law." This certification attaches to a person, not to a technology. When you use AI to assist document review, the signing attorney must be able to explain and defend the methodology, demonstrate that the results were validated, and certify that the review was reasonable under the circumstances. An attorney who signs a discovery response cannot deflect responsibility by saying "the AI did it."
The good news is that AI-assisted review is entirely defensible if done right. The key is to treat AI as a tool that enhances attorney judgment rather than a replacement for it, to establish validation protocols that demonstrate reliability, to maintain human oversight at every critical decision point, and to document everything. The steps that follow provide a framework for using AI in document review in a way that will withstand scrutiny from opposing counsel and the court.
Step 1: Understand Your Disclosure Obligations
Before you use AI in document review, you need to understand what you are required to disclose about your methodology. The landscape is evolving, but the trend is clearly toward greater transparency. Failing to disclose your use of AI when required can be far more damaging than the use of AI itself — it suggests you had something to hide and undermines your credibility with the court.
Start with the meet and confer under FRCP 26(f). This is the natural opportunity to discuss discovery methodology with opposing counsel, including the use of AI tools. The 26(f) conference requires the parties to discuss the subjects on which discovery may be needed, the form in which ESI should be produced, and any issues about privilege. The use of AI in document review fits squarely within this framework. Proactively disclosing your intended methodology demonstrates confidence in your approach and reduces the likelihood of challenges later in the case.
A growing number of local rules and standing orders now require specific disclosure of AI use in litigation. Some courts require disclosure of any AI tool used in the preparation of filings, while others specifically address AI in discovery. Before using AI in any case, check the local rules and any standing orders of the assigned judge. Failure to comply with a disclosure requirement is far more likely to result in sanctions than the use of AI itself. Even in jurisdictions without specific AI disclosure rules, the general obligation of candor toward the tribunal counsels in favor of proactive disclosure.
When preparing your disclosure, document your methodology as if you were explaining it to a skeptical judge — because you may need to. Your description should include the specific AI tools or platforms used, the types of tasks the AI performed (relevance classification, privilege screening, key document identification), the validation protocols you employed, and the human oversight framework that governed the review. A well-documented methodology memo, prepared at the beginning of the review and updated as the process evolves, is your best defense against a challenge to your discovery methodology.
Step 2: Establish a Validation Protocol
Validation is what transforms AI-assisted review from a black box into a defensible methodology. Without validation, you are asking the court to trust that the AI got it right. With validation, you are demonstrating that the AI's results were tested against human judgment and found to be reliable. Courts have consistently held that validated AI-assisted review is acceptable; unvalidated AI-assisted review is a different matter entirely.
The foundation of any validation protocol is random sampling with senior attorney review. Pull a statistically significant random sample from the document population and have a senior attorney — someone with deep familiarity with the case issues — independently review each document in the sample. Compare the senior attorney's coding decisions against the AI's coding decisions to calculate an agreement rate. This agreement rate is the metric you will use to demonstrate the reliability of the AI-assisted review to the court.
For relevance classifications, an agreement rate of 80-90% between the AI and the validating attorney is generally considered acceptable. This is comparable to or better than the agreement rates typically observed between human reviewers, which studies have shown range from 70-85% for relevance determinations. For privilege classifications, the threshold should be higher — 90% or above — because the consequences of a privilege error (waiver, disclosure of attorney-client communications) are more severe than the consequences of a relevance error. Consider implementing a two-tier validation protocol for privilege: the AI screens for potentially privileged documents, and every document flagged as potentially privileged receives a full manual review by a senior attorney.
Document every validation round in detail. Record the sample size, the sampling methodology, the identity of the validating attorney, the agreement rate, any disagreements and how they were resolved, and any adjustments made to the AI's parameters or prompts as a result of the validation. If you run multiple validation rounds (which you should, particularly early in the review when the AI is being calibrated), document the progression of agreement rates over time. An improving agreement rate across successive rounds demonstrates that the review process is being refined and that quality is being actively managed.
Step 3: Maintain Human Oversight
The single most important principle in defensible AI-assisted review is that AI is a tool, not a decision-maker. The AI can analyze, classify, flag, summarize, and recommend, but the final decision on every material question — is this document relevant, is this document privileged, should this document be produced — must be made by a human attorney who has exercised independent professional judgment. This is not merely a best practice; it is a requirement under the Federal Rules.
FRCP Rule 26(g) requires that discovery responses be certified by an attorney. This certification is a personal representation that the attorney has conducted a reasonable inquiry and that the response is complete and correct. An attorney who delegates this responsibility entirely to an AI system, without independently verifying the results, has not conducted a reasonable inquiry and cannot in good conscience sign the certification. The AI can do the heavy lifting of initial classification, but the attorney must verify and take responsibility for the results.
Senior attorneys should make the final calls on all material questions. This means that privilege designations should be reviewed and approved by an attorney with sufficient experience to recognize the nuances of privilege law. Key document identifications should be confirmed by an attorney who understands the case strategy and can assess whether a document is truly significant. And production decisions — what to produce, what to withhold, what to redact — should be made by an attorney who understands the discovery obligations and can certify compliance.
Implement a human-in-the-loop workflow for quality control throughout the review. This means that the AI's work is continuously monitored by human reviewers, not just validated at the beginning and end. Designate specific QC checkpoints where human reviewers examine a sample of the AI's recent work and flag any systematic errors. If the AI is consistently miscoding a particular type of document, the issue should be identified and corrected through prompt adjustment, additional training, or manual review of the affected population. Continuous human oversight ensures that errors are caught and corrected in real time, rather than accumulating undetected throughout the review.
Step 4: Document Everything
Documentation is the connective tissue that holds a defensible AI-assisted review together. Without documentation, your validation protocol is an unverifiable claim; with documentation, it is evidence of a rigorous, well-managed process. The time to create documentation is during the review, not after a challenge is filed. Retrospective documentation is inherently less credible and may not capture the details you need to defend your methodology.
Your documentation should include the platform and model information — the specific AI platform used, the model version, and any custom configurations or fine-tuning that were applied. Document the review protocol in detail: the workflow design, the roles and responsibilities of each team member, the review passes (first-level review, QC review, privilege review), and the escalation procedures for difficult coding decisions. Record all search terms used during the review, including the terms themselves, the date each search was run, the number of hits, and how the search results were processed.
If the AI was trained or calibrated using training data, document what training data was used, who prepared it, when it was created, and how it was validated. Record the results of every validation round: sample sizes, agreement rates, disagreements, resolutions, and any adjustments to the AI's parameters. Document all adjustments made to prompts, models, or workflows during the review, including the reason for each adjustment and its impact on review quality. Compile final statistics for the review: total documents reviewed, documents produced, documents withheld, documents redacted, and overall agreement rates.
Identify the supervising attorney responsible for the review and document their involvement at each stage. The supervising attorney should be someone with sufficient experience and authority to certify the discovery responses, and their documentation should demonstrate active engagement with the review process — not mere rubber-stamping of the AI's output. In the GWA PLLC case, over 1 million documents were processed and the client was found not guilty. In the Clayton Trial Lawyers case, DecoverAI helped secure a $15.4 million verdict. In both cases, the defensibility of the review process was never challenged, because the methodology was documented, validated, and supervised by experienced attorneys.
How DecoverAI Makes AI Review Defensible by Default
DecoverAI was designed from the ground up with defensibility as a core requirement, not an afterthought. The platform's foundational principle is that every AI-generated insight must be sourced and cited. When the AI identifies a document as potentially privileged, it points to the specific language in the document that triggered the classification. When it identifies a document as relevant to a particular issue, it cites the passages that support the relevance determination. There are no black-box conclusions — every output is traceable to specific evidence in the underlying documents.
This approach embodies what DecoverAI calls the "Trust but Verify" principle. The AI does the analytical work of identifying, classifying, and surfacing relevant documents, but every conclusion is presented with its supporting evidence so that attorneys can verify the AI's reasoning and apply their own professional judgment. This is not just good practice — it is what courts expect. A reviewing attorney who can see exactly why the AI classified a document in a particular way is in a much stronger position to exercise independent judgment and certify the discovery response.
DecoverAI generates a comprehensive audit trail automatically, without requiring attorneys or litigation support staff to manually document the review process. Every search, every classification decision, every validation round, every prompt used, and every human override is logged with timestamps and user identification. If the review methodology is ever challenged, the audit trail provides a complete, contemporaneous record of exactly what was done, when, and by whom. This automatic documentation eliminates the risk of incomplete or retrospective documentation that can undermine a methodology defense.
The platform's Prompt Store allows review teams to create, save, and share tailored queries that have been validated for specific case types and legal issues. Rather than starting from scratch with generic prompts, teams can use pre-validated prompts that have been refined through multiple rounds of validation in similar matters. The Prompt Store also ensures consistency across large review teams — every reviewer uses the same validated prompts, eliminating the variability that arises when different reviewers craft their own ad hoc queries. Combined with the automatic audit trail and source citation, the Prompt Store completes a defensibility framework that makes AI-assisted review not just acceptable, but demonstrably more rigorous than traditional manual review.