In 2012, Da Silva Moore v. Publicis Groupe, 287 F.R.D. 182 (S.D.N.Y. 2012), became the first judicial opinion to approve technology-assisted review for document production in federal litigation. Three years later, Rio Tinto plc v. Vale S.A., 306 F.R.D. 125 (S.D.N.Y. 2015), observed that it had become “black letter law that where the producing party wants to utilize TAR for document review, courts will permit it.” Whether parties may use predictive coding is settled.
How they use it is not. Predictive coding can save significant time and money in discovery, but parties routinely disagree on the methods and protocols for implementing it, and those disagreements — over seed sets, search terms, disclosure obligations, statistical benchmarks — can negate the savings the technology was adopted to capture. The fight has moved from the admissibility of the method to the defensibility of the execution.
There is a structural answer, and it is not more motion practice. It is the appointment of an e-discovery special master with genuine technical expertise, made as early in the case’s life cycle as possible — before the predictive-coding protocol is drafted, not after it has failed.
§ 01 · How predictive coding actually works
Predictive coding is, essentially, the use of keyword search, filtering, and sampling to teach a computer to identify responsive or privileged documents. A set of documents — the seed set — is pulled from the corpus under review. A subject-matter expert who knows the case codes that set for responsiveness and privilege. The software analyzes the expert’s coding, attempts to learn what makes a document responsive or privileged, and codes a new set of documents; the expert corrects the machine’s mistakes; and the cycle repeats until the model hits agreed statistical benchmarks. Only then is it turned loose on the full corpus.
Two features of that process matter for dispute resolution. First, there is no single correct way to do predictive coding: seed-set composition, training cadence, culling criteria, and precision and recall standards all involve judgment calls. Second, every one of those judgment calls is a place where the parties can disagree — and most of them are invisible to a generalist bench until they surface as motion practice.
§ 02 · The lesson of Rio Tinto
Rio Tinto is the cautionary tale. The parties stipulated to a Predictive Coding Protocol that was supposed to make the technology self-executing. Execution instead became a major point of contention. The parties sought judicial relief over which search terms would cull the document universe before predictive coding was applied; over whether Rio Tinto could challenge the adequacy of Vale’s training in the predictive-coding workflow after alleged delays in disclosing its seed set; and over whether Vale could compel Rio Tinto to update and correct its technical disclosures. The court resolved the issues, but the motion practice and hearings were time-consuming, expensive, and fueled by extensive argument between the parties’ technical experts.
The court in Rio Tinto did eventually appoint a special master — but only after months of trudging through protocol issues, by which point the litigation had already been significantly disrupted. That sequencing is the lesson. A predictive-coding protocol drafted without technical fluency does not prevent disputes; it schedules them.
A predictive-coding protocol drafted without technical fluency does not prevent disputes. It schedules them.
§ 03 · What the special master actually does
Appointed early, an e-discovery special master serves three functions. In drafting, the special master’s technical knowledge helps the parties write unambiguous protocol provisions — methods for selecting search terms and seed sets, precision and recall standards, disclosure obligations — that are mutually agreeable and that fit each side’s actual technical capabilities. In monitoring, the special master oversees compliance with the protocol and decides technical issues that may be beyond the court’s or the parties’ understanding, which eliminates the need for each side’s consultants to argue their positions before the bench. And as a neutral technical presence, the special master helps the parties find common ground on issues that would otherwise become contentious simply because the lawyers are inexperienced with the technology.
Validation belongs in the protocol, not in a post-production brawl. A well-drafted protocol states in advance which statistical benchmarks the trained model must meet, how the results will be sampled — including the documents the model has coded non-responsive — and how recall will be measured against the agreed standard. The measuring stick is reasonableness, not perfection: “courts cannot and do not expect that any party can meet a standard of perfection” in producing ESI. Pension Committee of the University of Montreal Pension Plan v. Banc of America Securities, LLC, 685 F. Supp. 2d 456, 461 (S.D.N.Y. 2010). A protocol that defines defensible in numbers before review begins leaves very little to fight about after it ends.
§ 04 · When and how to seek appointment
The practical protocol for counsel: (1) Move early — raise the appointment when predictive coding is first proposed, at the meet-and-confer stage, not after the first dispute has hardened positions. (2) Insist on genuine technical expertise; a special master who cannot interrogate a seed set cannot referee one. (3) Scope the appointment order to all three functions — protocol drafting, compliance monitoring, and resolution of technical disputes. (4) Fix the validation standards — benchmarks, sampling method, disclosure obligations — in the protocol itself. (5) Do the arithmetic honestly: special masters cost money, but the benefits they provide often produce a net gain from predictive coding that is never realized when the court and the parties are left to their own machinations.
Predictive coding remains useful technology with real potential to save time and money. Rio Tinto teaches that those benefits are not self-executing, no matter what the protocol says on its face. Appoint the technical referee before the technical fight — not after.
Adapted from Legal Executive Institute articles on predictive coding and e-discovery special masters (2016), co-authored with Michael Mann.