Posted On: Tuesday, August 2, 2016
For years, experts have predicted that computers would replace humans doing legal document review. Proper use of technology would result in cheaper, faster, and more reliable results. Though Technology Assisted Review (TAR) has now been available for several years, attorneys have been hesitant to embrace it. Many worried about whether the results could be validated and courts would accept it. Also, TAR required “seed sets” where a subject matter expert, often with a very high hourly rate, had to review thousands of documents at the start of a case. And if more documents were added, more expert review was needed. As a result, only the largest projects benefitted from TAR. Now, the latest generation of software, dubbed “TAR 2.0,” streamlines document review, improves results, and saves time and money for cases of all sizes.
The technology has evolved. Here at Target Litigation Consulting, we have had good success using Catalyst Insight Predict. This system does not require seed sets, so expensive experts are not needed to start the review, and the system adapts to the addition of new documents mid-review, in real-time.
With Predict you follow a traditional review methodology, where reviewers check out batches of documents and evaluate them, but behind the scenes a powerful algorithm is driving the review. Unlike a standard review, batches are not compiled in advance but created as they are requested. The system evaluates the reviewed documents and generates a batch of what it considers to be the most relevant documents. In other words, the system is always trying to identify the most relevant documents that have yet to be reviewed. This process is referred to as “Continuous Active Learning” (CAL).
This CAL process continues until reviewers are finding a low percentage of relevant documents in their batches. At this point, a quality control review of a few hundred documents is performed to evaluate whether the review team has in fact completed its work. If all goes well, the review is complete.
We had a client complete a review of almost 80,000 documents after reviewing only 41% of its collection. Systematic sampling showed that 8% of the documents were relevant (this is called the “richness” of the collection). Using Predict, the “richness” of batches soared to 70%. Reviewers were wasting far less time reviewing unresponsive documents and, when the batches no longer had a high percentage of relevant documents, the client was able to evaluate its review and conclude that it was finished. Assuming a conservative cost for review of $2 per document, the cost for this review was reduced from about $160,000 to $66,000.
It’s great to save money, but was it effective? Studies have shown that typical document-by-document review teams identify about 80% of the relevant documents in a given document collection (this is referred to as the “recall rate”). The calculated recall rate for this project was an impressive 94.1%. And this was not a fluke. We have had a number of projects with similar results.
Traditional TAR products required at least 50,000 documents to be effective. One Catalyst case study shows how a client was able to achieve 98% recall (that is, they found 98% of the relevant documents) after reviewing only 4,000 documents from a collection of 16,000. You no longer need a large project to use this TAR.
While we advise clients to make it known to opposing counsel that they intend to use TAR, the technology is now widely used and the new Federal Rules support its use.
Judge Andrew Peck, who is widely cited in cases involving technology and document review, stated in Rio Tinto PLC v. Vale S.A. (S.D.N.Y. 2015) “[I]t is now black letter law that where the producing party wants to utilize TAR for document review, courts will permit it.”
With document collections continuing to grow exponentially, the Federal Rules stress the idea that document reviews should be reasonable and proportional to the needs of the case. The Federal Rule of Civil Procedure 1 states that “every action and proceeding” should be “just, speedy and inexpensive.” Federal Rules of Civil Procedure 26 (g) discusses how discovery responses should be “reasonable.”
The Amended Federal Rule 26(b)(1), which went into effect on December 1, 2015, states: “Unless otherwise limited by court order, the scope of discovery is as follows: Parties may obtain discovery regarding any non-privileged matter that is relevant to any party’s claim or defense and proportional to the needs of the case, considering the importance of the issues at stake …, the amount in controversy, the parties’ relative access to relevant information, the parties’ resources, the importance of the discovery…, and whether the burden or expense of the proposed discovery outweighs its likely benefit.”
We are excited about this technology but technology is only as good as the people who use it. Make sure you are working with a trusted vendor who will help you navigate the process. To use TAR successfully, you want to have a project manager who can guide you through the process, lending his or her experience to your case. A good project manager will work with you to evaluate your document collection and help you to ask the right questions: Can it be filtered before review? What do you expect the richness to be? Is it a good candidate for TAR? Once you have a handle on your collection, you will work with your project manager to develop an efficient review plan, leveraging the appropriate technology to review your documents in the most efficient, cost-effective way. The result of an effective implementation will be a much faster document review, significant savings for your client and more time to focus on winning your case. There is no longer any reason not to use TAR and start reaping the benefits for you and your clients.