In a prior myth, I dispelled the notion that it is too expensive to use a Litigation Consultant. For my next entry in the Myths of Litigation Science, I turn my attention toward the decision that clients have to make after deciding to engage a Litigation Consultant – price. This can be a sensitive topic for many clients. The cost of research should definitely be a consideration in selecting a Litigation Consultant. However, since an investment in mock trial research is done with the expectation of information that will ultimately guide future decision-making about the case, the decision should not be made solely on price. It is a myth to believe that all mock trials are the same and that selecting a litigation research company based on price will always deliver the desired outcomes and information.

We have written previously on hiring a Litigation Consultant based upon credentials, so I will not belabor this point. Aside from credentials, there are several other questions clients need to ask themselves before selecting a Litigation Consultant to conduct a mock trial: 

        ·         How are the jurors being recruited? 

        ·         What is the structure of the mock trial?

        ·         What does the deliberation process look like?

        ·         What additional information is being collected during the mock trial?

        ·         What will the extent of the analysis look like? 

I will address each of these points and discuss the ramifications if not done properly. 

Juror Recruitment

It is common knowledge that plaintiff attorneys engage in conducting mock trials for their cases. In many instances, plaintiff attorneys conduct multiple mock trials before they take the case in front of a jury. However, it is also known that plaintiff attorneys sometimes present their cases to friends, family, and colleagues. Aside from the obvious problem of social desirability (see my article on social influence for more information), this type of recruitment does not allow clients to gain an understanding of potential jurors in the jurisdiction where the trial will be held. 

The same types of errors can be made when litigation consulting firms attempt to cheaply recruit jurors. Recruiting jurors that mirror the trial jurisdiction takes a significant amount of time to properly identify demographics, appropriately screen potential jurors for conflicts, and ensure that the individuals who show up on the day of the project are not simply “warm bodies filling seats.” Obviously, there is no way to duplicate the actual jurors who would be seated for a given trial, but a rigorous approach to recruiting is critical to the conclusions one will draw from the research and is far superior to placing an ad on Craigslist or Facebook and taking whoever responds. A rigorous scientific approach focused on finding engaged, articulate, representative, and dependable jurors costs money.

Structure

When we talk to clients about potential mock trials, there is a belief that all of them are structured the same: (1) Plaintiff counsel will give a “clopening” which typically involves an opening statement, evidentiary presentation, and closing statement; (2) Defense counsel will give a similar presentation; and (3) Plaintiff counsel will provide a brief rebuttal presentation. There is some variation in how these presentations are approached, but this is usually what clients think of when they envision “mock trials.” However, there are many other ways to structure a mock trial or focus group – which are two very different types of projects at Courtroom Sciences, Inc. 

One approach that cannot be utilized if the client is attempting to do the mock trial “on the cheap” is to provide jurors the opportunity to see and hear from in-person key witnesses. As we have found, witness testimony is one of the most important determinants of case outcomes. If witnesses perform poorly in depositions, settlement demands can be pulled, or the plaintiff’s requested amount can be increased. If witnesses perform poorly at trial, it could be the difference between a plaintiff or defense verdict. Sure, deposition clips can be played for jurors and they can provide their feedback on the videos. However, allowing witnesses to testify live offers two advantages: (1) jurors can see the witness testify live, allowing them a better opportunity to gauge the witness’s credibility and (2) the witness gets additional practice on testifying in front of jurors. This type of approach comes with a cost though.

Another approach that comes at a cost is conducting a two-day mock trial. Too often we hear jurors say, “I wish I knew about X, Y, and Z” or “based on what we know…” From our perspective, this is not something we want for our clients, especially in high-stakes litigation. A better approach is to take the necessary time to lay out the case facts, explain the evidence, hear from witnesses, and then allow jurors to deliberate. Trying to cut corners on any of these aspects can lead to less than 100% accuracy in the decision-making process. 

Deliberation Process

There is a plethora of literature to indicate that the responses jurors give individually are different from those in a group setting.[i] This is why allowing jurors to deliberate during mock trials is one of the keys to a successful research project. There are many ways that the deliberation process can be handled. The best way is to allow jurors to interact with one another with little to no outside influence from members of the litigation or research team. Outside observers should be “flies on the wall” and watch how jurors process information without directly participating in or guiding the discussion. Sometimes jurors incorrectly interpret jury instructions, sometimes they misremember case information, and sometimes they use inappropriate burdens of proof (i.e., beyond a reasonable doubt instead of preponderance of the evidence). It is important for the integrity of the research to allow jurors to make these errors since the real jury would not be given any guidance during deliberations for the actual trial.    

With some litigation consulting firms, a common approach is to quickly enter the deliberation room and attempt to provide an unsolicited “fix” to the jurors’ decision-making process. Even worse, some litigation consulting firms place a facilitator in the deliberation rooms. The job of this facilitator is usually to help move along the deliberation process. However, these facilitators have also been known to end up dominating the conversation or attempt to move jurors off topics that may seem harmful to their client’s case. When this occurs, clients have a skewed view of their case, or worse, an inaccurate belief of the value of their case. Aside from being methodologically incorrect, this approach can cost the client money. 

Additional Information

Most mock trials end with the completion of the jury deliberation process. Jurors are thanked for their participation, are reminded of their confidentiality agreement, and leave. There are few times, though, where Litigation Consultants will conduct a focus group discussion to obtain additional information from jurors before they leave. However, these are typically the exception and not the rule. The problem with not conducting this final focus group is that there is no way for the client to probe further on topics that were addressed during the deliberation process. In addition, there is often information that was withheld from the jurors during the presentations for various reasons (e.g., potential motions in limine). The focus group is a time to ask jurors about this information. The way to do this is important, though. To ask jurors, “would it have changed your opinion if…” is not the best way to approach these questions. Psychologically, jurors have made their decisions during the deliberation process. Even if the additional information would have changed their opinions, jurors are usually reluctant to admit that they would have changed their minds or they are not able to properly assess whether a given piece of information would have changed their thought process. A Litigation Consultant with proper research training will know how to ask these questions to elicit valid feedback. 

The focus group is also the time for the Litigation Consultant to probe deeper into the deliberation process and inquire about perceptions of specific pieces of information. Research has shown that just because jurors do not mention a given topic, does not mean that they did not use that information when deciding a case.[ii] It is quite possible that when clients watch the deliberation process and do not hear jurors talk about a specific piece of evidence, they may believe that this is not something that the attorneys need to focus on. It is the Litigation Consultant’s job to identify those pieces of information and engage in a discussion with jurors to figure out whether that evidence does matter. 

Analysis

Once the data from the mock trial has been collected, the question then becomes, “What does the final analysis look like?” I have seen mock trial reports that simply regurgitate questionnaire data that was collected on the day of the project. These may come in the form of tables or fancy graphs. However, what is often missing is a deep level of analysis about what the data actually mean. For example, it may be interesting to know that jurors did not find the defendant to be very likeable, but what does that mean for how they used this information to interpret the case? What does this knowledge mean for structuring the case going forward? How is this knowledge going to be used to inform trial strategy? How will the defendant, and to a greater extent the defense team, adjust for trial? When these questions are not answered in the mock trial report, the onus gets placed on the trial team to interpret this information. The problem, though, is that members of the trial team often do not have the training and expertise to speak to human behavior. This is where an experienced Litigation Consultant comes in, and this type of analysis takes time and costs money. Any client who is not being delivered a thoughtful analysis and interpretation of the data is not getting their money’s worth from the mock trial. 

In conclusion, clients who are making decisions about jury research based on price will likely find that corners need to be cut to keep the project under budget. That may mean reducing the quality of the results and subsequently the quality of the decisions that need to be made for defending the case. It is like anything in life. If you buy the cheapest television, your picture is not going to be as good as the one on the more expensive television. If you use the cheapest mechanic, the chances are higher that your vehicle will not be properly fixed. If you buy the cheapest vehicle, it is likely going to break down before a more expensive one. The list goes on and on. I am a firm believer in the old adage, “garbage in, garbage out” and “you get what you pay for.” Therefore, clients who are making mock trial decisions based solely on price are potentially setting themselves up to be “penny wise and pound foolish” in the way of an adverse verdict that could have been reduced by spending a little more on the front end.