American Enterprise Institute

April 23, 2007

[Edited transcript from audio tapes]

9:15 a.m.	Registration and Breakfast
9:30	Speaker:	David E. Bernstein, George Mason University School of Law
	Panelists:	Edward K. Cheng, Brooklyn Law School
		Joe G. Hollingsworth, Spriggs & Hollingsworth
		David Michaels, George Washington University and SKAPP
		Deborah Runkle, American Association for the Advancement of Science
	Moderator:	Ted Frank, AEI
11:30	Adjournment

Proceedings:

TED FRANK: One of the biggest changes in civil litigation over the last 15 or 20 years has been the rise of Daubert, the Supreme Court decision affecting how expert witness evidence is heard by courts. It has been very influential at the federal and even at the state level.

One of the prominent academics writing on Daubert is David Bernstein at George Mason, and when he writes the paper saying that Daubert is not quite working and needs to be reconsidered to a certain extent, that is obviously something we want to hear about.

We have a very impressive and large panel here reflecting a lot of different points of view, and we look forward to a productive discussion. We are going to lead off with David Bernstein, who has been teaching at George Mason Law since 1995. He is the author of over 60 frequently cited scholarly articles, book chapters, and think tank studies, including The New Wigmore: Expert Evidence. He is a past chairman of the Association of American Law Schools, Evidence Section, and a current contributor to the popular Volokh Conspiracy blog. Professor Bernstein.

DAVID E. BERNSTEIN: Thanks, Ted. So until relatively recently, any marginally qualified expert witness was able to testify to just about anything; this was commonly known as the let-it-all-in rule. And obviously, as Ted mentioned, since 1993 when the Supreme Court decided Daubert vs. Merrell Dow Pharmaceuticals, the rules for admissibility in federal court and in many state courts also have become much stricter. And then just to top it off, in 2000 the Federal Rules of Evidence were amended – Rule 702 in particular, to ensure that all expert testimonies are subject to a rather stringent reliability test.

But one of the oddities of the Supreme Court’s Daubert Trilogy – Daubert, Joiner, and Kumho Tire the court never really articulated what beyond the plain meaning interpretation of the rules was its underlying rationale for changing the underlying tests for admissibility of expert evidence. And this has led to a lot of critics of Daubert and 702 saying, “Well, why do we need a special test for expert evidence? After all there is a lot of unreliable evidence out there.”
Eyewitness testimony for example has been more or less empirically proven to be often unreliable. Why do we need something special for expert testimony?

So I started off by thinking about that and my view is that the implicit rationale for a special reliability test is to deal with the special problems posed by expert testimony that result from what I call adversarial bias, which is the bias that a witness has specifically because an expert is called by one party to an adversarial proceeding. This adversarial bias rises in at least three ways.

You have, first of all, conscious bias, which is the “hired-gun” problem when experts who have their pants for sale who will say whatever you want for enough money. And we have unconscious bias where an expert more or less psychologically identifies with the one side and to that extent the testimony is impurely objective. Often the testimony is unconsciously shaded by wanting to help the side that is paying them. This is a particular problem in the forensic science context because the forensic experts are typically part of the prosecution team - are paid [by] the government to be part of that team – and they identify with the prosecutor’s goal in convicting a particular suspect.

And finally, and maybe most importantly, we have a selection bias. When it comes to a typical fact witness in the case, the parties are basically stuck with who they have. There are just so many witnesses to a typical automobile accident, right? But when it comes to experts, there is basically an unlimited opportunity in many cases to shop around until you find the expert who is both a good witness - someone who is able to convey a material well to a jury, and also who will endorse the attorney’s theory of the case. This does not have to be a “hired-gun.” The expert could be extremely well credentialed and entirely sincere, but it does not mean their views represent the mainstream on this particular issue. If you contact enough experts and you have a bell curve of opinion, you will ultimately find someone who is on the tail.

Federal law has ultimately tried to deal with this adversarial bias indirectly. Adversarial experts are still by far the norm, which retains the perceived advantages of the adversarial system in giving the parties the freedom and incentive to find the most persuasive evidence that they can. But Rule 702 tries to limit this harm by applying a reliability test to try to ensure the adversarial bias does not dominate the process. So Rule 702 provides the expert testimonies admissible only if you have sufficient facts or data; if the testimony is a product of reliable principles and methods; and most significant, you have to show the witness has applied the principles and methods reliably to facts of the case, which in theory should really get rid of all junk science.

My goal for the rest of this talk is to discuss whether Rule 702 is successful on its own terms in the sense that it limits the harm from adversarial bias while not going overboard in excluding useful information from the jury and retaining the adversarial process. So I’ll talk about this in the context of three specific areas of law: Forensic science, causation testimony, and toxic torts cases, and finally, testimony based on an expert’s experience and training which may be idiosyncratically called connoisseur testimony.

Let’s start with forensic science. So forensic science, as I noted, is subject to adversarial bias, especially this unconscious bias, and it is not really self-correcting. Indeed, many forensic techniques do not even have adequate indicia of reliability. And even when a reliable technique is being used, it is not usually structured to discover errors in such context. If prosecutors and defendants have more or less equal resources, this probably would not be much of a problem. Rule 702 would serve to exclude a lot of unreliable forensic testimony.

But the problem is that rule 702 does not make that much of difference in the real world. It is not self-enforcing. You need the attorney to raise the issue or argue the issue. And the problem, to put it bluntly, is that most criminal defendants have public defenders and most public defenders are overworked, underpaid and inexperienced. And they typically do not have the resources, time and energy to really challenge or even look into expert testimony. And the problem, therefore, is not simply that federal courts are perhaps too inclined to admit the prosecutor’s expert testimony – although, I think they are – but often, really dubious testimonies are not even challenged. There is not even a Daubert motion made because the defense attorney does not know what to do with it and does not have the resources to look into it.

So I think you need something a lot more drastic than just rule 702 to solve the problems attendant to forensic testimony. Now obviously, I cannot go to a great detail about this, but I have a lot of citations in the paper. But I think the first thing you should do is take the forensic science services out of the hands of the prosecutor and make them a neutral body that is open to either side. You should not have just one service; you should have several. You should test them every so often blindly and if they fail routinely that should make them go out of business because who is going to pay a lot of money for forensic science services when you could easily cross-examine the experts? Say, “Hey, did your company fail two out of three of the last tests? Why should we trust you?” And, of course, it would not hurt if more resources were available to defendants to challenge expert testimony when necessary.

So onto expert testimony regarding causation and toxic tort litigation – and pharmaceutical litigation is included within that. The best testimony you could get for toxic torts, the gold standard, is a series of reliable epidemiological studies with a relative risk of more than doubling of the risk which needs the preponderance of the evidence standard.

And in fact, one can argue that if you are looking for truly reliable evidence, this is all you are really going to get into the toxic tort cases with very few exceptions. But the problem is plaintiffs rarely have such evidence; and it is not always their fault. It is expensive and time-consuming to do epidemiological studies. And if it is a new issue or an issue that just is not of general interest to the scientific community, no one is going to have done that.

So plaintiffs, by necessity, rely on evidence that is suggestive but hardly dispositive. They may rely on animal studies; laboratories studies on cells; anecdotal case reports; temporal relationships; regulatory actions; analogy; differential diagnosis – which is not really differential diagnosis but they call it after some reason in cases where they exclude other causes of injury; studies on humans involving much higher exposure levels in the plaintiff; epidemiological studies that are suggestive but not specifically significant or have a relative risk below two; and finally dechallenge/rechallenge evidence. Someone who is exposed to a substance, they have the symptom; they are un-exposed and symptoms go away.

Now, the problem is that none of this evidence is truly reliable. It is speculative; it is guessy; it relies on educated guesses by experts. It does not mean that it is always wrong. But the problem is that once these issues become subjects of litigation, it is not like using are getting neutral scientific opinion on whether this evidence is enough to show causation. What you are getting is, the plaintiff has shopped around until they find an expert willing to say, “Yes, this is enough evidence to prove causation.” And basically, since there are so many toxicologists, physicians, and so forth out there, the fact that a plaintiff was able to find one or even several experts who are willing to say, “Yes, this substance causes diseases,” really is not worth anything.

Again, in some cases though, it might be the case that you surveyed a group of neutral experts in the field and say, “Yes, – yes, it is all sketchy evidence in itself but if you put it all together – given with what we know about, how rare this disease is, and given what we know about the analogy, the animal studies, and the dechallenge/rechallenge – there is enough evidence.” But there is no way for a judge in his or her position to be able to determine whether this is a case where you have adversarial bias, which is bringing in an expert with an idiosyncratic opinion, or the experts actually reflect something resembling, if not consensus, at least a substantial body of scientific opinion. Now, the problem is that Rule 702, for the plaintiffs the problem is Rule 702 requires by its terms, the exclusion of all this evidence. If it has to be truly reliable, they cannot be speculative and guessy even if most people in the field would think this is sufficient.

So the result is if court applies Rule 702, we have a huge improvement over the prior let-it-all-in philosophy because it is better to – most of the stuff turned out to be junk science – it is better to exclude it entirely than to admit it entirely, but you also have a situation where you may be excluding useful evidence. And the other problem is that because courts are concerned, I think, that, well, what if there really is not enough evidence here? What if we are unfairly excluding a plaintiff’s causation testimony even though they have qualified people? How are we supposed to know?

A lot of courts are simply refusing to apply Rule 702 as written and Daubert-Joiner-Kumho has written, instead they are engaging in all sorts of circumlocutions and rather illogical opinions in order to evade 702. I was just discussing with Joe before we started some courts, for example, are just sighting pre-702 or even pre-Joiners, some of them even pre-Daubert precedents as if they are still good law. So ultimately, we have a situation where 702 is either being applied strictly and throwing out even potentially useful evidence or it is being applied much too liberally and letting in that speculative evidence that is only there because of adversarial bias.

So my ultimate conclusion is that we should try to move towards the system with regard to toxic tort evidence where the presumption is that we will exclude it. But if the plaintiff is willing to or even if the plaintiff is not willing – in the case where a judge thinks that there might be enough evidence here to go forward, but I'm concerned that Rule 702 – says it is not reliable. Why not find a panel of three experts, five experts, whatever the right number would be who are – I would not say neutral because that is a loaded phrase, but non-partisan – and ask them, “Okay, here is the evidence we have. Would you think this is sufficient to show causation?”

Even that is potentially problematic – I go through it more in the paper – but I think done properly this would be an improvement. And if nothing else, on the one hand, we will allow in the rarer cases where there is enough evidence for the case to go forward and moreover, will probably be an improvement even for defendants, because if judges knew they had this option or wanted to use it, they would be less leery of just excluding the evidence and throwing out the plaintiff’s case.

Finally, we have testimony based on experience and training – what I call connoisseur testimony. A huge amount of expert testimony is connoisseur testimony. There is even a lot of testimony that has an objective scientific basis to some extent, but ultimately, the interpretations of the results is based on an expert’s experience and training. I teach a class on expert testimony. I have experts come to my classroom every year; and I often ask them, “Okay, now you have given us your whole presentation. If I was cross-examining you in court, I would say ultimately, ‘Dr. so and so, what is the basis of your opinion? How do we know that we should believe you?’” Almost always, especially in the forensic context but for other contexts too, the expert will say, “You have to rely on my 20 years of training and experience.”

So that is not especially objective and because there is no objective way to ensure the expert testimony is reliable, you are left again with Rule 702 with the prospect of excluding all connoisseur testimony, which is inherently problematic. Besides some of the forensic context like, how you interpret fingerprints, or evidence of bones – is this a stab wound or an animal bite – some like to give perfume sniffers – from Kumho Tire itself – an individual who claims to be able (this a real case) to distinguish between Colombia seed marijuana grown in Colombia, and Colombia seed marijuana grown in Florida because he smoked and sold and bought a lot of marijuana over the years, and finally, my personal favorite, an individual who claims to be able to distinguish between the “moo” of a sedate cow and the “moo” of a distressed cow.

Now, all of these things are actually, potentially reliable forms of expertise. Courts should in fact ask the expert, Do you have any proof that anyone could do this? If so, do you have proof that you could do this? The proof could be that you get paid to do it by a private company. It could be that you show the court in fact, here are a couple “moos” from cows, one of whom has been poked with a hot iron and one of whom has not, and I could tell you [correctly] which one has been poked and which one has not. But even if you do that, you have not solved the problem of selection bias, right?

If you call enough cowboys, if you play a tape recording of two “moos” to enough cowboys, you are bound to find some cowboy who is mistaken in this particular case. So ultimately, you are left with a situation where I do not think any of this testimony given the adversarial context actually meets Rule 702. And even if it somehow did, I'm not sure it will be helpful to the jury. What is the use to a jury of having five cow “moo” experts saying this is a distressed cow, and five cow “moo” experts saying it is a sedate cow. Each side has people who have been cowboys for thirty years, and have been around cows. The jury has absolutely no way of distinguishing any of them because there is no objective test to do so. So again, in this context, even more than in the toxic tort context – it will make sense to me when feasible to just say, “Okay, let’s agree on a joint slate of five cow ‘moo’ experts.” And here are your five cowboys – let’s listen. Now if it comes out 3:2, you still have a problem. If it comes out 4:1 or 5:0, that is a pretty good evidence that the other side is not reliable.

So I think that while we know that it has been rough going to persuade courts to appoint experts, I think this context is maybe a lot easier than the toxic tort context. Because in this context unlike toxic tort context, there is not going to be a lot of distinctions in methodology, right? The toxic tort, you always know there are some scientists who always think that chemicals are dangerous, some who never do and there will be this issue of are they truly neutral in some way. Or if it comes to perfect sniffers or cow “moo” experts or even fingerprint experts, there really should not be any particular debate how you go about doing this.

So neither side should really object except on the grounds that we want to control our own case. But there should not be any methodological objections. Even with regard to toxic torts, by the way, if court really starts applying Rule 702 literally, you might be able to make some progress towards more court-appointed experts simply because plaintiffs might get tired of always losing. And they might say, “It will better let the court appoint some experts because we think we have a strong case than to always be excluded because our evidence relies on educated guesses.”

So just to sum up, I say the partial failure of the Daubert Revolution, and again, I think it is a vast improvement over letting any expert who is marginally qualified to say whatever they want, which is more or less what the status of expert testimony was in many jurisdictions in the early 1980s and even the late 1980s to some extent. But, it is also the case that it has not done much to solve the problem of junk science and what I like to call “quackspertise” – a little word I made up so feel free to use it and spread it.

And in forensic science and in the context of toxic torts and connoisseur testimony, it is better to exclude everything than to allow everything. But again, your two problems are that courts are reluctant to exclude everything, so you have some who are falling back and letting it all in. And in the courts that do exclude everything, maybe are over-excluding to some extent. Thank you.

MR. FRANK: Thank you, David. Leading off our distinguished panel of commentators is Edward K. Cheng. He is a professor at Brooklyn Law School where he studies scientific and expert evidence. He has written several law review articles on dealing with the evidentiary and ethical ramifications of having judges do independent research, and then other evidentiary issues. He clerked for Judge Stephen F. Williams of the United States Court of Appeals for the D.C. Circuit, and he was the Searle Fellow at Northwestern University School of Law. He also serves on the advisory committee of the section on evidence of the Association of American Law Schools. Professor Cheng.

EDWARD K. CHENG: Thanks, Ted. Well, I just wanted to thank Ted and AEI for inviting me to be here today. I also wanted to thank David Bernstein. He is both a good friend and in many ways a good sport. David presented his paper at my evidence colloquium at Brooklyn Law School last semester. And so in many ways, sparring with me is sort of round two for him.

So David’s article makes two very important contributions to the evidence field. One is that he asked the question why do we have Daubert – oddly not an issue that the academic literature or the commentary has spent a lot of time mulling over. The obvious response is, well, there is a lot of junk science and we need to find some way of excluding partisan-biased testimony from the parties.

But David raises a very good point which is that we do not do this in other contexts. The adversary system provides us with lots of opportunities for biased partisan testimony, and yet, we do not exclude and police the evidence that is brought in, in those contexts. And so David suggests some of the very special reasons why we might do this in the expert testimony area.

The other important contribution is that he raises the problem of what he terms connoisseur testimony. And this is a problem that has always troubled me; and I think has been long neglected in the literature as well. And here, I think, David has provided us with somewhat of an opening salvo, but for reasons that I’ll get into, I think he is certainly not going to be the last word.

So the two contributions I think raised a number of broader issues, all of which may give us some insight into the potential failures of the so-called Daubert Revolution as the article is entitled. First, with regard to why do we have Daubert? I think we can ask two specific questions. The first is why did Daubert focus on reliability as opposed to sufficiency? David’s section on toxic torts in his article on his presentation today suggested that, of course, junk science was the concern – that plaintiffs were using speculative or suggestive evidence to recover, when they really should have waited for better evidence.

The question is, well, why did the courts not focus on developing these doctrines about sufficiency of the evidence rather than focusing on the reliability of the evidence? In other words, there is a distinction here between this broader question of how much or how should we prove causation in toxic tort cases and the more narrow question, which Daubert asked, which is, What should we use to prove causation in toxic tort cases?

The scientific process – and maybe David Michaels will talk a little bit more about this – but the scientific process in proving causation is a multi-faceted one. You look at various kinds of evidence including the things that are often maligned, like animal studies and anecdotal evidence. I mean, that is all part of the process. Epidemiology sits on the top of that list but you want to have a holistic picture. David in his paper – although he did not mention it today – even talks about these evidentiary mosaics, that if all the pieces of the puzzle fit together, even if you do not have the big rigorous epidemiological study, maybe you can conclude that there is a causal inference to be made.

So ultimately, all of this has to do with scientific judgment and David is surely right that whenever it is about judgment, the idea of having biased experts becomes a problem. And so perhaps court-appointed experts might fix it.

But why, again, should we focus on admissibility rather than sufficiency? By treating scientific evidence as a reliability issue you have all kinds of perverse framing effects. The first thing is atomization. Because you are looking at each piece of evidence or each study or each witness, you are focusing only on individual pieces rather than the holistic picture which you should be doing when determining causation.

And the other thing is by focusing on reliability – meaning admissibility – you end up with this abusive discretion standard that you have in Joiner, so that judges end up having unreviewable or largely non-reviewable discretionary determinations as opposed to having de novo review in the courts of appeal and somewhat of a uniform approach to this issue.

The second issue in this area is why did Daubert focus on the judge as the gatekeeper? So even if the adversarial biased issue concerns you, why should judges be the ones to determine what is reliable and what is not? And in many ways, generalist judges are perhaps the worst place to place decision-making authority. Aside from being well-educated, intelligent people, judges, like most attorneys, probably fled their science courses, which is why they ended up in law school.

So, what are the alternative regimes? Well, one is the Frye Standard, which David has written about – I mean not the narrow Frye Test, which essentially was the let-it-all-in, but a more rigorous Frye Standard where you poll scientists in the area, so some jurisdictions did surveys of scientists to try to figure out what the general consensus was in the scientific community.

The other is, as David suggests, court-appointed experts, and I'm sure that Deborah Runkle and David Michaels will talk more about this so I will not go into it.
 
But, why did the judiciary not move the discretion to make these decisions into the court-appointed experts’ area? And why was it all about admissibility again? So why not have court-appointed experts and present this material to the jury as opposed to having court-appointed experts present this material to the judge so that the judge can make the Daubert decision.

Finally, I want to leave off with connoisseur testimony. And here, as I said before, I thought David’s article was an important jumping-off point. But I think the article reveals some tensions, which will surely invite some future analysis. So David first points to the fact that courts seem to be willing to accept testimony based purely on experience and training even though Rule 702 and the Daubert Trilogy suggest that they should exclude it.

Now, any time this happens in courts where the law or the doctrine differs from the actual practice or norms, courts are either unaware of the problem, which I think in this area, because it has been hashed over so much, that is not likely to be the case. So courts are likely to be worried about something, and I think that something is that connoisseur testimony can be potentially extremely accurate and helpful, even though it is irreducible to a statistical study or some kind of rigorous process. So David mentioned the marijuana expert. There is some intuition that maybe this marijuana expert has something to offer that goes beyond your typical chemical analysis in a crime lab.

David, in his paper, also seems to be conflicted about that decision – although he might disagree. He makes a bold statement saying that 702 demands exclusion of the connoisseur. He says that in toxic torts, we should reject clinicians that are not medical researchers, clearly suggesting a preference for these kinds of systematic studies. But on the other hand, I think the paper expresses somewhat of an uneasiness of depriving jurors of the “cow moo experts” or, David also brings up, the chicken sexer, the person who can look at chicks to determine whether they are male of female just by looking at the chick.

And the other thing is – remember that medical clinicians – diagnosticians are largely like cow “moo” experts and chicken sexers. They apply their training and experience, and they do not necessarily have a cook book that tells them whether or not you have one disease or another, or what might have caused your disease.

So the tension leads me tentatively to this conclusion, which is that the legal system should preference against connoisseur testimony, preference instances where you have a systematic method for arriving at a conclusion rather than the black box or the intuitive method of the expert. And why might that be? Well, maybe that is because there is too much of a danger for this ipse dixit or black box type testimony, particularly in the adversary context. There is no way that you can tell whether or not a particular expert is influenced by the parties paying them or influenced by other sort of factors which you do not want to have influencing their judgment.

So in conclusion, I just wanted to thank David for a very provocative piece. I look forward to what the other panelists have to say in the discussion that follows.

MR. FRANK: Deborah Runkle is the Senior Program Associate at the American Association for Advancement of Science where she is the project manager of the Court Appointed Scientific Experts Project that assists judges by identifying highly qualified and independent scientific and technical experts who will work for the judge. She is the Associate Staff Officer at the National Conference of Lawyers and Scientists and the American Bar Association Science and Technology Law Section as well is a member of the leadership group of that ABA section. Ms. Runkle.

DEBORAH C. RUNKLE: Thank you. I am not going to disappoint Ed Cheng about what subject I'm going to speak about. And thank you very much, Ted, for the opportunity to comment on David’s very thoughtful paper.

Although it was not the main topic of his paper, David does discuss the use of parties’ experts versus independent or court-appointed experts. I plan to spend my time discussing this topic with the particular emphasis on independent experts and AAAS’s project in this area.

But first, for those not familiar with us, let me introduce our society. The American Association for the Advancement of Science, or AAAS as we like to be called, was founded in 1848 and it is the world’s largest general scientific and engineering membership society. With our 262 affiliated scientific and engineering societies and academies of science, we serve 10 million individuals.

Several years ago, we launched Court Appointed Scientific Experts (CASE) – that is the title of our project – with the goal of assisting judges in identifying experts to serving cases with the significant scientific, technical, or health care component. I’ll speak about the reasoning in taking on this effort, which often agrees with the points that David has made; and how the project has worked. But first let me point out that this project was not designed specifically to weed out junk science. I have no idea how much junk science there is out there.

And I consider, perhaps in opposition to David, Peter Huber to be an unreliable source considering the withering and unanswered criticism of him from Ken Chesbro. In other words, polemics are not data. I also disagree with David that the level of quackery in the scientific communities is an important issue and problem regarding expert witnesses. But I'm a scientist; I ought to be defending scientists.

Here is what I do think: counsel-retained experts will support their client’s positions and reject candidates whose views are more nuanced or balanced. In fact, parties’ counsel may retain experts as consultants and if their opinions are not considered advantageous to their client, the lawyer will search for another expert. In short, obtaining the most informed scientific opinion is not the lawyer’s goal. Rather, the lawyer will focus on obtaining the best witness – the person best able to advance his client’s cause rather than the best scientist. And this is what David has labeled “selection bias.”

Next, even highly qualified and respected scientists could become advocates for the side that hired them. The close collaborative relationship between lawyer and expert promotes a feeling of unity of purpose. While this phenomenon may enhance the likelihood that client interests will prevail, it also increases the likelihood that the testimony will be one-sided and misleading. Given that the lawyers on both sides in litigation followed the same process of expert selection and preparation, it is often the case that neither side will focus on all the information necessary for a complete understanding of the disputed scientific issues if that evidence might hurt their client’s case.

Rigorous cross-examination is not always an antidote to one-sided or faulty testimony. Cross-examination may often be conducted in a disjointed manner, jumping from subject to subject, and is often intended to catch experts in errors or to make them look foolish or unprepared. Further, cross-examination naturally emphasizes only the weak points of the testimony of the other expert, eliciting admissions of uncertainty or fallibility even on points that have little impact on the expert’s overall conclusions. Finally, there may be real weaknesses in an expert’s testimony, but the opposing party’s lawyer does not want to highlight because it might damage his or her own client as well.

For the above reasons, it is possible that judges and/or juries will never hear a comprehensive, logical, and coherent explanation of the scientific or technical issues that may be at the heart of the litigation. Also, even in those instances where two highly competent and honest experts testify, they may reach opposite conclusions, and non-scientist judges and jurors have difficulty in determining which expert’s testimony should carry more weight in decision making. We believe that court-appointed experts could be a partial solution to these pitfalls, although we do not envision them replacing experts retained by the parties.

Here is some of our reasoning about the benefits of court-appointed experts. The testimony or advice of court-appointed experts is more likely to be objective. Although scientists are hired to serve the court will undoubtedly have prior opinions on some of the scientific issues in dispute, because they are working for the court, not a party to the litigation, the application of those issues to the facts of the case is more likely to be disinterested.

The problems arising from the battle of the experts, two experts presenting opposite opinions or interpretations of scientific evidence will be more easily resolved. Judges can, for example, ask their experts to evaluate the testimony of the opposing experts’ parties pointing out the strengths and weaknesses of each side’s arguments. Reputable and highly qualified scientists often reluctant to testify on behalf of the party, even if they support that party’s scientific position, may be of service to the judiciary as an important public service.

The appointment of a court expert may, in itself, affect the quality of the testimony of parties’ experts. Knowing that their testimony will be scrutinized carefully by an expert retained by the judge – particularly one suggested by a highly prestigious scientific organization – partisan experts may modify their positions, adapt a more balanced view than would otherwise be the case.

Similarly, the appointment of court experts can promote early settlement of a case. The knowledge that an independent expert will examine the scientific testimony and conclusions of their experts may cause counsel on both sides to reassess the potential weaknesses of their cases and decide that a prompt settlement, even before the judge’s expert issues a report, is in the clients’ best interests.

Supreme Court Justices Harry Blackmun and Stephen Breyer have both cited use of court-appointed experts approvingly. In Daubert, Justice Blackmun noted that the Rule 706 allows the court, at its discretion, to procure the assistance of an expert of its own choosing. And in his concurring opinion in General Electric vs. Joiner, Justice Breyer, quoting an amicus brief of the New England Journal of Medicine, wrote, “A judge could better fulfill his gatekeeper function if he or she had help from scientists. Judges should be strongly encouraged to make greater use of their inherent authority to appoint experts.” In this concurrence, Justice Breyer also cited AAAS as an appropriate organization to fill this role.

Although not used as often as I would like, the responses to CASE have been highly positive. Judges who have used the service have expressed satisfaction with and appreciation for the assistance the project has offered. Additionally, the judges have, with only one exception, been pleased with the experts who have served the court. We get many requests that do not lead to an appointment, often because the parties succeed in objecting to this process, even though judges do have the authority to appoint judges without the permission of the parties; and because most of the cases settle after AAAS begins the identification process. Importantly, all of our experts have enjoyed their service and would do it again if called on.

The courts requesting assistance represents seventeen different federal districts, the Court of Federal Claims, one state court, and one administrative law court. The types of cases for which assistance was sought: patent, environmental, product liability, construction, health care, breach of contract, antitrust, electric utility rate determination, personal injury, and a criminal case involving digital evidence are highly diverse, as are the types of experts sought: econometricians, rheumatologist, obstetricians, hematologist, chemist, environmental scientist, epidemiologist; and mechanical, optical, electrical, telecommunications, computer, architectural, and structural engineers.

CASE experts have assisted judges in a number of ways. By evaluating the testimony of dueling experts; by writing a report without follow-up deposition or trial testimony; advising on rulings involving electronic discovery or electronic testimony; conducting an independent evaluation; advising judges on names of potential experts submitted by the parties; consulting with judges after the appointment of an expert, or with judges who are appointing experts outside the scope of the project.

Here, in conclusion, are a few favorable reviews from judges. Two who used AAAS-recommended experts, those are the two on the top; and one who consulted with us extensively about appointing an expert in a highly visible case but ended not doing so on the condition that the parties not put ideologues on the stand.

And I would like to read what that judge said because I think it is important and it emphasizes our goals. He says, “Please let me take this opportunity to personally thank the American Association for the Advancement of Science for your assistance in describing how your project works and offering to be of help. Your assistance allowed me to offer to the parties a totally objective examination of the case by qualified and competent experts. For those of us who care deeply about the institutional integrity of the courts, the ability to make such an offer to the parties proves that willingness of the courts to get at the truth. The CASE project is an incredibly valuable tool to federal judges. In the future, I hope the project continues to serve the interests of science and the federal judiciary.”

Now finally in real conclusion, while David cited at least eight articles discussing court-appointed experts, all of those articles were either theoretical and in some cases, statistical. But he did leave out the only scholarly work on how the process actually works. So here is the citation to that article. Thank you very much.

MR. FRANK: Thank you. Our next speaker made a very strong sacrifice to be here today on his wife’s birthday. David Michaels is a research professor and acting chairman of the Department of Environmental and Occupational Health at George Washington University School of Public Health. He is also the Director of the project on Scientific Knowledge and Public Policy (SKAPP), which published two or three years ago a report on Daubert with an interesting perspective. He is a former assistant secretary for Environment Safety and Health at with Department of Energy, and he has won awards from the American Public Health Association and the Department of Energy. Dr. Michaels.

DAVID M. MICHAELS: Ted, thank you very much for inviting me. It is an honor to be here and talk about this paper and to be on this distinguished panel. This is my first time speaking at the AEI. 

As Ted mentioned, I direct SKAPP, [… and all of our ] papers are downloadable online and usable without any fees from our website defendingscience.org.

Let me say I enjoyed very much this paper, and I think I was primarily brought in to be the most skeptical and actually agree with much of it. I certainly share some of your concerns and I have always felt that the science I do not like should be excluded from courts. But the problem of course is no one may agree with what I think. I think that all of us who probably take the same position, you know, one man’s science which is good is the other man’s junk science.

I think the easy part and I certainly share all of your concerns about the quality of the science in the criminal justice system. In fact, to call forensic science a science in some of the instances that I think you have raised and others raised is really very generous phrasing. The things that we think of as science – even fingerprinting gets into that area – do not fit the reliability rules that any of us in the scientific community might like to apply. And really, what it all comes down to in the problems in the forensic sciences more than anything else is the issue of money.

The problem is that indigent defendants have no real ability to use the mechanisms of the court, Daubert being one of them, to challenge science that is faulty, fraudulent or even non-existent.
This calculus changes with wealthy defendants and you could see the example in the most egregious example of faulty fingerprint identification was this attorney, Mayfield, a couple of years ago who was arrested and publicly taken away in handcuffs because he was implicated in the Madrid train bombings. Obviously, everyone knows the story. The fingerprinting was wrong and several rounds of fingerprinting were wrong. Now, if was an indigent defendant in some other case, he would be in jail. I do not think anyone has a doubt about that. But he was able to essentially mount a case to show that this was absolutely wrong.

Another example is Houston Crime Lab, if O.J. Simpson had been arrested and the Houston Crime Lab had provided the forensic testimony that was used by the state, in 1995 that forensic lab would have been closed because OJ Simpson had the resources to show problems in that system. So money is the problem, and we will come back to that issue, because I think money is a big issue in Daubert across the board.

But I want to return now to what I think was really this sort of – basic question was the question of dueling docs, the adversarial system where you have, as everyone recognizes, both plaintiffs and defendants choosing scientists who take the most extreme positions because they are most successful in court. It is an imperfect and inefficient system but it reminds me of what Winston Churchill said about democracy in 1947, “It is the worst form of government except for those other forms that have been tried from time to time.”

The alternatives suggested by David and obviously, in action by a project that I have heard respect for, the AAAS, that judges should select non-partisan experts: I'm sure it can be successful from time to time or even in many cases, but it is also extremely problematic. And there are a couple of examples which I have gathered. Again, this is an anecdote; there are no big studies on this. I'm glad Deborah has published on this.

I have spoken with Ed’s colleague, Margaret Berger, about this at some length at Brooklyn Law School. Because Margaret had the honor of chairing the panel that selected experts for the judge in the silicon breast implant case. She is very well-noted, and to quote her directly, she called that system of selecting independent experts, “A most unpleasant experience.” I think that was an understatement on her part. But if you talk to her about it, certainly, it was a terrible, bad idea, and it could not be done.

Essentially, here are the reasons. One is it cost a fortune. In addition to the fees for the scientists, and obviously, they were not low because you have scientists really looking at new data, lots of data to look at. Because they were involved in actually more than advising the judge, they were then subject to cross-examination. They needed attorneys. The attorneys for the experts chosen by the judge ran to over a $1 million in costs, and obviously, who pays for those costs is a big question. As Margaret said to me, “You can’t throw them to the lions. You have to get them attorneys.”

They were doing the court a favor – as Deborah has really said, people really want to help out and I think justice is an important thing, so they say, “Yes, I’ll help.” But then once they get involved in this adversarial system, this is a strange world for physicians and scientists, where during cross-examination your statements are turned upside down and little pieces are taken out of context. Scientists who are trying to get to a consensus, this adversarial world really frustrate them, and they get very angry and they say, “I’ll never do this again.” And that was a huge case, and obviously it gets nastier the larger the case is because there is more at stake, but you would not want to do it for small cases because it costs too much.

The other thing is it is extremely hard to find independent experts because if you have an opinion about something, even Deborah referred to people who really want to help out and they have been working on this area. In many cases they have already been paid by some party or some connections to some party in the case. Scientists often do not believe that it has anything to do with conflict of interest and will tell you that they are not conflicted. They firmly believe, in their minds, that they are not conflicted. But in terms of the way courts work around conflict of interest, they are conflicted.

And what Margaret has said many times these scientists when you ask them who paid various things through your department or whatsoever, they do not even know. And so, finding the independent experts is certainly difficult. The other alternatives is to find really smart people who have not yet developed an opinion on the case, but that will cost you more money because they really do have to spend some time reading the literature to come to a conclusion for themselves and also that requires a lot of time.

In general, most people in the trial system like to wait until last minute to hire someone because they think the case may go away or be solved. So all of a sudden, if you need someone to provide you a report, and you have got two weeks or six weeks to do it, you cannot get a top scientist to give you that report in a two weeks or even in six weeks; you need six months because they are busy.

So in abstract, the idea is great, and it has been around for a long time, but it is problematic if you want to get if you want to get them involved in the case directly. The alternative is to get them to give advice to the judge, and Joe Cecil has written about this actually in this issue that I brought along. There is a case called Soldo, a Parlodel case, where the judge said, “Okay, I'm going to choose some independent experts and they will give me advice.” This was a woman who had an intracranial hemorrhage which led to a stroke right after giving birth. She had taken the drug Parlodel, which stops lactation. So the judge hired an epidemiologist, a clinical pharmacologist, and neurologist to say, “Is the testimony [as to whether] there is a causal relation reliable?” Well, one said yes, and two said no. These are all three leading people their field.

Now, the judge in this case threw it out. Of course, he thought two out of three was fine. You said two out of five would not be okay, but there is no right way to do this. You are saying the clinical pharmacologist who the judge chose, you know, is a junk scientist because there is little conflating, so they are sort of the science I do not like. It is unreliable science. Some of this epidemiology is not in junk science are all used the same way. And that just does not work.
I'm not saying there is a great way to do this. But that idea, if the judge chooses one, has the judge made the right choice? The judge chooses five, do you vote? Five out five, maybe I would go with the two.

So I truly think that Daubert has worked in terms of getting rid of what I think is junk science. I think the medical ecologists and the nuts who – I do not know if they ever won their cases, but they certainly in the position to get them into the court and not fair to the defendants – I think those cases probably have been eliminated by Daubert. But the once that are out there now really – there is a question and this idea of keeping and using the phrase “junk science” I think is really is a way to keep people out of court because what David is suggesting is the only reliable evidence is a statistically significant two-fold excess using epidemiology. And as Ed said, scientists do not work that way. You look at the whole picture and say, what does the epidemiology say. There are lots of epidemiologic studies that have more than two-fold excess, but do not reach the statistical significance. Statistical significance is a convention we like in epidemiology and science that says, “We think our result is not the result of chance.” Nineteen out of 20 times – but that has nothing to do with what the courts want.

It raises an interesting question which maybe I could throw out. We have in our audience Doug Weed, who is really one of the preeminent thinkers about epidemiologic theory in the country. And there is something I have always wondered about, if we really are going to say, as David does, that epidemiology trumps everything else. And if you have a two-fold excess that is reliable and really nothing else is reliable. That is what he is saying and there.

Well, there are lots in the epidemiologic studies where you could have a four-fold excess that is not statistically significant, but if you look at the distribution of likelihood of that particular result, a two-fold excess or 2.1-fold excess is more likely than not. The P value may be 0.1 or 0.2 or 0.3 or 0.4 but essentially [the probability that it is the result of chance is more than one out of twenty], but at the same time it is more likely than not, which is my understanding. And again, I'm not an attorney – but is what you are asking for in court to say, is the result more likely than not? And you could say, in fact, it is more likely than not that there is a doubling of a risk from the study, even though it [may not be] statistically significant. So I think that might be actually an interesting discussion.

Let me go back to the very final point around Daubert and money, which is where I really began. My impression of what Daubert is doing now, having successfully gotten rid of the trash, […] is now saying, Daubert challenges are inevitable. In fact, some attorneys have told me it is legal malpractice for a defense attorney not to raise a Daubert challenge. I have seen physicians and scientists, who I think are the top people in their field—the biggest lead poisoning expert in New York, John Rosen got Daubert-ed in a Metropolitan Life case, and the judge laughed it out of court—but at the same there was a Daubert challenge that raises the cost of every case to the plaintiff. And what that means is an attorney who works on a contingency basis is going to say, “Well, I'm not going to take this case on unless it is worth a lot of money.” So a person who has a case that may be a legitimate case, but it is not worth a lot of money is not going to get justice. And I think that is a big problem.

All right. Those are my sort of random thoughts. Again, thank you for inviting me.

MR. FRANK: Thank you Professor Michaels.

Too often in panels like this, we hear the academic side and we do not hear the practitioner’s side. We have rectified this problem by getting somebody who has actually experienced in the adversarial area, and one of the best at it. Joe Hollingsworth specializes in trials and appeals and leads a practice group of 60 attorneys at Spriggs & Hollingsworth, his own firm, with over a hundred opinions arising from his cases in federal and state reporters. He has been recognized by the National Law Journal three times in the year’s top ten defense wins. He is working in cases involving pharmaceutical and medical device product liability, toxic and environmental torts, and consumer product liability, and just a long list of accomplishments and achievements here in his biography in the package you have. I'm just going to turn you over to Joe.

JOE G. HOLLINGSWORTH: Thanks so much, Ted. That is very kind. It is unusual for me to be in front of a group like this. I am in my 34th year of private practice, and I do not often get to read articles by law professors who comment favorably on our firm’s work or on the work of our clients. This article, however, talks about six or seven of our cases, and it cites favorably and approvingly to the five or six that we won, and cites unfavorably and indeed disparagingly about the judge in two other cases in which we lost and in which the judge ruled against us under Daubert. So, I think this is the virtually perfect law review article in my 34 years of experience.
 
I want to focus, of course, on the part of this article the middle part that deals with causation, testimony, and toxics in pharmaceutical cases, and I have my own views about a lot of the issues that you have heard about already this morning, and I differ in a number of categories.

One is the issue of what is the philosophy of Daubert and why do we have reliability. Well, I think that we have reliability on the issue of admissible proof or not because that is the way the English jurisprudence system has been set up for 400 years. We want to get evidence in the front of these lay jurors, which permits them to make the right decision in these cases 80 or 90 percent of the time. We do not want to have the situation, which we had before Daubert, allowing lay jurors to rely on junk science where it is a jackpot justice system, which is what it is and was in West Virginia, Mississippi, Texas, etc., and still is today in places in my opinion where the Daubert standard does not apply.

That is why we had the rules for 400 years against hearsay. We want the most reliable stuff to go in front of these poor 12 people or six people who have to make these very difficult decisions, and we want them to make the right decision 90 percent of the time. That is why we have this system that we have.

So reliability is a very important component of this system, and reliability is as the Supreme Court said in the fourth case – there are four cases which address Daubert not just three – a very, very strict standard. It is an exacting standard. And the reason reliability is such a high threshold in cases is that unlike the question of sufficiency, with all due respective to Professor Cheng, it is applied at the stage as to whether or not evidence is going to go in in the first place.
The sufficiency test gets applied to the admissible evidence at a later stage in the proceeding and sufficiency in a civil case is only a 51 percent threshold. But what the Supreme Court said – and if you listen to questions by then Justice O’Connor and Justice Scalia during the three arguments of the Daubert Trilogy – you get a very clear philosophic impression and basis for at least those two justices’ view of admissibility being a very much higher standard and stuff that comes in in the first place, especially if it is from an expert, ought to be about 80 percent reliable, so these jurors will make the right decision. 

I think that aside from Daubert itself, the most important opinion in all this case law is Joiner, the second case, and Joiner is important. That is Joiner v. General Electric. It involved exposure to PCBs and whether or not PCBs caused a certain kind of cancer. What Joiner added to the doctrine is that not only are judges required to determine whether scientific medical causation proof is reliable, but also whether it fits and in that case the plaintiffs were relying on animal studies involving a different kind of cancer endpoint than was the plaintiff was alleging in the case in extremely high doses, which Rehnquist and the others in the majority – it was an eight to one decision in which Stevens only dissented, by the way. And whether or not other aspects of the case like the epidemiologic studies that were involved specifically in the case actually fit the case of the plaintiff and the epidemiological studies were not very good, frankly, but they involved other cancers, too.

By the way, another thing that Professor Cheng’s comments brought to mind for me was this issue, whether or not, the overall mosaic or the combined amount of scientific evidence is sufficient and whether or not just the individual pieces of the evidence should really be probed by judges. I disagree with Professor Cheng and his colleagues at Brooklyn Law School to the extent that they believe that the overall mosaic and the overall number of bricks in the wall are sufficient taken as a whole rather looking at the evidence individually as required under Daubert. If you look at Justice Steven’s dissent, he takes the position, which Justice Rehnquist specifically rejected in the majority opinion, that the mosaic and the evidence as a whole should be the basis for admissibility or not, rather than looking at the individual pieces of scientific evidence that the plaintiff’s causation experts rely on the first place.

The third thing I wanted to mention is the references in David’s really excellent discussion of causation proof and toxic tort cases to the Parlodel cases. David also brought – the other David – brought up the issue of the Parlodel cases. Those also were cases that we tried and litigated under Daubert in state and federal courts for over 10 years. And they ended up with what we refer to as the Parlodel Trilogy in the circuit courts, the 8th, 10th, and 11th circuits, and in cases David cites called Glastetter, Hollander, and Reiter. And they stand for the proposition that courts ought to look at the individual pieces of evidence and whether or not each individual piece of evidence is sufficient to support, in a reliable way under Daubert, an expert witness’ causation opinion. In those cases, by those three circuit courts come down almost uniformly that epidemiology is the most important piece of evidence, but where epidemiology is not statically significant, which it was not in the case of Parlodel, a lactation suppressant, by the way, which has been removed from the market, by the way. So, they were very difficult cases. But they had very extremely wide confidence that arose from the epidemiology, and as I said, it was not statistically significant.

And in that case, in that situation, those three opinions, from those three separate United States Circuit Court cases did – as David says in his paper – look specifically whether the animal studies were reliable and fit to the evidence of issue, which they were not. Whether or not case reports and case series, anecdotal reports in other words were sufficient which they were not because case reports are not controlled, whether or not laboratory tests were sufficient to show that this drug caused vaso-spasm or vaso-constriction in cerebral arteries or elsewhere in animals. In fact, the laboratory studies showed the opposite that this drug was a vaso-dilator.
So, those studies did not support expert witness’ opinions. Whether or not actions of the FDA were sufficient, which of courts held they were not, because those things are not the type of evidence that scientists rely on in the laboratory and whether or not internal documents of the corporation (every corporation has lots of bad documents in all these drug cases) in and of themselves could support a causation opinion of a plaintiff’s experts were held uniformly in these cases not to support an expert’s opinion. They might support negligence, breach of duty. But they did not support an expert witness’ opinion because they are not the kind of thing that the scientist relies on in the laboratory.

The two cases, that I said that we lost that David refers to and correctly, unfavorably in his article, were by a judge who held that because epidemiology in these Parlodel cases was either not there or not significant or just impractical to do because they were so expensive to undertake at that point in the litigation, that the best available evidence – these things that these other aspects of the evidence that I have already alluded to – were sufficient, made a determination that they were sufficient or that they should be admissible on the basis that they were “the best available evidence.” Justice Scalia will laugh if he heard that kind of analysis or had a chance to look at it because the test in Daubert is not best available. It is whether or not the evidence is reliable in the first place and if the evidence is not available there to the plaintiff’s cases, then it is just too bad. That is what Daubert is about.

David, the other David refer to the Soldo case, which was another Parlodel case, which was a case in which we had a two-week evidentiary hearing before a federal judge in Pittsburgh and in which the judge ultimately decided to select the panel. I have my own views on that. I guess that is one of the only parts of part two of David Bernstein’s article that I disagree with, and that is that these kinds of disputes should be handed off to panels of experts. The judge in Soldo did that. He still wrote an opinion, which conformed to the Parlodel trilogy right down the line. But guess what, by the time he had written that opinion, he not only had two out of the three experts inclined not to admit the causation testimony, but he also had all three circuits coming down on our favor at the same time.

And then, finally, on the issue of whether a doubling of the relative risk is sufficient or not, if you do have an epidemiology study that is statistically significant, and whether or not that should support an expert’s causation opinion in the first place, I say no. I say that the reliability – not sufficiency – test is much a higher one. It is back up at the 80 percent level that I referred to from the beginning. The kind of a level that will allow people to make the right decision in the right circumstance, lay jurors 90 percent of the time and a mere 2.0 doubling the risk does not do that. What that does is conflate sufficiency with admissibility in the first place. And while 2.0 may equate to a more probable but not that kind of conclusion based on the epidemiology study itself, it really ought to be a significantly higher relative risk than 2.0, I think, to support the admissibility of an expert’s opinion on causation. Thank you.

MR. FRANK: Thank you. David, do you want to response to what you have heard?

MR. BERNSTEIN: Oh, sure. I'll try to do it briefly. With regard to epidemiology, I think I agree with both Joe and David – who actually in some way agree with each other, oddly enough – that merely having an epidemiological study of 2.0 statistically significant is not enough to prove causation. I go into this more in the paper, but you should also look at the Hill guidelines, which is sort of an evidentiary mosaic in a sense because it is a look at this one thing, it does not really help you if you have one poorly done epidemiological study with a relevance greater than two, but you also have inconsistent studies or there is no plausible biological mechanism for it to happen.

Speaking of Peter Huber, I remember when I started looking into the breast implant litigation, I wanted to use that as an example, in Wall Street Journal op-ed that I wrote after Daubert, as an example of the kinds of things that should now be excluded, but I was not that up on the science. I asked Peter about it. He said, “You know, David, people who are diabetics get injections everyday with needles that are coated with silicone and no one has ever thought this is a causative problem. Why do we all of a sudden the problem with the woman who have a less silicone leaking into their body than people who get vaccines?”

So, same with electromagnetic fields. Field experts say, “Well, the Earth’s electromagnetic field is so much stronger than what you get from a cell phone or a power line. If that is not causing leukemia, why would the power line by your house do it?” So, there has to be some logic to causation to collaborate up epidemiological studies that are not that well done. So, everyone who says they meet the Hill criteria also and are consistent and what not will be top-notch evidence.

On the other hand, I agree there are situations, this where I may disagree with Joe, where not having epidemiological evidence would still in theory would at least be reliable. There is a guy named Karl Kramer who I often disagree with on this. He wrote a book recently with a couple of pages where he gives examples of situations in which the scientific community was able to establish something without that epidemiology. I think these are relatively rare, but the problem if you are the judge is that you do not know if you are in a situation, and that is why judges I think are often hesitant to exclude evidence.

One example I always knew about, and I do not remember his other examples but they are pretty persuasive, was thalidomide. Thalidomide has such a strong effect on birth defect. It was also so quickly backed up by animal studies. It was not as good as having the epidemiology, but there was a point where you could say, “Yes, we are convinced that thalidomide is causing birth defects,” even without having the epidemiological evidence.

That is why if you are a judge, you know about examples like this. You may say to yourself, okay, we have these breast implants or we have Parlodel or we have whatever. We have a bunch of experts who are well qualified who are willing to say, this is enough. So, is this a thalidomide kind of case or is this more like of clinical ecology kind of example? And while I agree that just putting together a bunch of bad evidence does not necessarily make good evidence, if you have someone who is a nonpartisan individual who does not have obligation, and you have got several people say, “We have examined all this and this is one of those rare cases where you can prove causation without having the epidemiology,” I think judges should attend to that.
 
Again, I do not think this is going to happen very often. But I think the fear among the judges of this being the thalidomide example, or this being one of those examples where there is enough evidence out there, I just do not know if the judge will exclude it. Actually, it works against Joe’s clients in a sense because the judges will again try to evade a Daubert rule.
I mean all judges, as we know, are not happy with Daubert-Joiner rule 702 to begin with. The 11th circuit, for example, they said, “Okay, well the Court in Daubert says it has to be reliable. But they meant only methodology, we shouldn’t look at reasoning.” And then Joiner shot that down. Then the 11th said, “Okay, well, we are only going to limit the truly scientific evidence. Everything else, we will just let it all in.” Nope, Kumho Tire shot that down.

So in lot of courts, they were implicitly or explicitly resisting the changes. And I think in a sense what we want to do is we want to try to conform what we are doing in the court room to as close as to what the scientific community will do itself because I think when you are doing that, judges will become a lot less reluctant to exclude evidence, when it should be excluded, and to admit it when it should be.

I had mentioned there are other tests out there like Frye. […] Ultimately, in a sense, I think Daubert erred in a way by asking the judges to play doctor, basically, to play scientist, as opposed to coming up with standard that was more of, again, not the old-fashioned Frye but a sophisticated, modern and general acceptance standard that said: Okay, how does the scientific community approach this? They would not approach it by selection bias, by cherry-picking the one expert who happens to be with the plaintiff. They would try to reach some sort of consensus and then deal with it that way.

So, in a sense again, it is better than what we had before, but I think there might have been a wrong turn in focusing on reliability per se as opposed to and trying to get the judge to figure out what is reliable as opposed to how the scientific community approach looking at the evidence of this sort.

MR. FRANK: Do any of the other panelist want to comment?

MS. RUNKLE: Yes. I would like to say something about the process of finding court-appointed experts. I know Margaret had a very difficult task – a panel was appointed to find a panel was, essentially, one of the problems. And we do, too, have troubles finding experts. Conflict of interest is often the case, every once in a while unwillingness to serve, but not very often is the case. I have not found it is strenuous or difficult as Margaret has even though we have had real problems.

I want to get to that case though, where the judge called three experts and he got two yeses and a no or two no, yes or something. I think the judge made a very bad mistake and I actually said so at that time. One of his rules was that the experts not be able to speak to each other and each issue an independent report, as I understand it. Am I correct?

MR. HOLLINGSWORTH: I think that is right.

MS. RUNKLE: Yes, that is my understanding. And I think that is not well advised. That is not the way scientists work, and if we followed David Bernstein’s last comment that we should try as much as possible to follow the way scientists work, when scientists write a joint paper that includes people from different disciplines, they get together and discuss it because one of them might learn something from the other discipline that he or she did not know before, which will make the joint paper more informed. And I think that is the model that that judge should have taken that there should have been a joint report, although each one could add some comments to it, and they should have been able to do their own work, and then get together and meet and exchange ideas and information.

So, I do not consider that a failure. I think that is just a badly organized way to do court appointed experts. And by the way, the breast implant case that is not how they did it. They each wrote a report, but then they got together and agreed on a final report and discussed what they knew from their various fields.

MR. MICHAELS: Well, let me just add, in the Soldo case what was interesting to me, if you read the opinions of the three experts, was that they each looked at the world through their own prisms. So, the epidemiologist said, “Well, there is no good epidemiology. Therefore, it cannot be causal.” The clinical pharmacologist said, “Well, they are very good clinical pharmacological studies, so it is causal.” And this is the issue when, in fact, if you put together a team you are better off, but on the other hand, how you stack that team – and I don’t mean “stack” as a pejorative word – but who you choose to be on it is going to have a big impact on it. It is tough. It is a great idea. But it turns out that I do not think it really gets you that much farther than really lining up the experts and letting them argue it out.

The other thing it is worth just clarifying the Hill criteria are really about this mosaic of evidence. In some ways, it is the same thing. The Hill criteria is not how you look at epidemiology, it is how you look at the evidence.

MR. HOLLINGSWORTH: Well, no. If you read Sir Bradford Hill’s introduction to the Hill criteria, what he says is, “You do not start applying my criteria until you have statistically significant epidemiology in the first place showing an association. And then you apply my criteria to determine whether that association may or not be causal.”

MR. MICHAELS: Well, that shows you how people could read the same things and totally disagree. I read his as saying things differently. For example, you have to look at the animal evidence, you look at the different studies, you look at the biological plausibility, what is your theory behind it, and you look at the mosaic of evidence. So, anyway –

MR. FRANK: Joe, do you have a thought on the breast implants? Do you think that was a system that worked or is there something that could have been done differently there?

MR. HOLLINGSWORTH: I watched that from afar, but watched it closely and I have spoken about it. My opinion is that a grave injustice was done in that litigation to the corporate defendants, half of whom are out of business now, given the fact that the data turned out to be just totally in support of the notion that breast implants and the leaking of silicone or not had no relationship whatsoever to immune dysfunction or immune system disregulation. I think that was due not to what the expert panel did, but mostly to the way that judge in the case handled the litigation. It was a complete debacle from the standpoint of how decisions got made based on the existence of reliable or not, science, and in that case, at least at the start of litigation before the expert panel got involved, a lot of science got great play and then, of course, made its way into the mainstream media. That was a classic case of inadequate junk science that never should have been admitted, but the truth about the science and the “junkiness” of it came out too late.

MR. CHENG: That’s a really interesting that Joe makes, which is that in the breast implants – as you do in many toxic tort cases – you end up with this timeline where you have the junkie suggestive science early, and then you have to end up relying on the epidemiologist doing their long-term studies which take years to do. If you make decisions too early, that poses a problem, and that is why you have the problem in the breast implant case because really the courts should have waited around for about four or five years until the better studies came out.

And I think the problem with making it a categorical rule (and so I agree with Joe on the breast implant cases) […] is that there are some cases where you are not going to see those epidemiological studies because you cannot ethically give people things that you think are toxic. And so, the question is how does the legal system deals with those cases where you have uncertainty and you have some evidence, but not particularly good.

MS. RUNKLE: When I looked at the breast implant cases, I of course looked at the experts, and I would attend every session of Judge Pointer’s proceedings, including I was allowed to be in on the depositions. I think the experts, at least some of them, at least three out of four, were very disgruntled. And I think part of the problem was that they were not prepared for what a deposition is like, and what kind of questions you maybe asked, about data retention policies and things of that sort.

We have learned from that. And we have tried to ameliorate that by having written a handbook for experts, which alerts them to these things and a handbook for judges, which advises the judges to discuss with the experts what a deposition is like, what kind of questions you may be asked, because we have found some of our experts – one of them was so scared of the deposition that he wanted to drop out of the case, for example, and the judge was able to talk to him back into the case. So, I think there needs to be more work with the expert than was probably done. I do not mean to be personal, but I do not know why their lawyer did not have them better prepared for what they should be expecting.

MR. BERNSTEIN: Ed has written a paper about judges consulting with outside experts, and I don’t know if you addressed this or not, but could you ask an expert for advice without telling the parties and having them potentially go through deposition?

MR. CHENG: So, I think you are talking about technical advisors. I guess you do have to tell the parties. The question is whether or not that expert is then subject to cross-examination, right? So, if you appoint a court-appointed expert, then, under 706, you do have to offer them up for a cross-examination and for deposition. But if you have a technical advisor, this advisor serves largely as a law clerk. It is a special law clerk that you hire to help you as a judge think through the problems.

I think there are sort of certain problems with this because you have due process issues. You have possible ex parte communication issues and so technical advisors are somewhat controversial.

MR. HOLLINGSWORTH: The biggest problem is you do not have a record. And if you do not have a record, then you do not have a right to adequate appeal. That is my main problem with it.

One further thought is this: Daubert does deal with the situation where a plaintiff’s evidence is not sufficient at that time even though it may be sufficient three or four years later to be admissible and it says, too bad. Too bad for the plaintiff. And other courts including, Judge Posner in the Seventh Circuit, have interpreted that to mean that law lags science, it does not lead it and it should not. The judges and juries and lawyers and lawyers’ experts and everybody else in the legal side should not lead science. Science should be making a decision and if it takes science three or four years to make the right decision and a plaintiff’s case comes before, that is too bad, and Daubert specifically says that. The rules of evidence are there to make a decision now, today, based on whether or not something is admissible in a trial. And if it looks like it may be a promising theory, but the evidence is not there yet, the principle, one of the fundamental principles of Daubert is that is too bad. And a lot of law professors and others have a great problem with that.

MR. CHENG: Well, yes. But think about the deterrence problems with that or the efficiency problems. Assuming the statutes of limitations bar the plaintiffs after a particular period of time, if you have a particular percentage of cases that the injury seems to occur early, but then the plaintiff cannot bring enough of evidence early on and then you bar those claims, you will lack optimal deterrence, because at the end of the day you will not have enough plane of recovery to deter defendants from sort of the wrongful conduct.

So, I think it is a tough problem, which is why I think the delay solution is more interesting. You allow the better evidence to show up first before you make a decision rather than cutting it off early on and then barring the plaintiffs with the statute of limitations provisions.

MR. FRANK: Does anyone want to defend cross-examination as a means for getting good expert evidence then? [Momentary silence] This is very telling. As moderator, I'm going –

MR. MICHAELS: I mean, again, as one of the non-attorneys in the panel. It does not strike me as a great system, but I have not seen a better one. What is the better system? That is really the question. Right now that is the status quo. I do not think we have seen the convincing argument that expert panels are going to solve the problem. So, we are just saying general cross-examination? I mean, we do not have to defend it because it is wonderful, but we defend it because it is there and it seems to work better than anything else we have.

MR. HOLLINGSWORTH: And it plays a really important role in the Daubert –
[Cross-talking]

MR. HOLLINGSWORTH: Cross-examination plays a real important role in the Daubert gate-keeping process. It is a process that happens before you are in front of the jury. It is a process that happens at a pretrial motions kind of phase. And it is always in front of the judge and many of these judges in the cases that were specifically referring to here this morning took testimony from expert witnesses for days and days in these cases before they made a decision about whether evidence is admissible or not. That makes a record and that record is based on cross-examination.

So, there are really two roles cross-examination plays that are important. One is, before he makes a decision and the other is if he makes his decision to admit testimony, it is true that cross-examination of an expert witness at trial by both sides is critical to the process and I think, going back to the DDT cases, the crucible of cross-examination is just critical in what we do. So, I do not want to remain silent on that point.

MR. BERNSTEIN: I just want to add, I teach evidence and whatever evidence we rule that we discuss, I always have a student who raises his or her hand and says, “Professor, if this evidence is so obvious and unreliable we want to exclude it, why do we not just allow the evidence to come in? A good attorney will cross-examine it.” And one of the responses I give as most attorneys are not that good.

But beyond that, especially in certain contexts like toxic tort cases, the reason that it is important to exclude the evidence beforehand is because they really should the case go to jury or not, because often trials in practice are not actually about the evidence, but about something else and good attorneys all know that if you do not have the law, argue the facts, and if you don’t have the facts, argue the law. So, good attorneys when they do not have a scientific evidence to back them will often argue the facts.

So, if you look at some of the bigger litigation fiascos we have had over the last x number of years: [one] is Bendectin, which allegedly cost birth effects and breast implant litigation. The plaintiffs’ lawyers made the case about companies that did not do due diligence. They misbehaved. […] [Whereas in] the breast implant cases, some [plaintiffs] were not actually injured – they allegedly had some atypical disease that no one could identify –  […] in the Bendectin litigation you had children with birth defects that really needed the money. The parents were clearly innocent. The companies arguably had not done enough background and certainly, regardless of whether one can objectively say they did or not, it is always easy to find bad document. Especially now with e-mail – as a lawyer, I’m always amazed at what people put an e-mails. Mostly in antitrust cases, you will say, “We will crush our opponents and make sure they get driven out of business!” This is legally irrelevant actually, because of course, you want to crush your opponents to drive them out of business. But in antitrust cases it looks really bad for the defendant. So, you have a case like this. You had this very confusing expert testimony that you have qualified people on each side. You have uncertainty.

And frankly, often you have evidence that goes over the head of the jury, especially with the way it is presented. You may hear the plaintiff’s expert evidence on week one, not hear the defendant’s expert evidence until week five, and the rest of the case is about damages, about punitive damages. Cross-examination can work well if the jury really thinks that the case turns on whether this expert has been reliable or not. But if the jury says we do not understand what is going on, we have lost track of that. What we really care about is we have an evil corporation that misbehaved then we have an injured person, an injured kid, who is going to need support for the rest of his life, you can imagine a cross-examination will not really affect the outcome that much one way or the other.

MS. RUNKLE: I did not mean to indicate that I'm totally opposed the cross-examination. I'm certainly. My point was that it does not necessarily solve the problems of this selection bias and the unconscious bias. I have been in a lot of trials and I have seen how the cross-examination is done and even the most knowledgeable and poised scientist can end up looking befuddled. […]

MR. MICHAELS: But in terms of the concerns that David raised, one of the ways I think you probably could use scientific experts productively is in helping jurors understand the method issues and say, what do you need to get the causation? Is this reasonable? And so, you could try to separate out those emotional from the scientific issues. Again, I do not know if there are rules against doing this in court. It strikes me that would be a useful use for the experts both to help the judge and the jury figure this out.

MR. FRANK: Let's throw this up into the audience.

MR. ODYNIEC: Hi, I'm Norm Odyniec. I'm a surgeon here in Washington and I’m also an expert witness in malpractice cases. My question is, is Daubert applicable in medical malpractice cases? And if it is, the utilization of it, namely the establishment of experts, could that obviate the jury trial and just have a panel of experts that could adjudicate the problem?

MR. BERNSTEIN: Daubert is applicable to all expert testimony. However, the vast majority of medical malpractice cases will be in state court because you have an in-state plaintiff and an in-state defendant. And at least locally, we do not have any Daubert jurisdictions. Virginia is not a Daubert jurisdiction. DC local court is a Frye jurisdiction. Maryland is a Frye jurisdiction. Pennsylvania is a Frye jurisdiction. New York is a Frye jurisdiction. I think North Carolina or South Carolina are also not Daubert jurisdictions.

So, if you are having cases anywhere in the local region, you are not going to wind up applying Daubert, and Frye has traditionally not been applied to medical malpractice testimony because physician testimony is not considered scientific. It is considered more of an art.

[…] But a lot of the issues in medical malpractice also are did you follow the correct procedures. So, that is sort of a non-scientific issue. Here everyone agrees what the standard is more or less. Did you follow what the standard of care is? When you saw the medical monitoring device on the women giving birth make this beep, should you have intervened at this point? The problem there is that there is just a hindsight bias. If you know that the kid wound up with a problem, it is easy to say, “Sure, given the ways things were done, it could not have hurt.” But basically, the short answer is that we really do not know essentially how Daubert will affect your average malpractice case because it is mostly in state court and often in jurisdictions that do not have Daubert and even when they do have it, since there has not much precedent yet on that.

MR. CHENG: There is actually an interesting sort of project that occurred in the 1960s about medical experts. So, I think it would be very difficult to get rid of the jury trial. But one thing you could do is have court-appointed experts that were doctors. And so, you have these neutral experts rather than the partisan experts. So, in the 1960s, the New York Academy of Science put together a list very similar to what Deborah does. They have a list of eminent doctors that would testify in these cases, and the judges were very happy with the cases and it seemed to be a great success. And then, for whatever reason, it petered off and that the interest sort of disappeared, and so now it is obviously defunct.

I think the interesting thing is, whether or not, Daubert and this renewed interest in scientific reliability will keep things going this time around. But, historically, these proposals have been made and they have disappeared.

Questioner: [indiscernible].

MR. BERNSTEIN: Well, one interesting issue is that some state local medical societies have determined that expert testimony is always a part of the practice of medicine. So, if you deviate sufficiently from the standard of care in your testimony, in other words, if you say things in court that no reputable doctor would say to their patient, then you can actually be deprived of your medical license. Now, the problem is that this leads to obvious antitrust concerns, […], because there is always a suspicion of the doctors are just looking out for their colleagues. But that is a new trend.

MR. MICHAELS: If I can add – just from a cynical basis – I have no particularly knowledge or interest in medical malpractice. But these same state and county medical societies never suspend their members from the most egregious and harmful medical procedures. And yet, they want to go after what they say in court. It is really remarkable to me.

MR. BERNSTEIN: One reason is they actually need a complaining doctor, right?

MR. MICHAELS: A complaining doctor, not a complaining patient.

MR. BERNSTEIN: Right. You need a complaining doctor. So, a doctor who will complain will be the defendant in the malpractice case but when it is just a patient, you do not have anyone to complain.

MR. FRANK: Following up on that, Philip Howard of Common Good has actually proposed something called medical courts where medical malpractice cases would be resolved by panels and medical experts rather than by juries. None of those have gotten off the ground yet though pilot projects are being talked about in various legislatures.

MR. BERNSTEIN: Just one last thing about this. And of course, in practice the way he works in courts is if you actually know anything, if you are a perspective juror and you actually know anything about the subject matter, that makes you highly likely to be voir dire-ed off the jury. So, we have a system where ignorance of the subject matter actually is much more likely to be on the jury. So, the extent you actually will have someone who will be able to critically look at the evidence, they are likely to be sent home. So, that is not one of the better aspects of our system.

MR. FRANK: I think it would be very interesting to see which side is the one that knocks these people off, because obviously one side does have that incentive if there is not a real scientific dispute.

MR. WEED: I'm Doug Weed. I'm a physician/epidemiologist at the Weinberg Group. I'll ask David, the speaker, this question, and others might want to chime in. I was kind of curious about something you said with what I felt was considerable certainty. I think you may have even said “the gold standard” or “the best evidence” would be a series of epidemiologic studies with a relative risk greater than two or greater than or equal than two.  And let's just throw in there, “and statistically significant” so we are not to get into that argument.

Now, my question is this, where did that come from? Or let me ask you another way, what is your justification for saying that with the amount of certainty that you said, and I'll give you some choices here. One would be, it works. That is, your experience in courts has been that it works. A second choice would be, experience in public health. That is to say, when that situation arises, then we know that that the most likely explanation is causality. So, those are two experiential possibilities.

And then there are two theoretical possibilities. One is, that is what legal theory says about scientific evidence. And the fourth choice – I hope this is not too complicated for you – the fourth choice is, that this is what scientific theory says about scientific evidence. And I'm just curious about what is the justification from those choices, and maybe you have some other one and that will be fine, too. I'm curious about where they came from.

MR. BERNSTEIN: Sure. A one place that it comes from certainly is precedent. I mean, legal precedent, which is that courts will say that that epidemiological evidence is [the standard]– I think that gold standard remark was probably quoted from a case at some point. But it goes beyond and it does go to legal theory to some extent because the standard for causation in a civil case is preponderance of the evidence, where you will have to prove as an individual plaintiff in the case is that if I have immune system disease and I also had a breast implant then it is more likely than not that – obviously, there is a background risk of immune system disease – so, you have to prove it is more it is more likely than not, not just that there are some causal or there are some association or even causal association between breast implants and disease that have more than doubled my risk.

Now, I am simplifying a little bit, which I do for legal audiences – obviously if I'm talking to society of epidemiologists and medical doctors I have to go into lot more details – but there are a lot of permutations on this. There is a case, for example, from the Eastern Districts of New York where these epidemiological studies were consistent, but they show only like a 1.7, 1.8, and what the judges there say is well, I'm going to allow this evidence in, if the plaintiff could show in an individual case that the relative risk for that plaintiff may be greater.

And what does that mean? Well, if it is, let's say, [the problem is] colon cancer, and there is a background rate of colon cancer.  Well, it is the background rate but the background rate may be higher if you have an uncle, father, or brother with colon cancer, if you smoke, if you do this, if you do that, if you have a bad diet. So, if you could show that your client is a clean-living Mormon with no genetic history of colon cancer, then it could be the relative risk for that person, your epidemiologist will be willing to testify is above two.

Now, I understand that a lot of epidemiologists think that studies with the relative risk of less than three have enough uncertainty that we should throw them out, or at least be very skeptical of them. There is actually a lot of skepticism there is actually lot of skepticism in the legal community initially about whether you can ever prove anything by pure statistical evidence without some other individual evidence of causation. Defendants originally in the ’80s when toxic tort cases started coming up, said we should not look at epidemiological evidence at all.
[…] But, eventually, people came to realize people who are bringing these toxic tort cases, they have animal evidence and this evidence and that evidence. It looks suggestive. But what if the epidemiological evidence is contrary, which it often it turned out to be in Agent Orange, in Bendectin, and all of a sudden defendants got religion and said, “Hey, this epidemiological evidence is coming out in our favor. If you do not even have that, maybe we should rely on it.”

So, I guess, the answer ultimately is, I do not really know what scientific theory says about this. But I think if you have a case that has to be resolved somehow, and you have the standards epidemiologists use, and you have statistical significance, and you have the other evidence to meet more or less the Hill criteria, you have a relative risk greater than two – the way you translate that into law is that shows general specific causation by preponderance of the evidence for the average plaintiff.

MR. WEED: Now what you have said is that this is a legal argument. Granted. Okay. But what you all said also is that this is about expert witnesses. This is not about the lawyers. This is not about legal theory. This is about the experts. Now, the experts are unlikely to have gone to law school. How comfortable are you that the experts agree with what you just said? Or put another way, if you do not know what the scientific theory says about this particular situation, but the experts do, how are you going to resolve that? I mean, is this about the legal theory of causation or is this about the scientific theory of causation? Or more likely is this about where those two come together at this nexus? And somehow that is the part is not getting sorted out just yet.

MR. BERNSTEIN: It may very well be the latter. I guess the problem is that the scientific community is not usually interested in individual causation.

MR. WEED: Well I am!

MR. BERNSTEIN: So, the scientific community (minus you) says that we have these epidemiological studies, and they are trying to have a statistical relationship. They are trying to figure out what public policy should be in this regard and whether Joe Smith was injured by this particular substance on this particular day from this particular exposure is not something that scientists are terribly interested in.

MR. WEED: I will just take one more just point of clarification. Mr. Hollingsworth was correct about the Austin Bradford Hill perspective. Austin Bradford Hill did say that this is all about association that had been demonstrated and given that, what would it take to make a claim about causation. That is the premise of the Austin Bradford Hill criteria. Now, whether everyone agrees with that or works it that way, it maybe a different story, but that is correct.

MR. FRANK: Thank you.

QUESTIONER: Thanks. Professor Bernstein, you made an interesting point about the course of the trial. How a jury will hear the plaintiff’s expert in week one, but not the defense expert witness until week five. I'm wondering, especially, in cases where it is impractical to have a judge-appointed scientific panel, would be it wise, as a general rule, to try to bifurcate the trial to have just the general causation (just the expert witness testimony) in first phase of the trial so that the jury can hear both sides of the case more or less contemporaneously?

MR. BERNSTEIN: I would say yes. In fact, I'll give you an example. In the Woburg litigation that is the subject of the book and movie A Civil Action, the plaintiffs wanted the jury to just come up with the verdict and that is that. The defendants argued for trifurcation. First, did the ground water from these two particular plants actually reach the water supply that these people were drinking? There were about eight other defendants in the case, who had all settled, and apparently were much more likely to have contaminated the ground water. So, first, did the ground water actually reach there? Second, if so, did these contaminants actually cause the cancer? And then third, what are the damages if so.

So, Charlie Nesson, who is a professor at Harvard, and who was helping represent the plaintiffs – to my everlasting chagrin he was hired by Merrell Dow to help them on Daubert even though everything he had written was contrary to their position in the case until then (talking about experts whose opinions are for hire […]) –  so, Charlie got up and said, “This is terrible. This is about misbehavior of the [corporations]” – and there is a quote in a book for those of you who are interested. So, we are talking about this is a case of corporate misbehavior and good and evil and this and that. I do not want to hear good and evil. I want to know about causation. So, the jury decided immediately that the ground water never actually reached the plaintiff’s water supply from these particular plants. Now, it is the end of the case.

Now defense attorneys tend not to be as enthusiastic about trifurcating, bifurcating, as theoretically they should be. I do have a theory about this, with all due respect to the defense attorney who is here, which is that what defense attorneys fear most – and this seems to be consistent – is the big loss. Losses within reason, within the bounds of what will be expected by the client, do not ruin the relationship with the client and do