« "The Pedagogical Prosecutor" | Main | "15 Years In Environment Of Constant Fear Somehow Fails To Rehabilitate Prisoner" »

March 4, 2014

Why the #@$%! are IQ tests, but not burdens of proof, the key issue in Hall?

The question in the title of this post is my basic reaction to what struck me as a very annoying SCOTUS oral argument yesterday in Hall v. Florida concerning how states must deal with the Atkins categroical Eighth Amendment constitutional bar on executing defendants who are mentlly retarded.  The transcript of the argument is available at this link, and Lyle Denniston has this SCOTUSblog summary of what transpired.  Here is the start of Lyle's recap:

If a state, trying to make it simple to decide who can be given a death sentence, opts for a choice that looks arbitrary, it is likely to have a difficult time in a Supreme Court that worries about the chances of error.  That was demonstrated anew on Monday, when Florida found itself in deep Eighth Amendment trouble with a rule that anyone with an IQ above 70 can be executed if convicted of murder.

A quite definite majority of the Justices — perhaps, notably, including Justice Anthony M. Kennedy — left little doubt that Florida and six other states will not be allowed to maintain an automatic test-score-based cutoff for those who could qualify as mentally retarded and thus can escape the death penalty.

Kennedy’s role is central because he has most often led the Court in narrowing the category of those eligible to be executed, to take account of reduced capacity to be held responsible for their criminal behavior.  He was among the most active in questioning Florida’s approach to mental retardation among those on death row.  And, on Monday, he added in some strongly implied criticism of a system that allows some inmates to remain on death row for decades — an issue that is not directly involved in the new case of Hall v. Florida.

Justice Antonin Scalia expended considerable effort to buttress Florida’s basic argument that the scientific community cannot be trusted to make the rules for eligibility for capital punishment, but except for some supportive hints from Justice Samuel A. Alito, Jr., this seemed to be a largely forlorn endeavor.

Most of the other Justices joined in the pursuit of an Eighth Amendment rule that would assure that the mental retardation inquiry was sophisticated and nuanced, so that the risk of error was taken fully into account.  While such a rule might not definitely hand over the details to the judgment of scientists and doctors, it apparently would not tolerate an approach designed simply to assure that fewer death-row inmates get off with a claim of mental disability.  That appeared to be Florida’s main objective.

This account of the tenor of the Hall oral argument seems right, and yesterday's SCOTUS discussion reinforces the all-too-usual modern death penalty dynamic of Justice Kennedy seemingly being the swing vote so that whatever he may think about hard capital issues becomes Eighth Amendment jurisprudence.  But beyond my standard annoyance with modern constitutional death penalty law now being all about what Justice Kennedy thinks, the Hall oral argument has me extra annoyed because the Court seems intent on focusing application of Atkins on the narrow medical issue of how exactly IQ tests can be used when assessing mental disability rather than on the critical legal/constitutional issue of how burdens of proof can be allocated when deciding who is exempt from the death penalty.

I suppose I should not be too surprised, based both on the cert grant and the approach taken by the advocates in Hall, that Florida's IQ line-drawing approach to Atkins took center stage during yesterday's SCOTUS argument.  But as Kent highlights in this effective post at Crime & Consequences, the discussion of statistics and IQ measurement error among the Justices was garbled at best, and really more an example of junk science on display rather than a serious exploration of whether and when an inexact measurement device (an IQ test) can itself establish at the margins placement of a defendant in an inexact medical category (mentally retarded/disabled) which SCOTUS has given legal significance via its Eighth Amendment jurisprudence.  

That IQ is, at best, an inexact measurement device is amply proved by the Brief of petitioner Freddie Lee Hall: that brief indicates that IQ measurements for Hall have ranged from 60 to 80 and that various doctors at various times have scored his IQ at 71, 72, 73, 74, 76, and 79.  This reality suggests that IQ tests can only provide a general fuzzy picture of a person's mental abilities and that it would be foolish and misguided for anyone to use IQ scores alone as dispostive "scientific" or "objective" evidence of whether a murderer is or is not mentally retarded/disabled.  In this sense, I suppose, Florida does look bad having a bright-line IQ line for administering Atkins.  But given that IQ tests will always scatter in these kinds of cases, I think the legal issue of who bears the burden of proof and at what level as to Atkins claims is what ultimately determines the reach of Atkins in any state.

If SCOTUS strikes down the current Florida approach to Atkins without speaking to burdens of proof, Florida could then just employ Gerogia's approach to "assure that fewer death-row inmates get off with a claim of mental disability" by requiring defendants to prove they have MR beyond a reasonable doubt (or perhaps even say beyond all doubt).  Already, Florida requires a defendant to prove his disability by clear and convincing evidence, and that standard (with or without an IQ requirement) necessarily entails that Florida will be able to execute murderers whom judges reasonably think, but are not clearly convinced, are mentally retarded/disabled.   The experience in since Atkins makes clear that states eager to limit who gets off death row can do so using legal standards like proof burdens rather than relying just on IQ numbers.  Thus if (and when?) SCOTUS only addresses how IQ tests are used in Hall and dodges any discussion of burdens of proof, Atkins application issues (and interstate Atkins disparity) will persist.

I suppose I am frustrated here in large part because SCOTUS has long been eager to avoid, in all sorts of settings, establishing any clear and predicatble constitutional rules for burdens of proof concerning facts or factors that impact only sentencing determinations and not guilt.  These burden-of-proof issues, which may be viewed as a Fifth Amendment due process concern and/or an Eighth Amendment concern depending on the setting, can be found lurking in the Apprendi-Blakely Sixth Amendment line of cases, but they have never gotten the independent treatment that I think they merit and needs  I have long been hoping Hall might finally lead to some useful burden-of-proof constitutional jurisprudence, but after the oral argument I am no longer hopeful on this front.

March 4, 2014 at 12:46 PM | Permalink


TrackBack URL for this entry:

Listed below are links to weblogs that reference Why the #@$%! are IQ tests, but not burdens of proof, the key issue in Hall?:


But isn't burden of proof the province of legislatures anyway? I mean aside from Woodson's abolition.

Posted by: MM | Mar 4, 2014 1:35:13 PM

The whole argument was patently silly and once again showed why using an IQ test in the first place is a bad idea. Not a single judge seemed to have even a basic understanding of what IQ test even is or does. And you're not helping matters either Doug: what is the medical use of an IQ test? An IQ test has no medical purpose. Intellectual disability/MR is not a medical condition, not by any ordinary common-sense meaning of that term nor by any professional meaning of that term. MR might result from a medical condition such as TBI but it is not in and of itself a medical condition.

Reading the oral argument left me annoyed and buttressed my belief that what is really driving the the 8A is not any rational approach to the law but sentiment. Maybe that's right. Maybe at the end of the day "cruel" can't be viewed through any lens other than sentiment. But if that is so I wish SCOTUS would stop dragging science and statistics through the mud and say what they mean: that Southerners are bunch of backwoods meanies who still can't get over slavery and they need to be shown the light.

Posted by: Daniel | Mar 4, 2014 1:38:24 PM

Yeah, I technically think Alito has the best argument. If the defendant has the burden of proving all three elements by clear and convincing evidence (as Florida requires and was not challenged), these test results strike me as perhaps sufficient by preponderance (taking into account SEM and the composite score as argued by Counsel for Petitioner), but I can certainly see a Judge going the other way for Clear and Convincing.

The counterargument is that the makers of the test themselves have suggested that the SEM is important to consider and to take things holistically if it falls within the SEM. If the state had the burden to establish absence of MR, I think using these tests improperly, in circumstances such as this, would be completely inadequate. But, if all agree that the state can impose the burden on the defendant, I can't see how inconsistent tests work in his favor (unless the burden of production were on the defendant and the burden of persuasion were on the state).

Posted by: Erik M | Mar 4, 2014 2:06:12 PM

The Supreme Court let stand a decision by the California Supreme Court banning the use of the IQ for the purpose of granting minority children extra services in the form of special education. This is the most validated test in history. Know the IQ at age 7, and predict the performance of the person at age 50. Because of the racial disparity in IQ, the Court banned its use for its intended purpose, educational classification and services.

Now, it is being used by the lawyer to measure culpability, for which it has zero validity. And the phrase "mental retardation" is an educational term measured by this test. The reason the Supreme Court has some inkling it does not know enough about this subject and wants to weasel out of making a decision that will bring it ridicule. If I were a prosecutor, I would demand a Daubert hearing right before the Supreme Court and easily rebut its very concept as relevant to culpability.

Beyond the misuse of the phrase as another excuse o coddle a vicious murderer, slow learning is an aggravating factor, not a mitigating factor. It says, punishment will be less influential, and only incapacitation remains as an aim of sentencing.

Posted by: Supremacy Claus | Mar 4, 2014 2:37:37 PM


The entire dialogue by the court regarding the SEM was based upon a fundamental misunderstanding of science. Science is a method, nothing more, which is why it is called the...wait for it...scientific method. Being a scientist is nothing more than agreeing to a common set of procedures. So the fact that the test is set at a 95% confidence level is simply making the statement that the science of psychology is willing to accept a 5% error rate. On his blog post Kent writes, "A result meeting that criterion is pronounced "statistically significant" and given the coveted asterisk, as if there were a big difference between p = 0.051 and p = 0.049. (There isn't.)" He's wrong, there is a big difference. The big difference doesn't lie in the math but in the community. Is a 5% cut off arbitrary? Yes. But a line has to be drawn somewhere and it has to be drawn ahead of time. A 5% error rate is not more or less arbitrary than a 30 day deadline in the law. Is there any big difference between a petition being filed on the 30th day vs the 31st day? On the calendar, no, but legally there can be a huge difference if the deadline is 30 days. And scientists decide a confidence interval ahead of time for the exact same reason that the legal profession decides on a 30 day deadline in advance. Because if one didn't there would be chaos. The deadline to turn in a petition would simply be whenever one felt like it. It's worth pointing out that in physics the standard acceptable error rate is usually p=< .001. So different fields can have different standards. It is simply what error the particular scientific community is willing to accept. Psychology isn't as exact as physics so we accept a higher rater of error.

So the problem with messing with the SEM isn't that it can't be done. It can be. The SEM can be set to 80%, 60% whatever one wants. But no reputable psychologist would consider a p=.2 /to be science/. Why? Because that is not what we have agreed upon as a community in the exact same way that no one in the legal profession would consider a petition filed on day 35 to have met the 30 day deadline. So the strongest argument in this case is that Florida has to take the test as it comes to it and it comes to it with an SEM. If SCOTUS doesn't want to deal with the SEM as it comes to it then its choice is to abandon the IQ test altogether. Of course in the larger picture Kent is correct: there is no meaningful difference in a death penalty context between an IQ of 71 or an IQ of 68 but this because an IQ test was not designed for a legal purpose but for the purpose of educational diagnostics. The court is trying to piggypack off of a concept that it neither understands nor appreciates.

Posted by: Daniel | Mar 4, 2014 2:52:01 PM

Good points, all, and I was hoping Daniel not to make matters here. I meant to describe "intellectual disability/MR" as a medical category, not a condition. Is this fair? And it seems the medical community does think IQ helps with assessing who is in this category, though I again agree that this is not what I would hope serves as the defining metric for any legal purpose, let alone eligibility for execution.

As for proof burdens, MM, I do not think this can be exclusively the province of legislatures at sentencing. Do you think a legislature say the default sentence for driving 100 MPH (or for dealing marijuana) is LWOP unless and until the defendant can prove beyond a reasonable doubt that nobody could have been seriously injured during the drive (or by the pot)?

Posted by: Doug B. | Mar 4, 2014 2:55:58 PM

Question for you appellate lawyers.

An affirmative decision by the Supreme Court applies to all the states. Does a refusal to cert to review a state appellate decision have the same effect of giving the state supreme court decision national jurisdiction because the Supreme Court thought it correct and needing no review? Or does refusal to cert leave the decision localized to its original jurisdiction?

Larry P v Riles was affirmed by the Ninth Circuit, so applies to that western region. Did refusal to review endorse it and make it apply to all circuits?

Posted by: Supremacy Claus | Mar 4, 2014 3:02:15 PM

SC, no. Denial of cert has no precedential or officially persuasive value.

Daniel, science and the law are two different ways of reaching a result one can feel confident about. There are many areas they don't overlap particularly when the purposes are different (see, e.g., FDA safety checks vs. liability for negligence). However, I'm not sure the two are all that different here. The goal here is the same: to determine the status of a person and decide what to do with that going forward. In that sense, I think the issue still is one of burden of proof. I would imagine, for diagnostic purposes, it's better to err on the side of finding a developmental disability. I'd argue that it's also entirely reasonable to err on the side of finding MR when it's unconstitutional to execute unless a person is not MR. However, if the burden of proving exclusion from the death penalty falls on the defendant, that means, in doubt, you fall on the side of "not MR."

I think this is very different from where science and the law usually conflict: For example, the law is better at finding historic fact, while science is better at finding a general rule. Science is (supposed to be) deductive, while the law is abductive and inductive.

Posted by: Erik M | Mar 4, 2014 3:30:58 PM

It was an annoying oral argument for sure. I think several of the justices especially Kennedy, are confused. Then, to make things a little bit more complex, Kennedy brings up that the last ten inmates executed in Florida have spent an average of 24 years on death row.
I suppose he looked at how long Hall has been on the row which is about 35 years. We can look at each case and point to reasons why the inmate is still alive. I know 3 reasons right away why Hall is still there. His attorney did not raise MR until 10 years after sentencing. The other two are Hitchcock and Atkins which have both benefited him.

Posted by: DaveP | Mar 4, 2014 4:38:35 PM

Well, I think his question there was actually an argument in support of Florida's position rather than against it. The thrust seemed to be one of determining MR now after dealing with the effects of prison conditions rather than MR back then. Alternatively, an argument that the delay was a long time isn't something that generally helps the defense in this situation. Post-conviction delay protects the Defendant by allowing alternative arguments not realized at the time to develop.

Lyle Denniston has always impressed me with an ability to read or listen to an oral argument and come out with clear conclusions about how the Justices are going to rule. After reading the transcript, I felt far more uncertainty. While Justice Sotomayor and Alito's positions were clear and Justice Scalia's position relatively clear, I did not find Kagan, Ginsberg, or Kennedy to be as clear cut here.

Posted by: Erik M | Mar 4, 2014 4:51:59 PM

Another interesting fact in this case is Hall and his cohort Mack Ruffin each were granted relief from Hitchcock. On resentencing, Hall got death and Ruffin got life imprisonment.

Posted by: DaveP | Mar 4, 2014 5:04:21 PM

Part of the problem with this case and similar cases is that there are multiple issues merged into one.

Step One -- is there a constitutional definition of X (in this case being mentally retarded/developmentally disabled/similar term for deficient mental capacity).

Step Two -- if there is an agreed upon constitutional definition of X that includes a statistical test, is everyone within the SEM of that result considered to pass (both the highs and the lows) or is that an issue for the finder of fact to consider (able to reject false passes as well as false fails) on a case by case basis.

Step Three -- if the answer to both Step One and Step Two is yes, can the State still describe its test based on a specific result of the statistical test or must it describe the result as being within a range (both are arguably statistically proper)? In other words in this case, can the State say that IQ must be below 70, but Hall's experts can testify that a 74 might really be a 69 and therefore make a submissible case or must the State expand to the range of 65-75.

Step Four -- For those within the range where the result is inconclusive as to the correct result, who bears the burden of proving the "correct" result?

Posted by: tmm | Mar 4, 2014 6:01:48 PM

This argument is so ridiculous. This guy scored a 79 and an 80 on separate IQ tests. Given that the only way to score that much higher than he putatively is on two separate occasions is to be incredibly lucky. With respect to the lower scores, well, it's very easy to score lower than you really are, particularly when you are motivated to do so.

What are the odds that a guy with a "true" score of 69 could score a 79 and 80 on two separate occasions? That's the question that someone should have asked Seth Waxman at oral argument.

Posted by: federalist | Mar 4, 2014 10:30:05 PM

He did sort of address that, by explaining what the makers of the test do in that situation. The answer is to take the composite score. He cited to a footnote in his brief that cites to a book that explains that in more detail.

If you know more about the subject, feel free to add to it. But I'll take him at his representation that the medical community doesn't just go "well, he got an 80, so that 60 must be meaningless" and leave it at that.

However, since the law values finality, I do think it's not inappropriate to debate at what time a score forecloses future scores. However, when one engages the medical debate, one should do so from a medical perspective. And if one thinks that diagnosis is important in this equation, that diagnosis should be done correctly.

Posted by: Erik M | Mar 4, 2014 11:22:11 PM

Let's take a broader overview. The overwhelming majority of IQ tests that occur before age 18 are done in an educational setting for educational purposes. I honestly do not know where this idea that they are "medical" or "clinical" tests come from because they are never administered in that capacity. They are administered to sort children for developmental purposes. IQ tests are administered either as part of a student's IEP or to place a student in some type of gifted and talented program. In fact, the overwhelming majority of IQ tests to children are administered for the later reason--advanced placement--rather than disability. There is nothing in the ADA or IDEA or any other law that requires an IQ test for a K-12 student with an intellectual disability. In fact, if you talk to any person who does disability accommodations at the college level it is a chronic problem that students will be on an IEP in the K-12 system and have almost zero paper work to back it up. So they get accommodations in K-12 and then get denied in college, leading to failure.

The reality that most IQ tests given in K-12 relate to advanced placement ties into federalist's point about malingering. Malingering isn't a typical concern with an IQ test because there is a real negative consequence to doing poorly: the denial of a benefit (advanced placement). So most psychologists don't pay much attention. But this incentive gets reversed when a person's life is on the line: here the tester actually has an incentive to do poorly (they live). Now to be fair if the person really is intellectually disabled they probably lack the intelligence to consciously malinger. But the malingering problem is one of the reasons why normally an intellectual disability requires evidence of that disability outside of the raw test score.

"What are the odds that a guy with a "true" score of 69 could score a 79 and 80 on two separate occasions?"

This is actually normal, assuming that the higher scores come later. It is known as the "test effect". If a person takes the same test multiple times they get better at taking the test. The improvement in scores doesn't represent any real gain in the underlying cognitive function, just an improved ability to take the test.

Posted by: Daniel | Mar 5, 2014 12:39:53 AM

The so-called "test effect" observed on an individual basis leads collectively to the Flynn effect, which is the substantial increase in average scores on intelligence tests all over the world. As this article (http://www.princeton.edu/~achaney/tmve/wiki100k/docs/Flynn_effect.html) indicates, it matters which IQ test is given when comparing scores obtained by a particular individual over a period of time. Thus, it would be useful to know how Mr. Hall's higher scores related to the periodic updating of the IQ instruments used in his case. In other words, if, after obtaining a score below 70 on a recently updated test Mr. Hall scored several points higher on the same or similar test just before the next update, the increase in his score might be attributable to the "test effect" or "Flynn effect." Or perhaps he was given an outdated test and scored higher on that test than other, more up-to-date test instruments. These factors seem to indicate that fluctuations in test scores are expected and do not necessarily demonstrate actual development of higher intellectual ability in an individual.

Posted by: Jan K-G | Mar 5, 2014 12:18:43 PM

"If a person takes the same test multiple times they get better at taking the test. The improvement in scores doesn't represent any real gain in the underlying cognitive function, just an improved ability to take the test."

That definitely applied to me vis a vis the LSATs. Before I did any prep, I took an initial cold/diagnostic test and scored almost 20 points lower than the score I eventually made on the real test a few months later, after many hours of further exposure to sample problems/tests. Also, of course, even that initial score was likely influenced by the fact that I had been taking somewhat similar tests throughout my life (PSATs, SATs, etc.). Anyway, my point is that I doubt there was any increase in my underlying cognitive abilities during that time. I just learned the test. No reason that principle shouldn't apply in general to a low IQ person who is given the same test over and over. He or she is never going to ace it, but even a marginal familiarity/increased comprehension of what is going on during the testing can make a difference when you are starting from such a low bar.

Posted by: anon | Mar 5, 2014 12:50:33 PM

If MR is to be a mitigating factor, let us make it accurate. The average IQ in prison is not 100, it is 85. If MR is 2 standard deviations below the mean, then for criminals, define it at 55. Atkins would not have MR (see the drug business survival from age 9, see the luring of a competitor into his car to shoot him after a ride - could any of the abolitionists here say they could pull those tricks off skillfully, or could the Justices?) Nor would the current appellant.

Posted by: Supremacy Claus | Mar 6, 2014 4:22:34 PM

Post a comment

In the body of your email, please indicate if you are a professor, student, prosecutor, defense attorney, etc. so I can gain a sense of who is reading my blog. Thank you, DAB