In psychiatry, "a rose is a rose is a rose" as Gertrude Stein put it. That's according to an editorial in the American Journal of Psychiatry called: The Initial Field Trials of DSM-5: New Blooms and Old Thorns. Like the authors, I was searching for some petal-based puns to start this piece off, but then I found this "flower with an uncanny resemblance to a MONKEY" which I think does the job quite nicely:
Anyway, the editorial is about the upcoming, controversial fifth revision to the Diagnostic and Statistical Manual (DSM) of the American Psychiatric Association (APA). A great deal has been written about the DSM-5 over the past few years, as
"the
(see, I can reference early-20th-century poetry too). But now the talk has moved into a new phase, because the results of the DSM-5 'field trials' are finally out. In these studies, the reliability of the new diagnostic criteria for different psychiatric disorders was measured. The new editorial is a summary and discussion of the field trial data. Two different psychiatrists assessed each patient, and the agreement between their diagnoses was calculated, as the kappa statistic, where 0 indicates no correlation at all and 1 is perfect. It turns out that the reliabilities of most DSM-5 disorders were not very good. The majority were around 0.5, which is at best mediocre. These included such pillars of psychiatric diagnosis like schizophrenia, bipolar disorder, and alcoholism. Others were worse. Depression, had a frankly crap kappa of 0.28, and the new 'Mixed Anxiety-Depressive Disorder' came in at -0.004 (sic). It was completely meaningless. The American Journal editorial was written by a group of senior DSM-5 team members. I'm sure they wanted to write a triumphant presentation of their work, but in fact the tone is subdued, even apologetic in places:
rough beast, its hour come round at last / Slouches towards Bethlehem to be born"
As for most new endeavours, the end results are mixed, with both positive and disappointing findings...Experienced clinicians have severe reservations about the proposed research diagnostic scheme for personality disorder...like its predecessors, DSM-5 does not accomplish all that it intended, but it marks continued progress for many patients for whom the benefits of diagnoses and treatment were previously unrealized.
Remember: this is the journal published by the organization responsible for the DSM and even they don't much like it. But the real story is even worse. The previous editions of the DSM also conducted field trials. These trials had a system to describe different kappa values: for example, 0.6-0.8 was 'satisfactory'. However, the new DSM-5 studies used a different, lower threshold. They simply moved the goalposts, deeming lower kappa values to be good. At one point, they wrote that values of above 0.8 would be 'miraculous' and above 0.6 a 'cause for celebration', yet this wasn't the view of previous DSM developers. The indispensable 1boringoldman bloghas a nice graphic showing the results of the DSM-5 trials, with the kappas graded according to the old vs. the new criteria. As you can see, the grass is greener on the new side.
The fact is that the DSM-5 field trial results are worse than the results from DSM-III, the 1980 version that's served mostly unchanged for 30 years (DSM-IV made fairly modest changes.) The reliabilities have got worse - despite the editorial's claims of 'continued progress'. It's true that the DSM-5 field trials were a lot bigger and conducted rather differently, but still, it's a serious warning sign. Finally, there was great variability in the results between different hospitals - in other words the reliability scores were not, themselves, reliable. Some institutions achieved much higher kappa values than others, but it's anyone's guess how they managed to do so. Still, there's great news: the DSM-5 is just a piece of paper (well, a big stack of them). Any psychiatrist is free to ignore it - as the creator of the more reliable DSM-IV (not III, oops)is now urging them to do.
Freedman R, Lewis DA, Michels R, Pine DS, Schultz SK, Tamminga CA, Gabbard GO, Gau SS, Javitt DC, Oquendo MA, Shrout PE, Vieta E, and Yager J (2013). The Initial Field Trials of DSM-5: New Blooms and Old Thorns. The American Journal of Psychiatry, 170 (1), 1-5 PMID: 23288382
