The science of doping
Nature 454, 692-693 (7 August 2008) | doi:10.1038/454692a; Published online 6 August 2008
The science of doping
See associated Correspondence: Ljungqvist et al. , Nature 455, 1176 (October 2008)Faber, Nature 455, 1176 (October 2008)
Donald A. Berry1
Donald A. Berry is head of the Division of Quantitative Sciences, chair of the Department of Biostatistics and Frank T. McGraw Memorial Chair of Cancer Research, MD Anderson Cancer Center, University of Texas, 1400 Pressler Street, Houston, Texas 77030-1402, USA.
Top of pageAbstractThe processes used to charge athletes with cheating are often based on flawed statistics and flawed logic, says Donald A. Berry.
Recently, the international Court of Arbitration for Sport upheld doping charges against cyclist Floyd Landis, stripping him of his title as winner of the 2006 Tour de France and suspending him from competition for two years. The court agreed with the majority opinion of a divided three-member American Arbitration Association (AAA) panel and essentially placed a stamp of approval on a laboratory test indicating that Landis had taken synthetic testosterone. Although Landis asserts his innocence, his options for recourse have all but dried up.
Already, in the run-up to this year's Olympic Games, vast amounts of time, money and media coverage have been spent on sports doping. Several doping experts have contended that tests aren't sensitive enough and let dozens of cheaters slip through the cracks. And some athletes are facing sanctions. Upon testing positive for clenbuterol, US swimmer Jessica Hardy was held back from the Olympic team and faces a two-year ban from the sport. She is attesting her innocence. China has already banned several athletes, some of them for life, on doping charges. Indeed, many world-class athletes will find their life's accomplishments and ambitions, their integrity and their reputations hinging on urine or blood tests. But when an athlete tests positive, is he or she guilty of doping? Because of what I believe to be inherent flaws in the testing practices of doping laboratories, the answer, quite possibly, is no.
In my opinion, close scrutiny of quantitative evidence used in Landis's case show it to be non-informative. This says nothing about Landis's guilt or innocence. It rather reveals that the evidence and inferential procedures used to judge guilt in such cases don't address the question correctly. The situation in drug-testing labs worldwide must be remedied. Cheaters evade detection, innocents are falsely accused and sport is ultimately suffering.
One factor at play in many cases that involve statistical reasoning, is what's known as the prosecutor's fallacy1. At its simplest level, it concludes guilt on the basis of an observation that would be extremely rare if the person were innocent. Consider a blood test that perfectly matches a suspect to the perpetrator of a crime. Say, for example, the matching profile occurs in just 1 out of every 1,000 people. A naive prosecutor might try to convince a jury that the odds of guilt are 999:1, that is, the probability of guilt is 0.999. The correct way to determine odds comes from Bayes rule2, 3, 4 and is equal to 999 times P/(1-P) where P is the 'prior probability' of guilt. Prior probability can be difficult to assess, but could range from very small to very large based on corroborating evidence implicating the suspect. The prosecutor's claim that the odds are 999:1 implies a prior probability of guilt equal to 0.5 (in which case P and 1-P cancel). Such a high value of P is possible, but it would require substantial evidence. Suppose there is no evidence against the suspect other than the blood test: he was implicated only because he was from the city where the crime occurred. If the city's population is one million then P is 1/1,000,000 and the odds of his guilt are 1001:1 against, which corresponds to a probability of guilt of less than 0.001.
The prosecutor's fallacy is at play in doping cases. For example, Landis's positive test result seemed to be a rare event, but just how rare? In doping cases the odds are dictated by the relative likelihood of a positive test assuming the subject was doping ('sensitivity') against a positive result assuming no doping (which is one minus 'specificity'). Sensitivity and specificity are crucial measures that must be estimated with reasonable accuracy before any conclusion of doping can be made, in my opinion.
The studies necessary to obtain good estimates are not easy to do. They require known samples, both positive and negative for doping, tested by blinded technicians who use the same procedures under the same conditions present in actual sporting events. In my view, such studies have not been adequately done, leaving the criterion for calling a test positive unvalidated.
Urine samples from cyclists competing in the 2006 Tour de France were analysed at the French national anti-doping laboratory (LNDD) in Châtenay-Malabry. This is one of 34 laboratories accredited by the World Anti-Doping Agency to receive and analyse test samples from athletes. The LNDD flagged Landis's urine sample following race stage 17, which he won, because it showed a high ratio of testosterone to epitestosterone.
Based on the initial screening test, the LNDD conducted gas chromatography with mass spectrometry, and isotope ratio mass spectrometry on androgen metabolites in Landis's sample. Such laboratory tests involve a series of highly sophisticated processes that are used to identify the likelihood of abnormal levels of plant-based androgen metabolites (from dietary or pharmaceutical sources) in a urine sample. The goal is to differentiate from endogenous androgen metabolites normally found in urine.
Mass spectrometry requires careful sample handling, advanced technician training and precise instrument calibration. The process is unlikely to be error-free. Each of the various steps in handling, labelling and storing an athlete's sample represents opportunity for error.
In arbitration hearings, the AAA threw out the result of the LNDD's initial screening test because of improper procedures. In my opinion, this should have invalidated the more involved follow-up testing regardless of whether or not sensitivity and specificity had been determined. Nevertheless, the AAA ruled the spectrometry results sufficient to uphold charges of doping.
During arbitration and in response to appeals from Landis, the LNDD provided the results of its androgen metabolite tests for 139 'negative' cases, 27 'positive' cases, and Landis's stage 17 results (see Fig. 1). These data were given to me by a member of Landis's defence team. The criteria used to discriminate a positive from a negative result are set by the World Anti-Doping Agency and are applied to these results in Fig. 1b and d. But we have no way of knowing which cases are truly positive and which are negative. It is proper to establish threshold values such as these, but only to define a hypothesis; a positive test criterion requires further investigation on known samples.
The method used to establish the criterion for discriminating one group from another has not been published, and tests have not been performed to establish sensitivity and specificity. Without further validation in independent experiments, testing is subject to extreme biases. The LNDD lab disagrees with my interpretation. But if conventional doping testing were to be submitted to a regulatory agency such as the US Food and Drug Administration5 to qualify as a diagnostic test for a disease, it would be rejected.
The problem with multiples
Landis seemed to have an unusual test result. Because he was among the leaders he provided 8 pairs of urine samples (of the total of approximately 126 sample-pairs in the 2006 Tour de France). So there were 8 opportunities for a true positive — and 8 opportunities for a false positive. If he never doped and assuming a specificity of 95%, the probability of all 8 samples being labelled 'negative' is the eighth power of 0.95, or 0.66. Therefore, Landis's false-positive rate for the race as a whole would be about 34%. Even a very high specificity of 99% would mean a false-positive rate of about 8%. The single-test specificity would have to be increased to much greater than 99% to have an acceptable false-positive rate. But we don't know the single-test specificity because the appropriate studies have not been performed or published.
More important than the number of samples from one individual is the total number of samples tested. With 126 samples, assuming 99% specificity, the false-positive rate is 72%. So, an apparently unusual test result may not be unusual at all when viewed from the perspective of multiple tests. This is well understood by statisticians, who routinely adjust for multiple testing. I believe that test results much more unusual than the 99th percentile among non-dopers should be required before they can be labelled 'positive'.
Other doping tests are subject to the same weak science as testosterone, including tests for naturally occurring substances, and some that claim to detect the presence of a foreign substance. Detecting a banned foreign substance in an athlete's blood or urine would seem to be clear evidence of guilt. But as with testing for synthetic testosterone, such tests may actually be measuring metabolites of the drug that are naturally occurring at variable levels.
Whether a substance can be measured directly or not, sports doping laboratories must prospectively define and publicize a standard testing procedure, including unambiguous criteria for concluding positivity, and they must validate that procedure in blinded experiments. Moreover, these experiments should address factors such as substance used (banned and not), dose of the substance, methods of delivery, timing of use relative to testing, and heterogeneity of metabolism among individuals.
To various degrees, these same deficiencies exist elsewhere — including in some forensic laboratories. All scientists share responsibility for this. We should get serious about interdisciplinary collaborations, and we should find out how other scientists approach similar problems. Meanwhile, we are duty-bound to tell other scientists when they are on the wrong path.
Buchanan, M. The prosecutor's fallacy. The New York Times (16 May 2007).
Berry, D. A. Stat. Sci. 6, 175–205 (1991). | Article |
Berry, D. A. Statistics: A Bayesian Perspective (Duxbury Press, California, 1996).
Berry, D. A. & Chastain, L. A. Chance 17, 5–8 (2004).
Plots show the distribution of 167 samples of the metabolites etiocholanone and 5 -androstanediol (a, b), and androsterone and 5 -androstanediol (c, d). Panels b and d show samples the French national anti-doping laboratory (LNDD) designate to be 'positive' (red crosses) or 'negative' (green dots); the values from Landis's second sample from stage 17 is shown as a blue dot. Axes display delta notation, expressing isotopic composition of a sample relative to a reference compound.
The Offense of ‘Intelligent Defense’
It didn't deserve a MacArthur fellowship, but I don't see anything defensively dull or dimwitted about Josh Koscheck turning his hips and extending his arm to shield Paulo Thiago from pouncing on him Saturday at UFC 95.
Brian Cobb butt scooting and attempting a weak double-leg doesn't warrant a Nobel Prize in defense either, but I would hardly call it brainless or idiotic.
Unfortunately, it seems they still weren't "intelligent" enough.
Let me be clear about what this is, or more pertinently, what it isn't. This is not an assertion that any of the evening's contentious proceedings would've wound up with a different victor if allowed to continue. This is not a criticism of any of the stoppages at UFC 95, nor is it an indictment of officials Dan Miragliotta, Kevin Mulhall, Leon Roberts and Marc Goddard.
In fact, given the current climate of refereeing in MMA, all of the event's referees did their jobs to the letter and their stoppages were just.
Instead, this is an inquiry into whether the refereeing standards in MMA are appropriate. UFC 95 was not an assortment of irresponsible stoppages but an illustration of the intensifying issue of what constitutes a justifiable end to a fight, as the margin between winning and losing in MMA has become hideously deformed.
Great editorial by Jordan Breen
Hermes Franca Update!
Hermes Franca Update
After having to pull out of UFC Ultimate Fight Night 18, Hermes Franca informed PDG that he will be having surgery in the next two weeks to repair a torn ACL on his knee and when he expects to return to the cage.
On how long he expects to be out of the UFC cage:
“I have to make sure that I take care of my injury and fully heal. I don’t care how long it takes as long as I am 100% when I return. Whether it takes eight months or one year, I want to recuperate to full health.”
FOR THE REST:
UFC 94 Preview w/PRO PICKS!
UFC 94 ‘GSP vs. PENN II’ Preview and LIVE Coverage! w/PRO Fighter Picks!
Hermes Franca – “Both guys are A+ level fighters and it is really hard to pick a winner for this fight. In my opinion the fans are going to be the biggest winners when this fight is over. In their first fight two years ago BJ came into the fight very strong and was able to push him around at the beginning but Georges St. Pierre won the second round and the third round was very close which lead to the split decision. I am going to stick with my pick that the fans are going to be the biggest winners.”
Rob Kaman –“This is a very tough fight to call. I think that BJ is the more animalistic fighter and that Georges St. Pierre is the more athletic fighter. Georges is always ready to go the distance and BJ’s conditioning has come into question throughout his career. This fight could end up being a fast win for BJ or decision win for GSP. They are both great fighters and like I said earlier, BJ is more animalistic in the sense that he enjoys punishing his opponents. Where as Georges St. Pierre is the more technical fighter, a pure athlete with the ability to knock out his opponents. I also think that GSP is going to have an advantage with his reach and it will be interesting to see how BJ counters that. If the fight ends in the first 3 rounds then I think it will be BJ. If the fight makes it into the championship rounds then I would have to take Georges St. Pierre.”
Silver Star Making Moves In The MMA Field
Georges St. Pierre, Rashad ‘Suga’ Evans, Karo Parisyan, Jeremy Horn, Rob Emerson and David Loiseau will have their signature line of shirts debuted at the Silver Star Clothing booth #22148, at the MAGIC trade-show in the Las Vegas Convention Center, February 17- 19, 2009.
Silver Star’s MMA collaboration includes: Rashad “Suga” Evans, who is known for defeating Forrest Griffin, the winner of the first season of “The Ultimate Fighter” reality TV show, and remains undefeated; UFC welterweight champion, Georges St. Pierre; one of the best welterweights in the world, Karo Parisyan; knockout master Rob Emerson; Jeremy Horn who have proved victorious against Forrest Griffin and Chuck Liddell; and David “The Crow” Loiseau. These fighters all have signature Silver Star tees honoring their athleticism and brute force, and more will be added to the Silver Star roster in the near future.
Silver Star is very excited to be working with these great fighters, and is presently in negotiations with Sean Sherk [one-time UFC lightweight champion] to bring him into the Silver Star fold.” said owner, Luke Burrett. “We have Rashad coming to sign autographs for fans and support the line at MAGIC, and I think this venture is going to be a very positive experience for all of us.”
Each high quality graphic tee made of 100% distressed cotton, and and includes foil accenting, with water-based dyes responsible for the powerful colors and no-hand-feel on the graphics. The art for these signature shirts is designed exclusively for the fighters, by Silver Star’s art team.
UFC Prime Time Episode 3 Ratings
MMAPayout.com has learned that the third episode of UFC Primetime (10-10:30P) delivered a 0.6 household rating — a .64 among Men 18-49, a .44 among People 18-49, a .60 among Men 18-34 and had an average audience of 662,000 viewers.
The overall average for the three week run for UFC Prime Time was 789,000 viewers.
The three week run for Prime Time has to be judged a success. The usual one week lead in to a PPV often seems a bit abrupt, and wanting at times. The multi-week format gives a sustained, staggered build that should pay off at the PPV box office. This isn’t a concept that the UFC and Spike should drag out for every PPV but hopefully the format will return to periodically to hype their bigger cards
Collegiate Wrestling Standout Makes MMA Debut February 7
A trend that Brock Lesnar and others started is about to continue. Collegiate wrestler Ben Askren, a 2-time NCAA Division 1 National Champion who went 87-0 over his last 2 years, will make his pro MMA debut on February 7 for the Headhunters Fight League in Missouri in an event headlined by former UFC fighter Din Thomas.
Meet Jon Jones - 'Bones'
Meet Jon Jones a.k.a. ‘Bones’
At the young age of 21, Jon Jones has made the most of his opportunities so far in the mixed martial arts world. Compiling a 7 – 0 record, he will be making his second appearance in the UFC on January 31st, 2009 against Stephan Bonnar. Jon took some time recently to talk with PDG about his rise to the UFC, his background and what drives him to be the best.
FOR THE REST:
Ricardo Arona eyes return to Abu Dhabi Combat Club (ADCC) in 2009
“I’ve got news, I want to fight at ADCC 2009. My idea is to talk to the organizers and see if there’s a possibility of my being invited, for all the titles I’ve won at the event. I don’t really want to go through the trials in Rio, especially since it’s already too close to the event to be able to train properly. That’s why my idea is to go straight into the main tournament, to go right up against all the toughest from around the world. I even called up Paulo [Filho] to see if he’d get motivated about participating, but he says he doesn’t want to. So I came up with the organizers’ Email, and I’ll be contacting them to see about an invite. I’m excited.”
N.Y. State Legislator Speaks Out Against MMA
The New York State Assembly’s Tourism, Arts, and Sports Development Committee begins its session on Tuesday, and among several pieces of legislature up for vote is bill 1-11458-A, designed to regulate the sport of MMA in New York. While the bill will not be voted on before next week at the earliest, Assemblyman Bob Reilly, a democratic member of the committee who represents the 109th district of New York, opposes the legislation and recently spoke to MMAWeekly.com about his feelings on the sport and attempts to bring it to the Empire State.
Ask the Doc: Dr. Benjamin on MMA, concussions and mental illness
In the wake up the recent deaths of MMA fighters Evan Tanner, Justin Levens and Justin Eilers, many fans are struggling to find a correlation.
While there may be none, some people, including researcher Chris Nowinski, see combat sports (and concussions) going hand-in-hand with depression and mental illness in later life.
In his latest “Ask the Doc” column, combat-sports specialist Dr. Johnny Benjamin discusses the topic, specifically as it relates to MMA, and why there’s so little relevant information available on it. He also gives two reasons why there’s so much controversy about trying to connect the two.
Q. Dave Meltzer recently wrote an article discussing three unrelated deaths of former UFC fighters; two of the fallen fighters were known to suffer from depression. Metzler notes Chris Nowinski’s studies correlating boxing to mental illness later in life. Many fans and fighters feel MMA is safer than boxing. MMA fighters are more likely to suffer a few concussive blows (i.e. knockouts) as opposed to a myriad of “padded” shots as in boxing. Many fights end without any substantial head shots. Is it fair to assume the same long-term consequences in MMA? Steve in Los Angeles
A. Steve, you are asking the million-dollar question.
Also, my hat is off to Dave Meltzer for even attempting to intelligently discuss this difficult subject. This and apparently many other topics are difficult for the MMA faithful to discuss reasonably without deteriorating into emotional outbursts, personal attacks and worse. Good articles are written to make intelligent readers think not to defame anyone or anything.
Do repeated blows to the head make MMA participants more likely to suffer with depression or other forms of mental illness later in life? It’s a great question and one that needs to be investigated and researched now rather than adopting a wait-and-see approach.
The current form of MMA is relatively early in its life cycle. We are talking less than 20 years. As major sports go, MMA is still in its infancy. Therefore, I would assume nothing with respect to the potential long-term health consequences. But as you’ve implied, I would learn a few things from the collective experience of other, more mature (older) contact and combat sports. It is also prudent to be proactive when it comes to fighter safety, since it is very difficult, if not impossible, to fully restore competitors’ mental health once it has traumatically been taken from them.
Retrospective (looking back after the deed is done) studies of professional athletes involved in boxing, football, soccer, hockey and rugby seem to suggest a link between repetitive blows to the head, concussions (MTBI, which is minor traumatic brain injury) and depression or dementia. This is a very controversial statement for at least two major reasons.
First, it is very difficult to prove a direct causal relationship. Did the accumulation of blows to the head directly cause permanent brain injury that led to depression or dementia? Or are the athletes that participate in these sports on the professional level more prone to depression to begin with? Do their inherent, aggressive, possibly somewhat antisocial personality traits allow them to achieve in these sports at a high level? Simply put, they may be a little crazy or unstable to begin with. That’s why they do so well in these contact and combat sports. (Absolutely no disrespect is intended to those that suffer with mental illness.)
Second, the powers that govern these major sports fear the cost associated with acknowledging a relationship between participation in these sports and subsequent dementia and/or depression. Simply put, if the sport caused it, somebody is going to have to pay for it. Forget lawsuits and punitive damages (which will most certainly come); just the cost of long-term care would be staggering. No one is prepared to pay that without a serious fight.
Professional MMA fighters should assume nothing and be prepared for everything. When your favorite fighters’ careers are over and no one is any longer screaming their names, paying them sponsorship fees and buying their pay-per-view appearances, who is going to pay their medical expenses and provide assistance to their often forgotten caregivers? Your heroes have families too.
Again, Steve, it’s a great question but not one anyone can answer definitively at this time.
Dr. Johnny Benjamin is MMAjunkie.com’s medical columnist and consultant and a noted combat-sports specialist. He was also recently appointed to the ABC’s medical advisory team and will help review and refine the unified rules of MMA. Dr. Benjamin writes an “Ask the Doc” column every two weeks for MMAjunkie.com. To submit a question for a future column, email him at askthedoc [AT] mmajunkie.com, or share your questions and thoughts in the comments section below. You can find Dr. Benjamin online at www.drjohnnybenjamin.com, and you can read his other sports-related articles at blog.drjohnnybenjamin.com.
Check out more UFC News at MMAjunkie.com. This story originally appeared on MMAjunkie.com and is syndicated on Yahoo! Sports as part of a content-partnership deal between the two sites.
Naked man arrested at Wal-mart looking for a MMA fight
Officers with the Covington Police Department arrested a nude man looking for a fight Sunday at the Wal-Mart on Industrial Boulevard.
According to the incident report, Jeffrey Pickett said he wanted to be a mixed martial arts (MMA) fighter and was looking for someone to battle. Pickett, who undressed in parking lot, said he knew he could find someone to fight with if he were nude.
The ’09 Wish List
Pre-“Ultimate Fighter,” pre-Affliction, pre-state sanctioning, fans will remember the UFC’s modest promotional resolution for 1999: run nine shows. (The campaign was dubbed “9 in ’99,” which is what happens when you can’t afford a marketing department.)
A sad testament to the state of the sport at the time: They could only manage six.
Ten years on, and the UFC is likely to meet or exceed the 20 programs it ran in ’08. Wishes have become largely extraneous, since most requested matches wind up happening sooner or later. The sport’s devotees have everything they could possibly ask for -- free shows, top talent, capable management.
This space, though, works tirelessly to find something to complain about. Some hoped-for events for the New Year: