Braun, Baseball And Getting It Right

While we wait for more facts, let’s dispose of two points currently circulating about the Braun case. The first take is that the Braun case was decided on a “technicality”. This argument is based on a wrong idea of how drug testing works. The labs don’t search for testosterone in urine the way you and I would search for a worm in an apple. The testing process can take a day or longer, and there are many steps involved. All of these steps – including sample collection, sample storage and sample preparation, along with the chain of custody that documents the journey the sample takes along the way – are integral parts of the testing process, no matter how technical a step may seem to us. Labs don’t skip steps and offer up excuses later. When a lab skips a step, it tosses out the result and starts over.

In order for a lab to use a test method, the test method has to be validated – the test itself must be tested. This requires the lab to document the process they’re going to use in testing, and then try out the process to see how it works. If the test is validated, that means that the process has been shown to produce reliable results. Labs are no better than their validated processes. Fail to follow the process, all of the process, and any assumption of validity disappears.

If lab test validation does not speak to you, then consider this: when athletes test positive, they are not permitted to argue that the positive test is a “technicality”. They can’t argue that substance in question did not really enhance their performance. Consider the case of cyclist Alberto Contador, who is facing doping sanctions for the presence in his urine sample of 50 picograms of clenbuterol. A picogram is one-trillionth of a gram. That’s not very much stuff, but it was more than enough to strip Contador of his 2010 Tour de France title and to bar him from this year’s Tour.

When an athlete is caught doping, we don’t want to deal with questions of how it happened, or why, or whether the doping did the athlete any good. Those questions take too long to answer, if indeed they can ever be answered. We just sanction the athlete and move on. In the anti-doping world this is called “strict liability”, and anti-doping folks like Travis Tygart defend strict liability to the skies. Strict liability has to work the same way in reverse. When the anti-doping agencies make mistakes, when they fail to follow the procedures they themselves put in place to ensure a fair process, we can’t spend months trying to figure out how the mistake affected the result, or how the result would have come out if the mistake hadn’t been made. We need to toss out the result and move on.

Let’s move on to a second issue: with the mistake made by MLB, can we ever bring the Braun case to a close? “Today was supposed to bring a resolution,” wrote David Schoenfield at ESPN.com. But the system didn’t work, and now we’re “left stranded.” “We don’t know what Ryan Braun did or didn’t do.”

My reply to old friend David is this: we have resolution, or at least, we have all the resolution we’re ever going to get. With drug testing, there are only two classes of athlete: those caught, and those not caught. Braun is now in the second category, the category of the un-caught.

We have every reason to believe that athletes can dope and get away with it. Consider the evidence. Barry Bonds never failed a drug test. Neither did Marion Jones. Numerous cyclists, from Floyd Landis to Tyler Hamilton, are confessed career dopers and were able to get away with doping for years. There are plenty of PEDs for which there are no tests, or where the tests never seem to actually catch anyone. As Victor Conte recently pointed out, some PEDs are so fast-acting that they clear in a matter of hours – unless MLB wants to conduct surprise tests at 3:00 a.m., an athlete can use these drugs without fear of being caught. Conte thinks that 30-40% of baseball players are using PEDs, and while that’s obviously a wild guess, it’s a wild guess by an expert that knows a lot about the subject.

Sorry to be so gloomy. But having won his case, Braun gets to be classified with the other 1,196 major leaguers who tested negative in 2011. He may have doped. He may not have doped. Baseball couldn’t prove he didn’t dope. That’s the best we can say about anyone subjected to PED testing, including the squeakiest-clean baseball player you might imagine.

68 thoughts on “Braun, Baseball And Getting It Right

  1. guest

    But wait…did leaving the sample in a basement cause the sample to contract SYNTHTIC testosterone? He got over on a technicality, never denied using, and was found to have a substance that he knowingly ingested as a synthetic performance enhancing drug.

    • LarryAtIIATMS

      um, guest. If you really, REALLY want to discuss how the labs test for exogenous testosterone, I'll be happy to engage you. The key fact to note is this: testosterone made in a lab is the same stuff as testosterone produced by the human body. It is chemically identical. If I put a molecule of the natural stuff next to a molecule of the synthetic stuff and gave you a godlike ability to examine the two molecules down to their atomic and subatomic properties, you could not tell me based on what we currently know which molecule is which.

      Lab testing for synthetic testosterone is based on a theory that synthetic testosterone will contain slightly less carbon 13 than natural testosterone. For those not up on their isotopes, let me explain. Testosterone is a complex molecule made up of three kinds of atoms: carbon, oxygen and hydrogen. Carbon atoms are found in three different isotopes. The most common isotope is carbon-12, where the atom has 6 protons and 6 neutrons. Around 1% of the carbon in the world is carbon-13, which has six protons and 7 neutrons. Certain plants, including soy, "prefer" carbon 12 — these plants contain a bit less carbon-13 than the 1% you'd normally expect to see. Synthetic testosterone is normally manufactured from soy. So if the testers find less carbon 13 than they'd normally expect to find in the testosterone (really, testosterone metabolites) in an athlete's urine sample, that's grounds for concluding that the athlete used artificial testosterone.

      But it's not as simple as all that, which I'm happy to discuss in nauseating detail. But let's try to keep the argument simple. The test used to detect artificial testosterone — isotope ratio mass spectrometry — is a very difficult test to perform. In effect, the testers are counting neutrons! The fact that they can do this sort of thing at all is a scientific marvel. But all agree, this is a test that needs to be performed exactly right — mess around with your protocol, and your neutron count will be inaccurate or unreliable.

      So guest, keep these two facts in mind. First, you can't "find" synthetic testosterone. At best, you can find a subtle property associated with a population of synthetic testosterone molecules. Second: that property requires you to run a sophisticated test with great precision, in according to protocol. In short, you can't conclude you've found synthetic testosterone UNLESS you follow test protocol.

      Follow up if you have questions.

      • GabeLezra

        This is a great response, and a great article.

        I come from a background of writing about soccer, and part of the problems we've seen crop up during the Braun case are major issues in dealing with soccer doping, as the drugs of choice in soccer tend to take very careful, very complicated tests to reveal. And with FIFA acting like MLB in 1994, well, there's just no chance soccer will have major reform (in my opinion). Not sure if this is relevant, though it is interesting to me.

        • LarryAtIIATMS

          Gabe, is FIFA drug testing under WADA supervision? I should know the answer to that question but I've never looked it up.

          • GabeLezra

            As far as I know, WADA allows FIFA to do their own testing, but "supervises" it in a sort of annual check of all their systems. I don't know if this is exactly how it works, but I do know that, in general, FIFA tends to test for the wrong things, and doesn't test enough players (and isn't careful about it). They are very vigilant about anabolic steroids in particular, but that's the last thing FIFA should worry about (well, not the last) because players are much more likely to do endurance-enhancing drugs like EPO and use things like HGH for injuries.

            There's also a cult surrounding these witch-doctor figures, like Bayern Munich's Dr. Hans-Wilhelm Müller-Wohlfahrt (yeah, awesome name), who all do these very weird experimental treatments that stretch the boundaries of what is really legal. Plus, it doesn't help that FIFA doesn't think that there's any reason to dope in soccer because it's a finesse team sport according to Sepp Blatter (yeah, he said that, or something exactly like it).

      • TimmyD

        the head scientist for the WDA states that temperature make zero changes to the specimen. They freeze it as well in some cases. Braun signed 3 seals that were uncompromised. The Montreal lab, considered the best in the world said the sample wasnt compromised. These are experts in the field. If this went to a court instead of the 1 arbitrator, he would be guilty. The head scientist has takes totally the opposite of yours, but I suppose youre the expert?

        • Larry@IIATMS

          Timmy, there is no such organization as the WDA. You’re probably thinking of Travis Tygart at USADA. Travis Tygart is an attorney, not a scientist. Maybe this isn’t the person you’re referring to. As there is no role such as “Head Scientist” at any organization with which I am familiar, you’re forcing me to guess who you might be talking about.

          You say that temperature makes zero changes to a specimen. The better way to think about this is, urine undergoes changes over time, and these changes can be slowed where the urine is stored at the proper temperature. There is no scientist who would argue otherwise. The more relevant question is whether urine would undergo any changes over time that could affect the measurements made in the Braun case.

          The anti-doping agencies would get their tails handed to them if they tried to argue these cases in a court. In court, these agencies would not have the luxury of “strict liability” and they’d be subject to deposition and discovery in accordance with the modern rules of evidence. On this point, I AM an expert.

          I tell you what. Read Will Carroll’s piece on Kindle. It will cost you $0.99, but the money goes to the Jimmy Fund. Will Carroll IS an expert. Then if you like, come back here to discuss. http://amzn.to/xqVsyo. It’s only 3-1/2 pages long, but don’t take offense, it’s 3-1/2 pages of information you do not appear to know.

  2. danrizzle

    Larry, I think you make an excellent point about the consequence of a positive test. Put simply, an athlete is strictly liable for a positive test of any substance prohibited under the CBA, whether or not the substance actually enhanced performance or is even capable of enhancing performance. That's the essence of strict liability. Is it so much to ask that the tester be held strictly liable to honor the collectively-bargained process? According to Cot's, Braun will make $7 million this year; a 50 game suspension would cost him over $2 million. That kind of punishment is too draconian for any mishandling of the sample or departure from testing protocol to be deemed a mere technicality.

    • LarryAtIIATMS

      Dan,I love it when guys like you come onto my site and make my arguments better than I did, in fewer words! Thumbs up.

  3. I feel better when Larry is on the scene. Good to have you back!

  4. Chris Dial

    "Labs are no better than their validated processes. Fail to follow the process, all of the process, and any assumption of validity disappears."

    One of the key components in validating an analytical test method is sample solution stability (at room temp and refrigerated) and a robustness characteristic which would include testing samples *when the process isn't followed*, so it isn't strictly correct to say not following the method means the sample data is tossed. If that happens, you have to check the deviation against the validation parameters which may show that even when tested outside the method, the result is still valid.

    Hope that helps.

    • LarryAtIIATMS

      Chris, you may want to indicate your experience and background. By inserting the word "robustness" into the dialog, you're suggesting that you may have expertise to bring to this discussion. In case I'm not being clear enough, I'm paying you a compliment. Thumbs up.

      All this being said, I once wrote a 12 part piece on ISO 17025 and how lab validation works in the context of WADA drug testing. The measure of robustness is really the same as the measure of intermediate precision and reproducability, see for example http://bit.ly/zFQAQC. I discussed these concepts here: http://bit.ly/wpNYEg. I'd be happy to discuss these concepts with you in greater depth.

      But to try to put it in simplest terms, "robustness" is not really a measure of what happens if the lab fails to follow its own process. Logically, we can understand that it's not possible to measure what happens when a lab deviates from process unless the nature and extent of that deviation is itself subjected to tests. So, for example, a test protocol might indicate that urine samples should be stored in a controlled environment at a specified temperature, or it might be sufficient to say that urine needs to be stored in a "cool" location. Presumably, if the specification is for a "cool" location, then the validation process considered what might be the temperature range in a "cool" location and determined that variations of temperature in this range would not affect the test results. Something similar would be going on when the procedure calls for overnight shipment of a lab sample by Federal Express — again, the validation procedure would consider the range of conditions to which such a shipment would be subject, and would (hopefully) determine that these different conditions would not affect the test results.

      My point is that robustness should consider the effect of different conditions WITHIN a testing protocol. It cannot and (to my best understanding) DOES not consider what happens if the protocol is not followed. Please follow up if you disagree or if you think I'm missing something.

      • Elliott

        Larry,
        The validated testing method for labs which do these tests almost certainly have to account for the fact that samples are not going to be kept in a "cool" temperature by the procedures themselves. Even if the collection officer had collected the sample, gone to fedex and shipped it immediately the sample is not handled by fedex or ups as temperature controlled. It is kept at room temp in the store, on the truck, in the plant, on the plane and then on a truck again until it reaches the lab. As a DCO myself, it is SOP for samples collected after the last UPS/Fedex pick up on a Saturday or Holiday (typically 6pm) that samples are stored overnight with the sample collection person. Personally I have a separate fridge in the basement for these samples. These samples are not stored in Tupperware as some outlets and the less informed are reporting, but rather Berranger kits engineered specifically for this purpose. They are glass bottles with tamper proof locking lids that require the breaking of the bottle itself to open, those bottles are then individually sealed in plastic, then placed in a secure styrofom insulated container and sealed. The container and all the bottles are individually numbered for the lab. So the fact that it was stored in a refrigerator does go along with the validated method and standard operating procedures. Also as a chemist, and you sound like a scientist yourself, I want to know what science Brauns legal team had that I have never seen in which a "clean" urine sample produces testosterone by being kept in any conditions. Also a 20:1 T:epiT ratio is really far out of the normal range. I don't see how the Arbitor found for him and if I was Braun I would be looking to make the Arbitor's report public to validate myself.

        • LarryAtIIATMS

          Elliott, great reply. Thanks. Thumbs up. DCO means what? I'm guessing that D is for doping and O is for officer or official, but C could mean collection or control.

          I'm not a scientist, just someone who's spent way too much time reading scientific reports and talking to scientists.

          In my post, I cited Drew Silva's piece to the effect that maybe MLB DID correctly follow its own protocol. The picture is not clear on this point, as we don't have all the facts. A good listen this morning is Will Carroll on WEEI, see http://bit.ly/xTNEGU. From what Carroll says, there's no way for us to know today if the collection officer used a Berringer kit or something comparable, or even if the sample was refrigerated.

          If the tester followed the validated protocol, then my arguments about not following protocol are academic. You could also argue if you like that if MLB's protocol requires urine samples to be shipped via Federal Express without local overnight storage, then it will be impossible for MLB to do urine testing after Saturday night and Sunday day games. I think it's MLB's responsibility to clarify all this in their protocol.

          If you're a chemist, I'd be crazy to go anywhere NEAR your question of how improper urine storage can change a T/E ratio! I know in general terms that there is concern in the protocols about pH levels and the possible effects of bacterial contamination on the accuracy of tests. You haven't argued that storage conditions could NOT affect the test results, so I can agree with you: I'd also like to see the science argued by Braun's legal team.

          But as for the 20:1 T/E ratio (and I'm reading that the measured ratio may be higher), I can't say this is extraordinary. I once wrote a long piece about the uncertainty in testosterone testing, where I cited a study of measured results after testosterone administration. The study is no longer available online but you can read my discussion of it here. http://bit.ly/zM4YPp. One subject in the study had a measured T/E ratio of over 90 to 1. The measured ratio is not exactly an accurate statement of the amount of testosterone found (or allegedly found) in an athlete's system, as the number also depends both on the timing of the measurement (testosterone clears the body quickly, so differences in measured T/E depend a great deal on the timing of the measurement) and the presence or lack of measured epitestosterone.

          Keep those comments coming, and follow up if you disagree or think I've missed something.

          • Elliott

            Larry,
            DCO is doping control officer. My point is that regardless of the storage temperatures, and I'll quote Travis Tygart here "And, importantly, they don't need to because synthetic drugs don't magically appear in urine because it took 48 hours versus 20 minutes to get to the laboratory." I talked to a handful of synthetic chemists in the lab today and they all agreed, storage temp would not have anything but a degrading effect on the testosterone in the sample. Now my lab is not a drug sample processing lab but rather a pharma company discovery lab. Absent the science behind the legal argument from Brauns attorney, he tested positive and had it over turned on procedure due to an arbitor who did not know the SOP. That is what is on MLB, not proving that what the collector did was approved protocol.

          • LarryAtIIATMS

            Elliott, Travis Tygart has never met a positive doping result he didn't like. To listen to Tygart, you would think that a lab never made a mistake, a DCO never made a mistake, an anti-doping agency never made a mistake and (naturally) he's never made a mistake. Worse, to question Tygart is to take sides against clean sport. The position he's taken in the Braun case is about as surprising as a sunrise.

            Do your lab buddies also conclude that storage conditions could have NO degrading effect on the epitestosterone in a sample? If that's the case, rewrite the protocol to make it clear that urine samples can be stored any which way, or that the storage requirements are built into the protocol solely to guard against false negative results. Then have the protocol validated.

            Look. It's not that I disrespect your opinion. But I've read ISO 17025. There's nothing in there about harmless error. ISO 17025 is pages upon pages of mind-numbingly technical stuff, all arguing that following process is everything.

          • BrienJackson

            A couple of points:

            1. What Larry said: the WADA and USADA have zero credibility, as far as I'm concerned, and I quite literally couldn't care less what they have to say about anything.

            2. The people carrying on as though arbitrators sit in judgement as Lords High and Mighty deciding which agreed upon and legally binding rules we're going to casually ignore today are amusing to me. Somewhat frightening, of course, but also amusing.

        • Chris Dial

          Elliott,
          there is no doubt that during development of large scale testing protocols such as an MLB drug testing program, samples *should* be subjected to what we call a "Shipping Study". That means you send samples through the normal handling procedures and verify a spiked recovery (for example) retains all label claim. And there is also the option of a "Custom Critical" temperature controlled shipping using a device like a TempTales that is packaged with the sample and continuously records the temperature the sample is exposed to.

          • LarryAtIIATMS

            Chris, terrific additional information. I will add "shipping study" to my list of lab method validation vocabulary. Thanks again. BTW, I assume that baseball knows about "custom critical" temperature controlled shipping, particularly now that they're testing blood as well as urine.

          • Chris Dial

            WRT shipping studies and custom critical, I wouldn't assume anything about MLB or anyone else. Shipping studies haven't been SOP until very recently, but a drug that is tested and fine in the warehouse for a month – when suddenly the truck transferring it gets stuck in a snowstorm and experiences 2-3 freeze/thaw cycles – well, we can have concerns.

            The FDA is more progressive than most agencies (IMO) – not just in looking at news ways to ensure drugs are safe, but relatively aggressive in moving toward electronic data/filings.

      • Chris Dial

        Larry,
        I am a pharma chemist (technically I *was* due to promotions). I have developed and validated methods for all manner of pharma products, as well as written internal SOPs for these things and performed countless investigations.

        Robustness is beyond different analysts (the general usage of repeatability and intermediate precision). What I am refering to is where you look to see the effects of small changes to determine if you have any critical parameters. Your linked LC/GC article hints at this. It is things like setting the wrong flow on the LC, or the wrong column temperature. Where the method proscribes, say 1.o mL per min and the analyst inadvertently sets 1.1mL/min flow, does it impact the results – does the system suitability still meet method requirements. These changes are *definitely* outside the standard testing method/protocol, and absolutely is part of a proper FDA guided MV.

        Another thnig I mentioned and I haven't see covered was the sample storage conditions. Storing at 5C (really 2C-8C) is done to minimize activity, and is broadly accepted. However, in method validation, one would typically demonstrate solution (sample) stability for 1 day, 3 day, and out to a week. Mostly because you want to make sure you cover the typical amount of time you may need to perform analysis *and* have solution stability in case there is a sample issue (positive test or whathaveyou) that the original sample can be re-analyzed within solution stability.

        Now, these are smart ways to enhance the quality of an analytical method, and maybe this method doesn't do that. Biologic samples usually require more significant cooling: -20C or for most biologics now, to stunt water reactivity, down to -75C. Either way, the method should have demonstrated sample stability, and we can know whether or not the sample was compromised due to storage temperature.

        I have no dispute with – the sample was probably not secured and tamper-proof.

        • LarryAtIIATMS

          LEARNING … SOMETHING … MUST … RESIST … CHRIS' WILL IS … TOO STRONG …

          Cool! Learning something.

          A question before I dive into more of what I know: following your example on LC (for those interested, that's liquid chromatography) flow, let's say that your validated method requires the technician to set the machine for a 1.0 ml per minute flow, and that the method is validated for robustness to show that the method is still fit for purpose even if the flow is 1.1 ml/minute. Let's also say that the lab tech believes that the actual requirement is for 1.1 ml/minute, and that this flow rate is documented and caught on review. Do you toss the test result out? After all, the actual flow might have exceeded 1.1 ml/minute, because the machine is slightly out of calibration, or because the machine's flow has some natural variation, or because the tech might have set the dial a little past 1.1.

          • Chris Dial

            It's complicated, Larry. If a lab developed/validated the method and has *easy* access to the MV report, then they will try to see if the error was within the validation parameters. If the lab simply transferred the capability (via an N=6 comparison or a few MV parameters analysis), then they would throw the data out. THis is a HUGE reason for taking two samples, or splitting the first sample.

            Also, today, flow settings are digital. And NOTHING is out of calibration until you prove it is. ;-)

          • LarryAtIIATMS

            I'm an analog guy. Also, inspired by Spinal Tap, if I knew how to perform GC/MS, I'd do it with a machine that goes up to 11.

  5. Allen

    I recall reading when the news of his positive test first broke that an unknown number of the players had successfully challenged positive tests prior to the finding being made public, and that Braun's positive test was leaked to the media before he had a chance to challenge the result. Is this accurate? If so, it suggests that the testing process is seriously flawed.

    • BrienJackson

      Jimmy Rollins claimed he knew of players who had successfully challenged a test result before, but the results were kept private. Positive test results are only supposed to be made public after the process has been completed and the suspension is imposed.

    • LarryAtIIATMS

      Allen, what Brien said. Also, there's a rumor that one minor leaguer successfully challenged a doping positive in arbitration. Honestly, we can never know for sure that such things happen, because the entire process is supposed to be confidential. Except, of course, when it isn't.

  6. jay_robertson

    Oh my. Thank-you for the "boring" details – I had no idea they were checking for isotopes of atoms in order to "determine" if there is synthetic testosterone. Or, as you say, the chance of it.

    When MSM reports a failed drug test, the impression that is given is that there IS a worm in the apple. Not that the worm may have crawled over a leaf on the tree and taken a leak.

    With procedures not followed, the leak becomes almost criminal. I can't see Ryan Braun being happy – I guess its better than the guys on death row who get released after 20 years on DNA evidence, but still – not a big career booster to even be associated with the "Cheat" word – regardless if its true or not.

    • LarryAtIIATMS

      Jay, it's one of my pet peeves. Every time I read that the testers "found" artificial testosterone in Braun's urine sample, smoke poured from my ears.

    • ron

      The leak is criminal I believe. I wonder if a federal investigation has been started reagrding the release of federal protected health information(HIPPA)…any thoughts?

      • LarryAtIIATMS

        My first thought is to flee to a safe location, where they've never heard of HIPAA.

        Baseball's drug testing program requires all players to sign a HIPAA waiver. Remember that we DO learn the general results of positive drug tests. Moreover, HIPAA is not a guarantee against release of confidential medical information. HIPAA simply provides rules to protect such information from disclosure. If MLB took reasonable precautions to safeguard this information, then there would be no grounds to complain under HIPAA even in the absence of a waiver.

        NOTE: this is far from any kind of reliable legal advice. I'm describing HIPAA in THE most general terms imaginable. The application of medical privacy rules to sports anti-doping efforts is a complex area of law. In other words, I could be wrong, I probably AM wrong in certain details, don't try this at home, IMHO and YMMV.

  7. You make this science teacher proud.

    • LarryAtIIATMS

      Gotta be a first time for everything.

  8. The MLB Drug Agreement indicates (well, implies) that specimens are to be given before a game, but if a player cannot deliver one he is permitted to return to pre-game exercises but will be under the watchful eye of the collector to make sure he doesn’t involve himself in shenanigans.

    http://leanblitz.net/2012/02/process-breakdown-me

    • LarryAtIIATMS

      Chad, good post. A lot of people around the 'net (me included) assumed that the sample was given after the game. But you're right, the rules appear to require the sample to be given pre-game. As the October game started at around 1 p.m. local time, this would have given the collection officer plenty of time to get the sample to an open Fed Ex location.

      • Elliott

        not necessarily, the athlete is notified of selection before the game but cannot produce a sample until afterward, that is when the collection would take place. He would be kept under supervision throughout that period though, so someone would have been in the dugout with him presumably.

        • LarryAtIIATMS

          Elliott, agreed. The rules seem to assume that if the athlete can produce a pre-game sample, that's when the sample would be taken. I assumed that this is what happens most of the time. I could be wrong. Thanks for clarifying.

  9. Eli

    I disagree with the premise by which he was acquited. I hate bringing things like this but I believe if it had been another Dominican player he would have been found guilty regardless.

    • LarryAtIIATMS

      Terrific point, though the problem you've mentioned has nothing to do with being Dominican. Athletes like Ryan Braun and Alberto Contador can afford one level of due process in the anti-doping system. The typical athlete cannot afford to do anything but accept the testing results and move on. Of course, this is not just a problem with drug testing.

    • BrienJackson

      Well, it's also probably the case that had he been just another Dominican player no one would have cared to break the confidentiality rules in the first place.

  10. LarryAtIIATMS

    Elliott, I liked your first comment better. With all due respect, I don't understand the scientific basis of concluding that "there is something up" when the T/E ratio is 20:1, regardless of storage conditions. I think we should strictly determine the direction in which we reason, from cause to effect, and at minimum make sure we don't reason in both directions at once. In drug testing, we reason from process to result: if the lab follows its validated process, then we can trust the result. If not, not. I know of no exception to this rule when the T/E ratio is over 20:1. I do not understand how one starts with the result, ANY result, and reasons backwards to validate (or excuse deviations from) the process — that is, unless you're validating a test on a known standard or population, and you know before you perform the test what the result is supposed to be.

    I'll use an analogy that's not perfect but I think it will serve: if the arson investigator concludes that he can't prove that Mr. X started the fire, we don't argue against him by saying it was a really big fire.

    You're arguing backwards in a second important way: you're asking me to prove how MLB's deviation from validated process could have caused an invalid result. That's not up to me to prove. Lab testing requires validated processes. Argue if you like that MLB followed the validated process. Don't ask me to make meaning out of a process that was never validated. I'll argue that no such meaning is possible, and I'll freely acknowledge that I have no idea how to put the egg back together once it is cracked.

    Regarding your statement about T/E, I assume you're aware that NO ONE thinks that T/E measurements are good for anything other than screening samples for IRMS testing. If you need info on this, just say so.

    • Elliott

      Larry,
      later I'll have more time to digest this whole thing, today I was simply furious after listening to talk radio and espn blast the sample colletor for doing his job as he was instructed. As far as the testing goes I made the second argument partly as a scientist and partly as a cynical fan. I'm going to think some more on this tonight when I have some time. Thanks for the conversation. and feel free to email me in the future

      • LarryAtIIATMS

        Yeah, I completely get that. Not that we actually know what the guy did, but it appears he thought in good faith that he was following standard operating procedure.

  11. LarryAtIIATMS

    Ron, even if we assume that someone leaked this information, I do not know that this someone works for MLB. It could have been someone on Braun's team for all I know at present. Or it could have come out accidentally — maybe someone left a key document on the copy machine at the Kinko's.

    • ron

      and assuming leads down a dark dangerous road…I thank you for responding to my thoughts…I did enjoy the Testosterone discussion above…well thought out and informative…

  12. forged

    So how does the players union and MLB improve the process so that 1) false positives in testing occur less frequently, and 2) that the news isn't leaked until after the initial result has gone through arbitration?

    As Larry said in his original (very insightful) post: "We have every reason to believe that athletes can dope and get away with it."

    • LarryAtIIATMS

      forged, please start with my discussion above with Ron. I don't know enough about how the leak happened to comment on how to prevent future leaks.

      As for preventing false positives … well, some will dispute that the Braun case is technically a "false positive". It can be argued that the process worked here, that Braun's test results have been tossed out and that if not for the public disclosure, all would be as it should be. I'm not quite as full of sunshine as this, but we have to be realistic. There were something like 3,600 samples tested under the MLB program in 2011, and this number will be larger in 2012. With these numbers, there are going to be problem cases requiring special handling and arguments back and forth. We can argue that the rules governing urine sample handling should be revised to make it clear how to handle samples taken over a weekend. I don't know that anything else is required in light of the Braun case to make the PROCESS work better. My purpose here is to get people to understand the process.

      When you quote me about getting away with doping, the problem you're mentioning is false negatives, not false positives. The false negative problem is, in my view, a limitation on what can be accomplished with drug testing. This limitation is worth its own post, some day. But in brief: while the tests are getting better, the doping grows increasingly more difficult to detect. If we're waiting for the day when the testing can catch all of the doping, at best we have a long wait. My own point of view is that drug testing serves as a valuable deterrent — it can discourage the basically honest athlete from doping, and might cause the doping athlete to dope more cautiously. These are good things. I don't think we can expect any more than this from an anti-doping program.

      • forged

        Thanks for the response. Good food for thought.

  13. Bill

    I understand the concern about the science and integrity of samples. I really do. But in the end, I stopped caring about the "integrity" of the game a long time ago. I believe that most of all of the players of today came up in a "balme the other guy" society and mentality. Every one of them has tried to lie their way out of it. What I still DO care about…are the records set by Hank Aaron, Willie Mays and Babe Ruth. They weren't perfect either. Hell, I saw Hank smoking a cigarette in the dugout. But I'd guess their peformance wasn't enhanced by injections or pills.

    • BrienJackson

      /facepalm

      • Bill

        Cute! You were playing WOW while I was playing baseball.

        • BrienJackson

          I'm sorry, my sarcasm meter must be off, right?

          (Obviously I'm snarkily referring to greenies here, if anyone needed that said explicitly)

          • Bill

            It's OK Brien. You can have your Barry Bonds and your Ryan Braun and your Manny Ramirez. I'll take Hank Aaron and Willie Mays, setting records against the top pitching of any generation, any day. And you go ahead and compare accusations of "greenies" to proven steroid and performance enhancing drug use if it makes you feel, well, bigger and stronger.

          • No, your players are every bit as tainted as “mine.” There won’t be any agreeing to disagree on this, you’re just being the worst kind of willfully ignorant.

          • LarryAtIIATMS

            Bill, the argument you're having with Brien is impossible to resolve. You can't really compare greenies to steroids and say, this drug provides more performance-enhancement than that drug. We can't get 100 baseball players to participate in a double-blind study where we give some of them amphetamines and some of them testosterone and see how their seasons go. The best study on steroids was performed by master statistician Nate Silver, and he concluded that steroids appear to be connected to greater power production but not strongly. Yes, I've heard people scream bloody murder about how could this be, but I'm just reporting what I've heard.

            Here are two facts that I think I CAN defend: (1) today's athletes don't take "greenies" largely because the stuff is very easy to detect in testing, and not necessarily because there are more effective ways to cheat, and (2) amphetamines are very, VERY dangerous drugs if taken without careful medical supervision. Not to say that recreational use of anabolic steroids is a safe thing, and not to deny the difficulty of trying to argue that drug "A" is more dangerous than drug "B", but amphetamine use scares the hell out of me and I would never make light of it.

  14. jay_robertson

    Oh Flloook. Just read Sherman's Hardball – what I said about Braun being under suspicion for the rest of eternity. Proof in his last para – I won't do a cut and paste, so I don't violate rules – but…

    He maintains that IF Braun wants to clear his name, he needs to hire an outside, independent lab, and have weekly BLOOD tests, and then publish the results. OK – so we don't care about box scores any more – even if the poor guy did that, folks would maintain that he could just take the drugs around the testing and still dodge.

    I'm sorry – but it looks like the inmates want to run the asylum.

  15. Sabrina

    The best thing about waking up in the middle of the night is reading a great article by Larry. I agree 100%, lab process not followed then results are inconclusive. The question of whether Braun is guilty or not doesn't matter anymore. The question becomes why wasnt protocol followed. For some reason this case reminds me of Roger Clemen's trainer. Why would you keep needles that you injected him with? Now my new aurstion is why would you keep a urine sample at home?

    • LarryAtIIATMS

      Y'know Sabrina, I don't POST these pieces in the middle of the night. You can keep up with IIATMS and still get your beauty rest! Thanks for the nice words.

      Keeping the urine sample at home is better than leaving it in the trunk of your car. But keep Elliott's comments above in mind: it's not the collection agent's fault if he was doing what he was told to do, or he wasn't told what to do with a urine sample if the Fed Ex office was closed.

      • Chuck

        Larry, excellent article and thoughtful comments by the scientific community. One problem. If all that matters is obtaining the sample and testing it (no matter how it is secured or stored or transported or tested), why have a legal proceeding? The answer of course is that our system of punishment invariably permits the accused to defend himself. That is done usually by attempting to prove he did not commit the offense or by successfully challenging the evidence put in by the prosecution. Braun did the latter. Since MLB had their own guy on the panel, and no MLB player had ever prevailed at this stage, I would conclude that either his lawyer is Clarence Darrow II or, more likely, two arbitrators could not reasonably find that Braun had ingested synthetic testosterone. And that conclusion could only have arisen from their doubts about the integrity of the testing process. Not a scientist, just an humble drudge of a lawyer.

        • LarryAtIIATMS

          Chuck, I'm not a scientist, I'm also a lawyer. I'm trying to report on the science, but I don't claim to speak for the scientific community. Hell, half the people I've met in the BASEBALL community aren't speaking to me. I've just talked to a lot of scientists and read a lot of technical documents over the last 5 years.

          You're making a great legal argument for a different legal case. I understand that the prosecution may make a number of little mistakes in the gathering and handling of evidence, none of which individually mean very much, but all considered collectively causes a jury to doubt the truth of the prosecution's case. But that's not THIS case. In THIS case the prosecution did not handle the evidence in accordance with the prosecution's own substantive rules. These rules are not a matter of legal procedure — they're the scientific rules that baseball validated in order to ensure that their procedure produces reliable test results.

          There's a scientific principle to uphold here. Baseball validated procedure "A" for drug testing — it took the necessary scientific steps to ensure that procedure "A" produces valid test results. Then in the Braun case, baseball failed to follow procedure "A". Instead, it followed procedure "B". We don't know what kind of results procedure "B" produces, because procedure "B" has never been validated.

          Yeah, baseball is arguing that procedure "B" is nearly identical to procedure "A". But that's not a scientific argument. We're not playing horseshoes here; it's not good enough in science to do stuff that's nearly, or almost, what you're supposed to do. The assumption is that every step in a procedure is important to the result, or else why include it in the procedure in the first place? Sure, perhaps there's a step in the procedure that's not necessary to produce a valid result, but the only way to know that scientifically is to remove the step from the procedure and validate the revised procedure. (In case you're wondering, the opinion of Travis Tygart is neither necessary nor sufficient to validate a test procedure.)

          Yeah, folks are asking us to explain how procedure "B" might produce results different from procedure "A". It's best not to get drawn into that argument, because the premise of the argument is false. Scientific testing doesn't work by performing procedures that are "just as good" as validated procedures, or worse, by performing procedures that can't be conclusively proven to be worse than validated procedures. The tests are valid because the procedures have been validated. If the lab fails to follow a validated procedure, then in effect we're at a CSI scene, gathering evidence and trying to figure out what it means. This isn't what labs do. Labs follow validated procedures.

  16. Locke

    Very interesting article and discussion. But if I could, I'd like to take a step back for a minute and ask a simple question. Has there actually been any acknowledgement that there was a finding of synthetic testosterone? Other than rumor or speculation from a leak?

    • LarryAtIIATMS

      Thanks for the compliment on the very interesting article. Yours is a very interesting comment and a strategically sound step back. I HAVE been meaning to say that we have official confirmation of very little in this story so far. But the original leak, er, report, of the Braun story indicated that (1) Braun's urine sample had flunked the T/E test and was thus deemed to have tested positive, then (2) MLB decided to "perform a secondary test to determine whether the testosterone spike resulted from natural variations within Braun's body or from an artificial source" and that [t]he test indicated the testosterone was exogenous, meaning it came from outside his body." See http://es.pn/uZ7mOT.

      It's a reasonable guess that this "secondary" test was a carbon isotope ratio test. I don't know of any other test that purports to look at the nature of testosterone (really, testosterone metabolites) in a urine sample and, apart from the absolute or relative quantity of testosterone found, pronounce that the testosterone by nature is exogenous.

      The problem is, there are details in the report that sound odd to me. For example, the report makes it sound like the carbon isotope ratio test was performed out of an abundance of caution, in a special effort to bend over backwards to be fair to Braun. But in the real world this isn't how the testers do their job. The typical procedure is to use the T/E test as a screening test to see if anything might be odd in a given sample, and to do the carbon isotope ratio test to determine if the athlete doped with testosterone. In other words, it's the carbon isotope ratio that provides the proof. It's a really scary thought that in 2012 MLB thinks it can sanction an athlete based solely on the T/E test. The T/E test is known to throw off way, WAY too many false positives.

      BTW, so long as we're on the subject, a lot of the early testing of baseball players was probably based solely on the T/E test, since the CIR test did not come into common use in doping testing outside of baseball until 2004. Yes, that means exactly what you think it means.

      • Locke

        I've seen a lot of people argue that another "fact" stacked against Braun is that the other two examples taken at the same time did not test positive. But once you understand the process better, that conclusion cannot be logically drawn.

        Because the testing is secret (well it's supposed to be), we don't know conclusively that the other two tests were negative. Successful appeals are supposed to be secret as well, so as to not unfairly sully the reputation of the cleared player. Admittedly, this isn't likely, but we don't know.

        But more importantly, we cannot conclude that the other two samples passed a CIR test because we have no way of knowing if they actually had them performed. Having read your comments on the Saugy 2006 Study, we know that some individuals who have taken testosterone supplements, might still have a relatively normal T/E ratio. Heck, some might even have unremarkable CIR results.

        So if something was amiss with all of the samples taken at that time (for whatever reason) we can't conclude that the other two weren't effected as well because we don't know if they had a CIR done.

        • LarryAtIIATMS

          Locke, agreed. No one says that Braun's urine was handled in a way that would cause ANYONE's urine to test positive. The debate is around whether Braun's urine was handled in a way that caused HIS urine to test positive. Urine is diverse stuff.

          We can guess with respect to the other two October 1 samples that either (1) they passed the T/E test, or (2) they flunked the T/E test but passed the CIR test. That would mean that if all three samples would have tested negative if they'd been tested without delay, then there was something unique about Braun's sample. But we're guessing, and I don't draw any firm conclusions from these guesses.

  17. Locke

    Larry – if you wouldn't mind, based on what you know about all of this, what is your opinion on the testosterone component of testing in general?

    Would you prefer to see the T/E eliminated and all samples use the CIR test as some have suggested?

    The more I've read, the more I think that due to the way testosterone is metabolized by the body, the vast variation of natural body chemistries, and just how difficult it is to differentiate between andro & exogenous testosterone, this aspect of testing just isn't ready "ready for prime time" to the degree required for us to trust the system. While false positives & convicting the innocent type errors are certainly the most troubling possibility, at the very least, given the success of beating the system that we know of from BALCO, it doesn't seem good enough at catching the cheaters to justify all of the other concerns.

  18. LarryAtIIATMS

    Locke, mmm.

    I have a concern about false positives in doping tests. But I want to be an honest reporter, so I have to say: not too many people share my concern. If you want to dive deeper into this problem, you should look for work by Donald Berry, who is a medical statistician. Berry might ask first, how many false positive tests would we be willing to live with? One a year? One every five years? One every ten years? Depending on our aversion for false positives, you'd have to perform test validation on potentially thousands of biodiverse and geodiverse subjects, rather than the dozens that are typical in test populations.

    But I think the mainstream argument, one made by smart and responsible voices, is that we shouldn't worry too much about false positives. If the tests were only 99.9% accurate against false positives, we still expect to see a few false positive tests every year (roughly 3,600 tests were run annually under baseball's old collective bargaining agreement, with more tests than this scheduled under the new deal). We're just not seeing anything that looks like that. These same mainstream voices will argue (with good supporting evidence) that the rules are designed to avoid false positives, that the anti-doping forces have built in controls and safeguards in an effort to prevent false positives. Some will argue (again with cause) that these safeguards result in a goodly amount of false negative results — that as a result of our desire to make sure that the innocent do not suffer, a number of the guilty can get away with doping.

    I understand your comment about not being "ready for prime time", but I think that the right metaphor is something else. Doping is growing harder to detect, not easier. It's comparatively easy to test for stuff like amphetamines, which are not produced naturally by the body. It's a lot more difficult to test for doping that's done with substances the body produces naturally. Over time, doping comes to more closely resemble natural human biochemistry. Consider this: CIR testing for testosterone is based on the theory that exogenous testosterone is manufactured from stuff like soy, that is naturally light in carbon 13. But not all plants share this property. If testosterone can be manufactured from corn (and I'm not saying that it can, though I'm curious), the CIR test would be useless, because corn is not light in carbon 13.

    So, what do we do?

    One possible solution is to look to the "biological passport" that's been put into place in cycling. There's no way I can accurately describe how the biological passport works in a comment. But in essence, conventional testing is a one-time snapshot, while the biological passport is a motion picture. Conventional testing looks for drugs separately; the biological passport looks at numbers of biologically related measurements in combination. Conventional testing names the PED; the biological passport tells us that something is going on with the athlete's biomarkers that is highly unusual in the absence of some form of doping. Conventional testing looks for a drug; the biological passport looks for the EFFECT of the drug. And so on.

    At the moment, the biological passport focuses on blood biomarkers associated with cardiovascular capacity, such as red blood cell populations. We don't have a biological passport in place to detect use of anabolic steroids. There are other problems with the biological passport, including the need for frequent testing and the considerable complexity of trying to determine what is "normal" when looking at a combination of factors changing over time. Also, it's pretty clear that athletes CAN cheat the passport — the passport sets up limits on what is cheating, but athletes CAN cheat within those limits.

    I think this is what it comes down to. We can hope for an anti-doping system that will keep doping within limits, and perhaps deter the more risk-averse from doping at all. There WILL be a cost for all this: more testing, more invasive testing, more expensive testing. Whether the effort is worth requires a cost-benefit analysis, where the cost is uncertain and the benefits even more so. The bottom line may be, what kind of sport are you interested in watching?

    For me, I have no interest in baseball at the moment we assume that a particular ballplayer is probably doping, based on nothing more than what we know or think we know about the nature and extent of doping in baseball as a general matter. Your mileage may vary.

    • Locke

      Thanks for the response. I'll admit to being a Braun fan, and I know that to some degree, that clouds my objectivity. I know it's there – and try to separate it out but to what degree that's successful, I don't know. That said, while the Braun situation kicked it off, it's really intellectual curiosity that fueled a ton of reading on the topic – and again, a large thanks to you for contributing a great deal to my understanding of the topic.

      I'll take your word on the biological passport concept and save that for another day. But one thing that kept coming to mind while I was reading was that perhaps simply tracking individual results over time might be an improvement. With a baseline reading, the individual variation from one person to the next could be accounted for and at least to some degree, factored out. I'd think people smarter than me would have thought of that, but still have to ask the question.

      Finally – caught me on the "not ready for primetime" comment. The funny thing is, I struggled with the right words & took a few minutes to attempt to think of a better one but life (and a little laziness) kicked in so I just let it go.

  19. Is it possible that the Ryan arbitration case was ruled in his favor not because the sample was compromised (not preserved or refrigerated) or that the analysis was conducted improperly (tech did not follow protocol exactly) but that the chain-of-custody was questioned and ruled invalid?

Comments are closed.