Clinical Trial Cherry Picking

I have designed lots of trials and anybody designing a trial designs the inclusion / exclusion criteria in such a way as to give the test drug the best possible showing. Similarly, trials are designed with the minimum number of subjects necessary to show the desired result for the very simple reason that the sponsor must test and follow-up all those enrolled in a trial and the cost of doing so for a simple drug is $20K and if there is a lot of imaging involved (like in this trial) that number increases quite rapidly. I have seen several trials with a per patient cost exceeding $100K.

 

So are trials slanted to some degree?  Yes, nearly all of them are. When you see a drug that BARELY manages a p<0.05 even after the deck is stacked it is reasonable to question just how well the drug works. Conversely, no matter how much the criteria are manipulated a drug with a p <0.01 is highly effective and, depending on side-effect profile, it will be a market success.

 

What you can’t do is compare apples and oranges. SOC when this trial started was one thing, SOC when the last patients were treated was something else. All patients got multiple drugs, and all patients had tumor debulking surgery. It matters where a patient is treated; somebody undergoing surgery at UCLA might well have a better outcome that somebody who got their surgery elsewhere even though the subsequent drug regimen was identical. Duration of life for GBM is getting longer, with or without DCVax, which makes back of the envelope estimates of how well a drug is contributing to a mixed result meaningless. There are well-established statistical methods to sort out these issues, and only when those results are made public can you make any informed statements, and uninformed statements and decision tend to be financially expensive. Short cut estimates are fun to discuss, but no regulator is going to make a decision on anything other than a full statistical package.

 

Part of a trial approval is the statistics plan, which should be predefined. Most companies have all the statistical programming done before the trial even starts, which reduces the job of analysis to a few seconds of computer time once the data is locked. The excuses for not releasing top line data ran thin months ago, and I am not sure what term comes after “thin”. I can think of one, but my mother told me never to say that word.