Dr Sanjay Kaul

In a talk entitled "Noninferiority in ACUITY lies in the eyes of the beholder," Kaul suggested that the definition of noninferiority used in the trial was not strict enough and that the drug would not have met the criteria he considers more appropriate. But ACUITY investigators countered that the clinical benefit of bivalirudin shown in the trial was the most important issue.

The ACUITY trial, which was reported for the first time at the American College of Cardiology meeting in March 2006, showed that bivalirudin monotherapy was "noninferior" to heparin or enoxaparin plus a GP IIb/IIIa blocker in ACS patients heading to the cath lab. The main primary outcome was a composite of safety and efficacy, and the benefit of bivalirudin was driven entirely by the safety component, with the drug showing a slight increase in ischemic events but a large reduction in bleeding compared with heparin/enoxaparin plus a GP IIb/IIIa blocker.

Kaul made several points as to why he was uncomfortable with the statistics behind the trial.

First, he suggested that ACUITY should not have had a noninferiority design at all. "The ACUITY trial pitted bivalirudin against an active control (heparin or enoxaparin plus a GP IIb/IIIa blocker) that had not been previously proven reliably and consistently to be superior to heparin in terms of the primary efficacy end point—a composite of death/MI/urgent revascularization. Because of this, strictly speaking, the ACUITY investigators should not have done a noninferiority study. They should really be looking for superiority," he explained to heartwire.

Strictly speaking, the ACUITY investigators should not have done a noninferiority study. They should really be looking for superiority.

Second, he said that noninferiority designs should not be used for trials that have a composite end point including both safety and efficacy measures. "There is a fundamental problem of combining safety and efficacy as the main outcome in a noninferiority study. The unconventional use of a composite efficacy and safety outcome biased the assessment of noninferiority in favor of bivalirudin. Typically, the noninferiority claim should be confined to efficacy alone. Although combining efficacy and safety into one composite outcome might be desirable to inflate the event rate and enhance trial feasibility, it can often be misleading because drugs that are ineffective but safe can be made to appear better than effective drugs."

And third, Kaul believes the noninferiority margin used in the study was too wide. "The ACUITY investigators decided on a wide noninferiority margin (25%) on the basis of clinical judgment among the trial committee members, given the advantages of the drug—it is easier to use and cheaper and safer than heparin plus a GP IIb/IIIa blocker. Statistical reasoning for this 25% margin is not apparent. This 25% margin means that they were willing to accept a 25% worsening in outcome, because of the other benefits. In my view, the 25% margin for noninferiority was too wide. Recent major noninferiority trials in ACS have used much smaller margins than this—for example, SYNERGY utilized a margin of 10%, and the A to Z trial used a margin of 11%."

Kaul noted that the previous bivalirudin trial, REPLACE-2, used a higher margin than 25% and also had a composite end point that was driven by safety. "Because of these design flaws, the FDA did not accept that noninferiority had been met in REPLACE-2," he noted. "The methodological deficiencies in ACUITY may have similar implications for regulatory acceptability.

"The noninferiority margin in ACUITY is better than the one in REPLACE-2, but my argument is that this should not have been a noninferiority trial in the first place, and, if they insisted on doing an noninferiority trial, they should have had much tighter margins to minimize the chances of falsely concluding noninferiority (a type 1 or a false-positive error)." Kaul suggested that 15% would have been more appropriate as a noninferiority margin and presented analyses showing that the results would not have met these criteria.

A p value is no replacement for a brain. You need to think about the clinical benefit of the drug rather than just statistics.

During the discussion period, ACUITY chief investigator Dr Gregg Stone (Columbia University, New York, NY) said his head was spinning from Kaul's presentation. Kaul commented to heartwire later: "Actually, my talk was intended to 'unspin' the ACUITY spin."

Stone was quick to defend his trial. "A p value is no replacement for a brain. You need to think about the clinical benefit of the drug rather than just statistics. Bivalirudin is simpler to use and is cheaper than a IIb/IIIa blocker, and it reduces bleeding dramatically. You can argue whether a small and nonsignificant increase in ischemia is relevant, but overall the safety advantage clearly tips the balance in favor of bivalirudin," he told heartwire.

Another member of the ACUITY steering committee, Dr Harvey White (Auckland City Hospital, New Zealand), voiced similar views. "The steering committee decided that 25% was an appropriate margin, and the trial fulfilled that margin. You have to look at what the trial showed from a clinical point of view—a 0.5% increase in ischemic events vs a 3% reduction in major bleeding that was much more prognostically important."

But cochair of the session, Dr Christoph Bode (University of Freiberg, Germany), was supportive of Kaul. "He gave a provocative talk. We need to be more aware of what the statistics mean and where the boundaries are. It's good that he raised awareness of this," he commented to heartwire.