Chicago, IL - The selection of abstracts for presentation at meetings may be tilted toward certain kinds of studies, but blinding the review process lessens any involved biases, suggests an analysis of acceptances at recent American Heart Association (AHA) scientific sessions [1].

"Adoption of blinded peer review by scientific research meetings is a reasonable, low-cost intervention with substantial benefit," concluded Dr Joseph S Ross (Yale University, New Haven, CT) and associates in the April 12, 2006 Journal of the American Medical Association. A number of the coauthors have held leadership roles in the AHA.

In the group's analysis, abstract reviewers—whether or not they were blinded to each study's submitting author and institution—more often chose submissions from the US, non-US English-speaking countries, prestigious institutions, and US government agencies—the latter as compared with private industry. There was no evidence for bias based on sex.

The newly adopted practice of blinded review for the last three of the five annual AHA sessions from 2000 to 2004 in the analysis, compared with the preceding two open-review years, "attenuated but did not eliminate" the apparent preferences.

There certainly is the capacity in a system like this to give extra credit to groups based on who they are rather than what they've written.

"The pure fact is that blinding led to a difference in acceptance rates by some of these characteristics that really shouldn't be judged," coauthor Dr Harlan M Krumholz (Yale University) told heartwire. Even though the reviewers were almost certainly being careful to judge only the science, that finding "pretty much shows that they were taking other factors into consideration," he said. "It should give us all pause. We don't think we have biases, but. . . ."

The analysis encompassed the more than 67 000 abstracts submitted over the five years, of which 28.5% were accepted. The US accounted for 85% of the reviewers; of the remainder, nearly 40% were from English-speaking countries. Comparison of patterns in the open- and blinded-review processes showed some consistent differences.

According to Ross and associates, there are likely valid reasons for some of the acceptance-rate differences. "Still, we cannot exclude the possibility of residual reviewer bias," they write. "Successful blinding may require more than removing the author's name and institution."

Deleting names and places from an abstract can still leave plenty of clues to its origin, observed Krumholz. "The fact that a higher percentage get admitted from the US may represent biases that you can't get rid of." On the other hand, "it may be that the quality of the science is better because there's more investment in this country than in some others." And the lean toward English-speaking countries "could represent that the abstracts are clearer and better written, which is a legitimate way to grade them."

But that may explain only part of the observed bias. "There certainly is the capacity in a system like this to give extra credit to groups based on who they are rather than what they've written," Krumholz said. "You could argue that, well, if I know that this is from a really well-regarded group, then I can make the leap of faith that it's good science. But that's not really how it's supposed to work. It's supposed to work by judging the quality of what's written on the page."