The informatics community has become quite engaged with a discussion surrounding a publication in the Archives of Internal Medicine addressing the impact of EHRs on measurable clinical quality.

Th article, is entitled "Electronic Health Records and Clinical Decision Support Systems: Impact on National Ambulatory Care Quality" and was written by Max J. Romano and Randall S. Stafford. Using the 2005-2007 National Ambulatory Medical Care Survey and National Hospital Ambulatory Medical Care Survey and 20 quality indicators, their study, performed at Stanford, found:

Electronic health records were used in 30% of an estimated 1.1 billion annual US patient visits. Clinical decision support was present in 57% of these EHR visits (17% of all visits). The use of EHRs and CDS was more likely in the West and in multiphysician settings than in solo practices. In only 1 of 20 indicators was quality greater in EHR visits than in non-EHR visits (diet counseling in high-risk adults.....). Among the EHR visits, only 1 of 20 quality indicators showed significantly better performance in visits with CDS compared with EHR visits without CDS (lack of routine electrocardiographic ordering in low-risk patients.....). There were no other significant quality differences.

They concluded:

Our findings indicate no consistent association between EHRs and CDS and better quality. These results raise concerns about the ability of health information technology to fundamentally alter outpatient care quality.

Source: http://archinte.ama-assn.org/cgi/content/full/archinternmed.2010.527

In an insightful accompanying piece, Drs. Clement McDonald and Swapna Abhyankar of the Lister Hill Center at the National Library of Medicine point out many of the reasons for the disconnect between the Annals study and the work that has gone before. They conclude:

Regardless of the differences, we know from multiple randomized controlled trials that well-implemented CDS systems can produce large and important improvements in care processes. What we do not know is whether we can extend these results to a national level. The results of Romano and Stafford's study suggest not. However, we suspect that the EHR and CDS systems in use at the time of their study were immature, did not cover many of the guidelines that the study targeted, and had incomplete patient data; a 2005 survey of Massachusetts physicians supports this concern.

---

This writer suggests one read the primary study only after reading McDonald and Abhyankar's commentary. Perhaps like this writer, the reader will develop clearer view concerning the hazards of drawing conclusions with the data we have today in a way that is accurate, timely and insightful. More often than not, one fails to make the case early in the game.

Imagine this hypothetical early 20th century article:

"Randomized trial shows that horse-drawn carriages are superior in performance to Henry Ford's new horseless carriage."

No surprise! It takes a long time for even good ideas to become the conventional…and an even longer time to prove the impact of change.

Economists, not surprisingly, have often tried to study the impact of computers and information technology on productivity and social good.

Often quoted in this respect is the MIT economist Robert Solow who in 1987 said:

"You can see the computer age everywhere but in the productivity statistics. "

Context for his quote

Reviewing Cohen and Zysman's "Manufacturing Matters: the Myth of the Post-Industrial Economy" in the July 12, 1987 issue of the New York Times book review, Solow writes of the book:

"The authors state: ' We do not need to show that the new technologies produce a break with past patters of productivity growth....[that] would depend not just on what the technologies represent, but rather on how effectively they are used.'"

In his review, Solow added:

"What this means is that they [Cohen and Zysman], like everyone else, are somewhat embarrassed by the fact that what everyone feels to have been a technological revolution, a drastic change in our productive lives, has been actually accompanied everywhere, including Japan, by a slowing-down of productivity growth, not by a step up. You can see the computer age everywhere but in the productivity statistics. "

source:

Robert Solow: "We'd Better Watch Out."

NY Times Book Review, July 12, 1987, p. 36

My own experience suggests that often people pursue a collective course based on a collective belief grounded in intuition, preliminary findings, and simple common sense. The people I've known with such courage do so knowing that – at the very best – formal validation of large-scale social impact will be realized only years after a great effort has begun. Along the way, one tries to think things through, to stay focused on the important things, and to make incremental progress on the really important aspects and not merely the interesting ones.

But all too often, solid evidence for even the greatest of efforts is to be found only when looking through the rear view mirror.