Our view of the JAMA article

We’ve gotten a few quizzical emails about the JAMA article showing that a communication skills clinic elective didn’t improve how patients rated quality of communication. What do we make of it?

Don't forget...

The residents and nurse practitioners acquired new skills. For those who are skeptical that physicians change, this is evidence to the contrary. Lectures don’t work; carefully facilitated small groups can enable learners to acquire skills.

This skill acquisition happened with facilitators who were trained for this project. They didn’t have much small group experience. Our facilitation model doesn’t require years of experience.

Then why didn’t it work? Some hypotheses:

Our measurement tools don’t capture the full patient experience. We used the Quality of Communication and Quality of Dying measures, the best of what’s out there. But there may be a basic conceptual flaw – we are asking patients to say that they are “highly satisfied” with a conversation that may be the most disruptive and difficult they’ve had in their life, even when done well from the clinicians’ perspective. We don’t have direct empirical data on this, but this hypothesis might also explain the recent NEJM study by Weeks et al showing a negative correlation between communication (as measured by 5 generic questions) and accuracy of prognostic understanding. Perhaps we need to be asking patients what the conversation covered rather than whether the doctor did a ‘good’ job.

Measurement needs to occur sooner after conversations are held. The number of questionnaires returned by patients indicating that they couldn’t rate the clinician because they had not had a significant conversation, or couldn’t remember the clinician well enough was sobering. Add that many patients were receiving questionnaires months after they had met the clinician—and we wonder how well the patients could really remember what happened.

Intervention needs to occur when the clinicians really need it. The clinic elective was designed to fit in to the residents’ schedule rather than designed to target times when these conversations were most likely to happen. We used a clinic block slot because it was feasible. But in retrospect, figuring out how to embed the intervention in a clinical experience where these conversations happen a lot would have made the intervention more powerful as ‘just-in-time’ learning.

We should build on individual baseline skill levels. The baseline self-assessment from the study showed that resident assessments were all over the place—from quite accurate, to massive self-overrating, to massive self-underrating. We ought to feed that back and start the learners there.

Findings that made us think...

Discussing serious stuff might make people sad and upset. This seems almost ridiculous to state, but I think we’ve been hoping that great communication skills mean that we can skip the sad step—that might not be true. And maybe the study intervention gave learners the ability to start the conversation—but not enough skill to deal with the sadness?

Sick people might view these conversations differently. Note the subset analysis that showed that the patients who rated their own health poorly rated trained clinicians as significantly better. It’s a subset—but we wonder if experiencing the reality of one’s own illness prepares you in some way.

Are randomized trials really the best way to test an educational intervention? In this study, we treated the communication skills intervention like a drug—give it to all the residents, regardless of their skill level, and see if it changes them (which it did), and if that effect trickles down to the patients (which it did not using these instruments and these methods). But as an educator, I know that learning has to be tailored—and we didn’t do that. Maybe we need to target interventions to performance level? 

What's next?

Seth Godin defines failure as ‘a project that doesn't work, an initiative that teaches you something’, and (borrowing from Nietzsche) declares that ‘failures that don't kill us make us bolder’. We’re with Seth—and our north star is still culture change. The way clinicians talk to patients still isn’t good enough, and it’s still central. We’re sticking to a strategy of innovation and empirical testing, and we’ll seek collaborators who can think big and deal with reality. 

Our next steps are courses for learners at different levels, expanding learning into the peer social space, and using technology to extend feedback and community, and looking at how communication is distributed across teams. If you’re interested—let’s keep talking.

(this post is under construction--hyperlinks coming!)