How important is experience when it comes to Multisystemic Therapy?
Everyone agrees that experience matters. The more time and practice you put into something, the better your performance. But, what if it there was a situation where it was the experience of your colleagues and organization, rather than your own, that mattered more to your future performance? This was the unexpected finding in a recently published study out of Sweden, “The Swedish Implementation of Multisystemic Therapy for Adolescents: Does Treatment Experience Predict Treatment Adherence?”
Before going into what the study said, there are several important things to understand.
- MST is an evidence-based program, and hence, fidelity to the model is critical for achieving the results found in efficacy and effectiveness studies. One key test of fidelity is the Therapist Adherence Measure (TAM). Research has shown repeatedly that TAMs predict whether a youth will be living at home, in school or working and not committing new offenses.
- In the Swedish study, TAM scores did, indeed, predict better outcomes.
- Experience in this study was defined as the number of families with which a therapist worked.
The study used data that was collected as part of a larger effectiveness study on MST in Sweden. In “Does Treatment Experience Predict Treatment Adherence?” the writers were hoping to discover what mattered when it came to therapist fidelity to the model and hence, better outcomes for families.
In 2003, seven Swedish teams implemented MST at about the same time. Call this group Wave1. About three years later, a new group of three teams, Wave2, started in different cities than the first.
Now let’s imagine three different groups of therapists.
- The Trailblazers all started working in Wave1 teams at inception.
- The Replacements also worked with teams in Wave1, but came in two years later.
- Finally, the Rookies arrived with the Wave2 teams.
As a brief thinking exercise, try to guess which of the three groups had the best adherence to MST and by extension, the best clinical outcome after reaching the same level of experience (i.e., treated an equal number of families). Would it be the Trailblazers, the first group of therapists to use MST in Sweden? The Replacements who came next? Or maybe the Rookies who benefited only from being in a country that had more experience with MST than the other groups?
In fact, the results showed that it was the Rookies who achieved the highest adherence scores after the same amount of experience, followed by the Replacements and finally, the Trailblazers.
Does that seem odd? On the one hand, it would certainly seem to make sense that the Replacements might do better after the same amount of experience as the Trailblazers since they could benefit from working alongside professionals who already understood how to implement the model. But, on the other hand . . . what to make of the Rookies? They had never done MST before, and they came into a community and agency that had never done MST before, either. Yet they achieved more with the same amount of experience as either of the other groups.
If you think that is odd, then here is something even odder.
After a certain level of experience, the adherence scores in each of the groups did not change in any significant way. Thus at some point, adherence reached a level as high as it would ever get, but those levels were different for each group. Once again, the Rookies outperformed the Trailblazers and Replacements. So, not only did the Rookies get higher levels of adherence after the same degree of experience, they maintained higher levels of adherence no matter how much experience was gained by the other groups.
How can that be?
It’s a logical question, but unfortunately not one that could be answered clearly by this particular study. It does, however, suggest that the longer the presence of MST in the organization and even in the country itself, the better MST is implemented. When you think about it, this doesn’t seem very odd at all and is quite reassuring. MST operates as a kind of feedback loop on itself and relies heavily on the collective experience over time of people and the organizations tasked with doing the actual work with families. MST is not just about therapists performing a set of tasks well. It’s about a collective process that involves the interactions between multiple systems. A process that involves not only initial startup support and staff training, but an ongoing relationship between the developers of the model, those assigned to disseminating it and those who are attempting to implement the model in a specific location.
