What Does That Lancet Study On Early Autism Intervention Really Show?


Autistic children whose parents participated in an early intervention program have a reduced likelihood of having intensified symptoms by age 10, says a new study published in The Lancet.

The authors say that theirs is the first long-term look at the outcomes of an intervention like this. But a close examination of their data raises questions about how distinctive the claim of improvement with the intervention program is.

In their study, the Lancet authors used a program called PACT (Preschool Autism Communication Trial), a one-year intervention that involves 12 therapy sessions and 20 to 30 minutes of regular practice work that parents do at home with their child. The intervention includes analysis of videos of parent-child interaction to identify areas where parents overlooked a child cue. Teasing out these oversights and correcting them, goes the hypothesis, will make parents more aware and improve how parent and child synchronize their communication.

Intuitively, that would make sense: Any parent spending focused, organized time with a child at home on a daily basis following a plan to enhance communication might see improvements. Given that a considerable portion of what makes autism autism is communication differences with neurotypical people, identifying ways that non-autistic people can bridge the gap would likely be helpful.

But … how helpful was this intervention, really?

When I first began to write this post, my intention was to say that something like this would work for any family. That what works for autistic children would probably just work for all children. That there’s not anything particularly special about seeing benefits when one becomes a “superparent,” as the BBC breathlessly phrased it in their fluffy report on the study.

But … then I looked at the findings. One visual from the paper is particularly revealing: Figure 2. It’s a forest plot showing the odds that either the intervention did or did not result in symptom improvement at 13 months from beginning the intervention or at an average of 5.75 years after that 13-month endpoint. As in the generic example below, the plot has a solid vertical line (1.0 below) where a data dot (a square, in this case) will sit if the treatment had no effect one way or the other. If a square is to the right of the midline, the treatment had a “benefit.” If it’s to the left, the treatment had a negative effect on the presumably desired outcome.

A generic example of what a forest plot looks like. The dots are an odds value, and the horizontal lines on either side are the range of other plausible values for the population. The line at 1.0 represents the "no effect" value.

A generic example of what a forest plot looks like. The squares are odds values, and the horizontal lines on either side are the range of other plausible values for the population. The line at 1.0 represents the “no effect” value.

To the right in the Lancet Figure 2 is a list of numbers. Each entry gives the value of the square and then a range known as the confidence interval. This interval represents the low to high values that could potentially be the real treatment effect in a population. So the forest plot visually offers a central or average value for the effect, but we have a range of other values that could also be plausible. Somewhere in that range lies the true population mean, and it’s not necessarily the value that’s represented with the eye-catching square on the forest plot.

What leaps out about this list of values is that the “no effect” number lies within many of these ranges, as the authors themselves note in their paper. So even for results where the square sits well over on the side of “big positive effect,” the range of possible values is quite large and includes the possibility of no effect whatsoever.

The confidence interval is also visualized on the tree itself as horizontal lines extending from either side of the square, as in the above generic image. Wherever this horizontal line crosses the “no effect” midline, “no effect” is one possible outcome at the population level.

For 13 of the 20 outcomes in Figure 2, the confidence intervals include the “no effect” possibility, even though 18 of the 20 have a square on the “positive effect” side of the midline. If all you see are the eye-catching squares, you’re going to overlook the subtle message of those intervals.

As an example, the authors found that at the long-term follow-up, results for a clinician measure of autistic symptom severity, the ADOS CSS, suggested a good-sized effect of treatment on a reduction in these scores. So the forest plot places the square for this finding at 0.70, far to the right of the “no effect” midline. Yet the confidence interval is -0.05 to 1.47. That means that the zero point–the “no effect” midline–lies within the range of possible valid outcomes with this measure. That 70% effect is a possibility, sure. But so is a negative 5% effect or a huge positive 147% effect. Indeed, that wide range of possible values is a signal that these results carry a lot of uncertainty, which is typical for a study population this small.

The authors of the paper report effects results for the 13-month point, when the children were preschoolers participating in the intervention, and at the final follow-up point, when the children were an average of 10.5 years old and ~6 years past the intervention. The researchers had a specific reason for doing this: Some of their measures that showed a significant effect at 13 months no longer did so at the longer-term follow-up, but presenting them across the entire time period retained the significance.

For example, as Table 2 of their paper shows, mutually registered communication between parent and child, or parent synchrony, seemed to have improved considerably in the PACT treatment group at 13 months from treatment initiation. But by the follow-up 6 years later, the PACT group and the no-PACT group had about the same amount of improvement … each group went from a 31% rate at baseline to 44.4 and 43.1%, respectively, at the final follow-up. PACT ultimately made no difference. Yet an improvement in parental synchrony is a “targeted” parental behavior in the PACT program.

But the paper authors reported that for the “overall time period” (from study initiation to follow-up), the trajectory for the PACT group was significant relative to the no-PACT group. What’s interesting is looking at their figure reporting how the “treatment-as-usual” group fared relative to the PACT group.

Although the figure shows significance for differences in the trajectory of PACT vs no-PACT outcomes, the relevance of that is unclear. If this were a cancer drug and the outcome were symptom improvement for cancer patients, then the trajectories would matter even if, in the end, mortality rates were the same. The people who’d had some symptom improvement might have had better quality of life for awhile, at least.

But the stated goal here is to see persistently improving outcomes in parent-child synchrony over the long term, as the teen years approach, not transient improvement at the midpoint that then disappears. These graphs actually suggest that in the end, both groups pretty much landed at the same place; indeed, the non-PACT group showed the better directly compared trajectory for parent synchrony.

What the data in the figures and tables really seem to suggest is that in the short-term, at the 13-month follow-up, there was improvement in some of the endpoints. But by the long term follow-up 6 years later, for most endpoints, “no effect” was one possible outcome for this population, and the PACT and no-PACT groups had considerable overlap.

If we focus on the forest plot results and look only at those with confidence intervals that don’t cross the midline and that represent the long-term outcomes only (rather than 13 month or “overall time period”), we find that PACT may have beneficially affected child communication initiations with the parents from baseline to follow-up. We also find a beneficial effect of PACT on three parent-reported outcomes that weren’t blinded–the parents completing the tools knew which group they were in, obviously.

Finally, it’s worth noting that the two groups differed in two key factors: Parents in the PACT group were likelier to be part of a two-parent household and have a higher education level versus parents in the no-PACT group.

Overall, children in the PACT group became more likely to initiate communication with their parents, and parents were more likely to see improvement in their children, as reported on various scales. I’d hazard that whether their child is autistic or not, well-educated, relatively well-off parents made aware of their gaps in communication with their preschoolers might learn to respond better to those preschoolers and think they see improvement in their child’s behavior over time.

Indeed, what these researchers seem to have uncovered is that if parents attend to what their child is trying to communicate to them, the entire relationship–including the parent’s perspective on the child–may improve.

It’s a straightforward message that autistic people have been trying to send for years, especially to counter the rigid, prescriptive and sometimes cruel applied behavioral analysis (ABA) approaches currently favored by many parents, clinicians, and insurance companies. With ABA, the burden is on the child to somehow understand and modify behavior in situations that they often aren’t even old enough to intellectualize. With PACT–as autistic people have long argued should be the case–the onus is on the parents to work harder to understand and communicate with their child, rather than to try to “fix” them.

Maybe, then, what I thought I might need to reconsider when writing about this study still stands. Working at parent-child communication–with the work starting very early on the parents’ part–isn’t something that is effective for or applies only to autistic children and their families. This study didn’t include groups of non-autistic children having versus not having this intervention, but if it had, I bet these minimal improvements in communication and parental perception would show up in these families, too.


Leave a Reply

Your email address will not be published. Required fields are marked *