Friday, May 11, 2012

Autism Misinformation: Putting My Foot Down (Part 1)

Jessica Severson

I am putting my foot down.

As the parent of an autistic child I hear a lot about vaccines and about half a million other things that people think cause autism.

I'm hyperaware of the attention autism gets in the media. So I know about the CDC's new stats on autism rates. I know about the debate on whether the increase in autism is due to more awareness and diagnosis or more actual occurrences. (Personally, I find the former to be a serious factor, though who's to say how much.) And I see all the articles that come out week after week about the millions of things that are linked to autism.

There's a recurring problem here. Valuable research is done. Research is disseminated. Information is reported. Articles are read. Findings are spread. And what starts in a lab ends up in a Facebook status. What starts as truth ends up as mistruth in something like a child's game of telephone. Along the way, piece by piece, truth fades away in favor of headlines and pageviews and gossip.

It's getting just plain stupid. I'm starting to suspect these articles have nothing to do with serious research but with a search for traffic and hype, an attempt to ride the wave of a trendy topic as concerned parents read every horror story they can find.

Let's consider, piece by piece, a case study in how real research becomes egregious misinformation.

Part One: Research

It starts with scientists. It starts with research. They write up their findings and publish them in a peer-reviewed scientific journal. In this case there are several papers published over a few years about chemicals and their link to brain development. They cover a wide variety of issues and present a wide variety of conclusions. All of them suggest further study.

Maybe they have bad methodology or use statistics incorrectly. Only a few people would ever know the difference. That's not my concern today. Bad science is one thing, but bad information on good science is another. So let's assume we have good, solid science in this research.

Part Two: The Conference

Scientists and researchers with similar interests get together and discuss their findings. It's not that difference from any other conference. There are panels and presentations.

Part Three: The Op-Ed

Next, a group that works on environmental hazards for children publishes a paper. Not a research study but an op-ed in a peer-reviewed journal. In this op-ed they review the conference from Part Two and encourage the study of environmental factors and their relationship to neurodevelopment disorders. Autism is one of many neuro-ish disorders and is mentioned by name in the piece and its title. It's unclear to me why they zero in on autism -- they have a couple vague pieces of evidence that are autism-specific, but the vast majority of what they're looking at has never been demonstrated to have any kind of relationship to autism, not even a correlation.

Problem #1 is the unnecessary autism name-checking. Problem #2 is much worse, it's the list of 10 chemicals they suggest for future study. The list itself isn't a bad idea, I guess. They're suggesting places for potential research, which certainly needs to be done. But it does reek a little bit of the kind of thing magazines do, you know what I mean, 10 Ways To Get Your Guy All Fired Up! and such. Still, it's their prerogative.

So let's examine their evidence for these suggestions. They cite at least one paper for each of these chemicals. I checked them all. The vast majority of them have never shown any connection to Autism (or even ADHD, another diagnosis they name-check). In fact, many of them show that with exposure to these chemicals, the outcome differentials between exposure and non-exposure is five IQ points.

FIVE IQ POINTS. Statistically significant? Perhaps. Practically important for a parent? No.

IQ itself is a strange and vague thing. And five points isn't going to move your super-genius down to the level of a typical person. They'd still be a super-genius. And adding five points to someone on the opposite end of the scale isn't going to make them typical, either. It's hard to imagine what difference you'd see between two people whose IQs are five points apart.

Such statistical differences may well be a sign to warrant further study. And they may be a sign that these chemicals affect neurological development. But it's getting a bit ahead of ourselves to say they are suspected of being tied to autism. Many of these papers are in areas of research that are just beginning. Many of them involve homogeneous groups (for example, all the participants are Mexican-American migrant workers) which makes issues of genetics and heredity very difficult to account for. Many involve parents self-reporting by filling out surveys rather than having the children examined by professionals.

Let's be fair. These are the very beginnings of research. You'll need to do all sorts of rigorous testing and consideration to make real connections. Of course more research is needed. And it's important that we keep that in mind as we move forward.

(Though, of course, no one else will.)

Part Four: The Press Release

The op-ed is about publicity so it's the beginning of the problem. But it gets worse. A press release comes out with the list of ten chemicals and already the twisting starts. These are chemicals suggested for further research, but suddenly they're a List of the Top Ten Toxic Chemicals Suspected to Cause Autism and Learning Disabilities. This, unsurprisingly, is the headline you'll see all over the internet when news organizations report on the press release. Already it's turned from suggestions for research into a watch list.

It gets worse. The press release has this second headline:
The editorial was published alongside four other papers — each suggesting a link between toxic chemicals and autism.
No, actually that's not at all accurate.

Let's start with the first paper, which examines the possibility of a connection between maternal smoking and autism. What's their conclusion?
The primary analyses indicated a slightly inverse association with all ASDs[.]
What does that mean? Among the autistic kids vs. non-autistic kids, there was actually less maternal smoking in the autism group. The paper does point out that when it comes to "subgroups," for instance ASD without intellectual disability or Asperger's, there may be a possibly positive relationship. But there are so many caveats I can't even get to them all. Let's just take this one:
The ASD subgroup variables were imperfect, relying on the child’s access to evaluation services and the documentation by a myriad of community providers, rather than direct clinical observation.
This means that when they're saying some groups of ASD kids may have this relationship, they didn't actually classify these kids. They never saw these kids. They're relying on data collected by other people. Not even by a consistent set of people. It comes from eleven different states and who knows how many providers. Who's to say how accurate any of it is. And who's to say whether these kids are correctly classified at their particular place on the spectrum.

So take all that with a whole jar full of salt and you're still looking at, overall, no connection with smoking. If anything, the data would indicate smoking has less autism rather than more.

After this, there are two papers on the same chemical. One of them does not contain the word "autism" anywhere. (One of its references has it, but nowhere does it appear in the text of their paper.) The second paper is better. It focuses on the chemical's effects in particular processes which have been linked to autism. This is very micro-scale science, there are no people involved, just cells and chemicals. It's important research, but there's a long stretch between cellular interactions and a person's diagnosis. It didn't involve any analysis with autistic individuals. This is certainly the most useful paper of the bunch by a long shot, but it still just sets the stage for further research.

The fourth paper is a review. That means it asserts no new information but summarizes the research on a particular issue, specifically pesticides and autism. Technically I suppose it does assert a link, but none of this is new information.

So I think we've pretty much destroyed the headline in that press release. There were not four articles suggesting a connection between chemicals and autism.

Is it likely that the writers who take this press release and write articles on it are going to read the papers it cites? Are they going to realize that what they're saying isn't actually true? They should. Of course they should. But they don't.

This list has chemicals suspected of being tied to neurological development. And we should just leave it at that. It's not that they shouldn't be studied. They should. But we shouldn't be throwing out buzzwords like ADHD and Autism when the research doesn't show any firm data.

[To be continued on Monday, when Jessica will discuss News Articles and Readers.]

This essay previously published at Double X Science.