Do younger pupils really ‘mimic the habits of obese children in older classes’? Answer - probably not!
A tweat posted by @GuardianEdu yesterday (2 January 2010) attracted my interest:
‘Younger pupils mimic habits of obese children in older classes http://bit.ly/7KUEez’
The link takes you to the full news article published today in The Observer (3 January 2010) that is based on research conducted by a team from Ontario, Canada. Here’s some of the key extracts from the Observer article:
‘Children at schools where older students are obese or otherwise overweight are significantly more likely to suffer weight problems themselves, researchers report. For each one per cent increase in the prevalence of obese students aged 16 to 18 years, the odds of a student at 14 to 16 years old attending that school also being overweight increased significantly.
"It was the one risk factor that held true across every school we looked at," said Dr Scott Leatherdale, the chair of research at Cancer Care Ontario and lead investigator with the School Health Action, Planning and Evaluation System. "Schools that had a large number of obese younger students were disproportionately likely also to have a high percentage of overweight older students. The association was completely consistent."
"It could be that younger students look up to older students, and so emulate their sedentary behaviour and bad eating habits and do not judge the older children's body shape," he said. "Or it could be that the school doesn't encourage enough physical activity among its students, and the older students' weight issues are an indication of that.”’
The research from which these findings are taken is due to be published in the Journal of Youth Adolesence. Fortunately, the journal article itself is available to download as an ‘online early’ article at:
Leatherdale, S. T. and Papadakis, S. ‘A multi-level examination of the association between older social models in the school environment and overweight and obesity among younger students’, Journal of Youth Adolesence, DOI 10.1007/s10964-009-9491-z
In this instance, the news article does seem to broadly reflect the findings as reported in the journal article. As the authors conclude in the article:
‘[T]he junior students in our sample were more likely to be overweight or obsese if they attended a school with a high prevalence of senior students who are obese. … For each 1% increase in the prevalence of obese senior students at a school, the odds of a junior student at that school being obese increased (OR 1.19, 95% CI 1.16-1.24, p < .001). … This finding is consistent with previous empirical research which suggests that characteristics of the school a student attends can have important impact on their weight status.’
Having decided that the prevalence of overweight and obese senior students represents a risk factor in terms of junior pupils being overweight or obese, the authors conclude by arguing for the need to use indicators like this to develop more targeted school-based intervention programmes.
The statistical analysis upon which this article is based need not overly concern us here. For those interested, the authors used multi-level binary logistic regression to consider what factors (pupil-level and school-level) were associated with the chances of a pupil being overweight and also obese. The analysis itself, involving a sample of 12,049 grades 9-12 pupils from 76 Ontario secondary schools, seems to be technically correct. However, and as is often the case, it is the interpretation of the results of this analysis that are problematic. In this case, you don’t need to be a statistician to be able to identify some of the problems relating to the interpretation of the results from this study. Here’s some of the key ones:
First, the authors place a lot of emphasis on the importance of the school environment in providing an environment that influences levels of obesity in pupils. And yet, when reading the “fine print” of the study, the authors report that schools could only account for 1.8% of the variation in levels of obesity between pupils. In other words, school-level factors themselves would seem to have only a marginal influence on whether children are obese or not (with between 98.2% of the variation between children being associated with non-school based factors). The only reason why this finding is statistically significant is because of the huge sample size (n=12,049) where any difference, however minor, is likely to be statistically significant. This is therefore a good example of where findings may be statistically significant but not practically significant.
Second, the authors claim that with every one percentage point increase in the proportion of senior students who are obese, this increases the odds of junior pupils being obese by a factor of 1.19 (i.e. increasing their chances by 19%). This seems to be quite a notable relationship. However, quoting odds ratios like this can be misleading, especially when you’re dealing with only a small absolute number of children. In this case, the proportions of pupils in each school who were found to be obese were small, with only 6.5% of children categorised as obese on average within each school. In this instance, what therefore seems to be large changes in the odds of children becoming obese will only reflect very small changes in the actual numbers of pupils (in this case, literally, a handful of pupils).
Third, and even putting the above points to one side, there is a more fundamental problem relating to inferring a causal relationship from a correlation. While it may be true that the proportions of junior pupils in a school who are obese is correlated with the proportions of senior pupils who are obese, we cannot assume that there is any direct (or, indeed, indirect) causal relationship between the two at all. Unfortunately, this does not stop the authors from hypothesising wildly about how junior pupils may be emulating their senior counterparts. This is just irresponsible and misleading. Moreover, and in this case, it is what also generated the headline in The Observer news article – a headline with absolutely no evidence to substantiate it at all.
Fourth, there is actually a much simpler – and far more plausible – explanation for this correlation that the authors remarkably do not seem to recognize. It is not surprising that the proportions of junior and senior pupils tend to be similar within individual schools given that they tend to come from the same neighbourhoods or catchment areas. In this sense, any between-school variations found are likely to be due to variations in the socio-demographic nature of the catchment areas of the schools rather than anything that the schools themselves are doing.
Fifth, and finally, the authors do not make any distinction between different types of school-level factors; in particularly, those that are related to the school itself (the school ethos, the curriculum taught etc.) and those that are actually related to the wider neighbourhood in which a school is located and thus are not influenced by the school at all (i.e. socio-demographic characteristics – including levels of obesity – that may vary from school to school but that are not caused by the schools as such but simply reflect their different catchment areas).
This is an important distinction to make. In this case, the absence of such a distinction has lead the authors to making misleading claims regarding the existing and direct influence of the school environment on levels of obesity. This is not to say that schools cannot play an important role in helping to promote healthier lifestyles and reduce levels of childhood obesity but this is a different issue. The authors are focusing here on attempting to identify the existing influences that schools have.
So, here is another example of where the basic point that should have been drilled into any undergraduate student taking a research methods course – that correlation does not necessarily imply causality – seems to have been lost. In this present case, we cannot simply blame the sensationalist and misleading reporting of a journalist. As we saw, the news article broadly reflected the findings from the journal article itself. It may be, therefore, that part of the problem here is that as the research is based on a rather advanced and complex form of statistical modeling then people are going to be less likely to feel qualified in assessing the validity of the claims being made and thus more likely just to take the key findings ‘on trust’. However, and as demonstrated above, this is a dangerous approach to take and can easily lead to important policy decisions being made on the basis of false evidence.
Ultimately, this is why the peer-review process is so important in, hopefully, helping to identify and remove some of the more blatant cases of poor quality research. With this in mind, the key question we are left with is how did this article, with the claims it is making, get through the peer-review process of the Journal of Youth Adolescence seemingly unimpeded? Did none of the reviewers (or the editors) recognize and/or raise any of the concerns highlighted above?
