PAULCONNOLLY.NET

The DCSF (Department for Children, Schools and Families) in England released a new report yesterday providing more detailed analyses of GCSE attainment in England by pupil characteristics (for the report see:  http://www.dcsf.gov.uk/rsgateway/DB/SFR/s000900/index.shtml). I haven’t had time to read it yet but three points emerge from news coverage of this report that are already worth commenting on.

The first is political spin. The figures seem to be good or bad depending on who you listen to. For the government, the figures are encouraging. A tweat from the @DCSF yesterday declared: “New GCSE figures: attainment gap between those on free school meals and their peers has narrowed”.

However, positive stories are never as newsworthy as negative ones and so here’s how @bbceducation reported the findings in their own tweat that came out within a few hours of the one from the DCSF: “Poor white teenage boys in England have slipped further behind other youngsters in their GCSE results”

So, here’s the first lesson – never trust any press release or news story to give you the full picture! You will always need to look at the evidence for yourself. However, two further lessons emerge when you do begin looking at the evidence. Infact we need look no further than the statistics reported in the press releases/news items accompanying these tweats to begin to find these lessons.

Both tweats imply notable changes in performance and yet the actual changes reported (and remember the statistics in the press releases/news items are the ones that have been cherry-picked to back up their respective positions) are marginal. Take the finding on free school meals (FSM) for example. On the DCSF’s website, they report that the proportion of those pupils eligible for FSM gaining the expected level (five good passes at GCSE) rose by 3.4 percentage points over the last year. As they go onto claim, this is: “a faster improvement than the 3.1 percentage point rise for non-FSM pupils”.

BBC Education did little better in relation to their news story – this time choosing to emphasise the negative results. The gap between poor white boys (those in eligible for FSM) and other white boys (those not eligible for FSM) widened from 29.8 percentage points to 31.6 percentage points. A whopping 1.8 percentage point increase!

So here are the two further lessons from this cursory review of tweats and press releases/news stories. The first is the need for all those involved to be much clearer in their headline reporting of the actual size of any effects found. Most people won’t even go beyond the headlines and will thus simply be left with the impression that either things are getting better (the DCSF line) or worse (the BBC line) for young people from disadvantaged backgrounds. And yet in both cases, the change is marginal. Unfortunately, marginal changes are no good to politicians or the media.

The second lesson is the need to step back and look at trends over time. We can expect minor fluctuations in statistics year-by-year, simply due to random variation in the make-up of any particular cohort of school pupils. Without further information we have no way of knowing whether these (very minor) changes reported do actually represent an underlying trend or are simply random fluctuations.

So, the next thing I need to do – and what I’d advise everyone else to do as well – is to read the full report for myself; only then can we develop a more balanced view of what is going on and determine whether some or all of these findings are actually indicative of real trends at all or may just reflect random fluctuation.

Posted by Paul Connolly at 11:25 AM | Permalink | Comments (0) | TrackBacks (0)

The next issue of Effective Education (Volume 1, Issue 2) is now in press and due for publication before the end of this month (December 2009). For more details see: http://www.informaworld.com/effectiveeducation

EFFECTIVE EDUCATION

Issue 2, Volume 1, 2009

Unobserved but not unimportant: The effects of unmeasured variables on causal attributions, Robert Coe (University of Durham, UK)

The Effectiveness of the Success for All Reading Programme on Primary EAL pupils in Hong Kong, Alan Cheung (John Hopkins University, USA)

How First Year Students perceive the Fit between Secondary and University Education: the Effect of  Teaching Approaches, M. Torenbeek, E.P.W.A. Jansen & W.H.A. Hofman (University of Groningen, The Netherlands)

The ‘Re-imagining’ of Evidence under New Labour: policy and practice in education in uncertain times, Robert Hulme (University of Chester, UK) and Moira Hulme (University of Glasgow, UK)

Quantitative measures of respect and social inclusion in children: Overview and recommendations, Colin G. Tredoux (University of Cape Town, South Africa), Noraini M. Noor (International Islamic University of Malaysia, Malaysia) and Lisa de Paulo (University of Cape Town, South Africa)

Posted by Paul Connolly at 05:16 PM | Permalink | Comments (0) | TrackBacks (0)

A news item appeared on the BBC website this week with the headline: ‘Children who use technology are “better writers”‘ (see:  http://news.bbc.co.uk/1/hi/technology/8392653.stm). The claim is based on a survey of 3,001 children aged 9-16 commissioned by the National Literacy Trust that explored their use of new communication technologies such as: blogs, texting and social network sites. From the news item it appears that the main finding is that: ‘of the children who neither blogged nor used social network sites, 47% rated their writing as "good" or "very good", while 61% of the bloggers and 56% of the social networkers said the same.’

In response to these findings, Jonathan Douglas, Director of the National Literacy Trust, was quoted as saying: ‘Our research suggests a strong correlation between kids using technology and wider patterns of reading and writing. […] Engagement with online technology drives their enthusiasm for writing short stories, letters, song lyrics or diaries.’ Moreover, and in response to the claim that the use of blogging, texting and social networking sites damages literacy, he went onto state that: ‘Our research results are conclusive - the more forms of communications children use the stronger their core literary skills.’

Now, I’ve not had time to read the full report as yet but there are at least two problems with the conclusions being drawn above:

1. From the news item, it appears that the survey did not actually measure children’s literacy skills. Rather it focused simply on their own self-perceptions of how good their writing is. At the very best, therefore, all that can be claimed here is that the more that children use such communication technologies, the more that they are likely to have positive self-perceptions of their literacy skills.

2. While there may well be a ‘strong correlation’ between these two things, it is impossible without further evidence to make any claims about what may be causing what. It is certainly premature for the Director of the National Literacy Trust to conclude that they have ‘conclusive results’ showing that it is children’s use of such technologies that increases their core literacy skills. While this may be the case, there is also an equally plausible explanation: that children with greater literacy skills (or, in their case, a greater perception of their literacy skills) are more likely to then use literacy-based technologies such as blogging, texting and social network sites more. Moreover, there may not be any direct relationship between the two at all. It may be, for example, that there is some other factor - for example a child’s socio-economic background - that has an influence on both literacy skills and use of technology. Thus, the more affluent a child’s background, the more likely they are to have higher literacy skills and also to have greater access to, and thus make greater use of, such technologies. It is not inconceivable, therefore, that literacy skills and technology use are completely unrelated.

This second point – that correlation does not equal causality – is a fundamental one that students should have learnt from any basic research methodology course. The fact that a well-respected organization such as the National Literacy Trust can be confusing the two in their own research findings is a poor reflection on the state of educational research in the UK. Moreover, the fact that the BBC’s own ‘technology reporter’ can simply report such claims uncritically, as in this news item, does nothing to help improve the situation.

Posted by Paul Connolly at 05:01 PM | Permalink | Comments (0) | TrackBacks (0)