This morning I awoke to a few people in my twitter stream retweeting a piece of research by the Society for New Communication Research about the increased reliance of journalists on social media. As someone who works every day with BBC journalists training them to do this, I was sceptical of the numbers, so did some digging.
The report was actually published in February this year, and you can read the executive summary here, and see a slideshare presentation here. The headline findings suggest social media is being taken very seriously in newsrooms.
48% are using Twitter and other microblogging sites
66% of journalists are using blogs
48% are using online video
25% are using podcasts
Overall, 90% of journalists agree that new media and communication tools and technologies are enhancing journalism to some extent.
For those of us who use social media everyday, and understand its value, it’s tempting to see these figures and quickly bookmark the document in delicious, happy they support the way we believe journalism should be headed.
The bottom line is that while the research is valuable as a qualitative study of some journalists’ attitudes (and the quotes listed in the slideshare offer some interesting insights into these), this research isn’t a quantitative representative study, however tempting the statistics are. When you look at the methodology, you can see that only 341 people took the survey and the survey was web-based.
Unfortunately I couldn’t find out how the survey was undertaken, and without that information it’s impossible to do a proper critique. For example, the executive statement claims a 95% confidence ratio, which doesn’t even make sense without other information. (Apologies to the authors if I’ve missed the detailed methodological explanation, but it should have been in the Executive Summary).
I’m particularly interested by how the web survey was distributed. Was the survey sent via email to journalists. If so, how were the journalists chosen? What was the ‘population’? To produce a representative survey, you need to be clear about the total population, and sample randomly from that population. In a world of fluid labour markets with increasing numbers of freelance journalists, how do you get a clear population, particularly when this survey boasts about its worldwide sample.
If it was a web-survey on a site where it was hoped journalists would stumble, you’re already self-selecting journalists who are spending more time on the web, and the types who are more likely to fill in web surveys. And even if it was an email with a web survey link attached, it would be self-selecting. That is fine, but the authors should have disclosed the number of journalists who received the email, and those who went on to complete the survey.
The reason I was sceptical about the figures, as much as I would like them to be true, is that I have spent the last 6 months leading training courses with BBC journalists about social media. While the journalists I have trained are mostly enthusiastic about learning more about these tools, my anecdotal evidence from training about the same number of journalists as were involved in this survey, does not support these figures.
That does not mean the BBC are behind in this regard. That also does not mean I have a self-selected sample because the people coming on the course are in some way ‘behind’. In many ways the journalists coming on the course are the ones who are ready to embrace it. But in very busy newsrooms, where resources are becoming increasingly limited (and I recognise the BBC is in a very special place in comparison to other newsrooms where income is not guaranteed), most journalists have not had the time to spend experimenting with these new tools. When your day is spent desperately trying to meet deadlines, there is no time to have a play with twitter, find relevant blogs, spend time verifying who the authors are, or battle with Facebook’s privacy settings.
Most journalists I train are aware of all the tools mentioned in this report, and want to feel more confident using them, but want time to experiment, and learn the subtleties before relying on them for newsgathering.
While they might have answered the question ‘have you used twitter in your reporting’ with a ‘yes’ because a colleague pointed them in the direction of twitter when a breaking news story unfolded and they were intrigued to see what was being said, that is different to actively using twitter as a newsgathering tool everyday. So when 48% of journalists from this survey said they have ‘used twitter’, how is ‘used’ defined? Does ‘used’ equal occasional or frequently lurking, or does ‘used’ mean regularly engaging with sources and contributing to the community?
I want to stress wholeheartedly that this post is not meant to highlight the shortcomings of BBC journalists. I have talked to a number of other people who lead training at other news organisations across the world, and I know everyone shares my experiences.
I offer anecdotal evidence to show how wary we have to be of research that is not statistically representative. This research is worthwhile and interesting, but beware of drawing conclusions which can not be drawn.
Social media offers everything the authors suggest in terms of disseminating news, finding story ideas and sources, monitoring sentiments and discussions, researching individuals and organisations, and keeping up on topics of interests and participating in conversations with the audience, but I would argue it is far from universally accepted in newsrooms.
There are some journalists who absolutely understand the value, engage daily, are wary of the potential pitfalls and their reporting is significantly improved because of it. The majority of journalists understand social media offers a range of new tool for newsgathering and building community, but understand it should be treated with caution, and the subtleties appreciated. I would say the same of this research.