In times past, counting up scholarly citations– i.e. how often other academics were using your work for their own research – was one of the only ways to know how widely read and appreciated a piece of research had been.
But today, a number of journals publish more modern alternative metrics, or altmetrics, such as how many times articles have been tweeted, shared on Facebook, downloaded, or written up in news reports. And institutions and scientists can track responses to their work using services offered by new nonprofits and companies.
“TWITTER GIVES YOU CONNECTIONS BEYOND THE IVORY TOWER THAT YOU DON’T NORMALLY HAVE”
But what does it mean to have a paper go viral on social media? And what’s more important: tweet-ability or the traditional citation from the scientific community?
A study analyzing Twitter links to biomedical articles, which was published last month in the Journal of the American Society for Information Science and Technology, indicates that popularity on Twitter is probably not a reliable measure of scholarly esteem. Twitter mentions show a low correlation with citations, so a tweetable paper isn’t necessarily a well respected one.
“It’s one of the largest studies that looked at the connection between scientific articles and their [mentions] on social media,” said Emily Darling, a David H Smith Conservation Research Fellow at the University of North Carolina who studies both coral reefs and how scholars use Twitter. “It was interesting that there’s no connection to scholarly impact.”
“I was surprised that the correlation was so low,” said Euan Adie, cofounder ofAltmetric, a London-based company that tracks Internet response to scholarly works. The study reinforces a point Adie and his Altmetrics colleagues often make: high levels of online engagement do not necessarily say anything about an article’s scholarly quality.
Lead author Stefanie Haustein, a researcher at the University of Montreal, and colleagues used data from Altmetric to track tweets of links to papers published between 2010 and 2012. The study encompassed 1.4 million studies indexed on PubMed, a site that catalogs biomedical articles, and Web of Science, a site that tracks scholarly citations. Haustein’s team found that less than 10 percent of these articles were ever tweeted at all. They used only papers published in 2011 to check correlations between tweets and citations.
“Social media was never meant to replace traditional statistics like journal impact factors or article citations,” Darling writes in an article published last week on The Conversation that responds to Haustein’s study. Rather, “Twitter gives you connections beyond the ivory tower that you don’t normally have,” she said. As a conservationist, for instance, she hopes to spread her research to policymakers and resource managers, so they can take actions based on it. And she wants to inform the public.
But Haustein says that more work is needed to distinguish between the various reasons an article is tweeted—whether to spread important information or for fun. For instance, Altmetric last week released a list of the top 100 papers that received the most online attention in 2013, including tweets, Facebook posts, news stories, blog posts, and more.
Some papers were clearly shared because they were of vital interest to the public. A study related to the Fukushima nuclear disaster in Japan topped the list. It was an open-access paper that chronicled the levels of radioactive cesium found in freshwater fish around Japan, and it was tweeted largely by members of the public, with the highest representation in Japan.