Skip to main content

Sociology Graduate Students: Research Impact Metrics

Overview of sociology resources for beginning graduate students

What is Citation Analysis?

Citation analysis measures the impact of a published work (e.g. book or article), researcher, or institution based on the number of times the work, person, or institution has been cited by others.  You can use citation analysis to measure your own scholarly impact, which is helpful for convincing employers and funding agencies that your work is important.  You can also use citation analysis to identify important work in your field.    

Different citation analysis tools are more or less effective for different disciplines - depending on the publishing and citing norms in the field.  For a general overview of citation analysis tools and how to use them, please see the Impact Factors and Citation Analysis LibGuide.  

Measuring Scholarly Impact

Article/book impact: The value of particular works, such as journal articles, conference proceedings, and books, can be measured by the number times they are cited by other works and alternative metrics such as tweets, blog posts, likes, bookmarks, etc. 

 

Journal impact: The importance of particular academic journals can be measured by the number of times their articles are cited and where they are cited. 

 

Researcher impact: The success of particular researchers can be measured by the number of works they publish and the number of times their works are cited. 

 

Institutional impact: The prestige of a department or area of research within an institution can be measured by the collective impact of its individual researchers compared to those at other institutions. 

Using Multiple Tools

Best practice in calculating scholarly impact requires using multiple tools.  In their study of the work of Library and Information Science faculty, Meho and Yang (2007) found surprisingly little overlap in the citations produced by Web of Science, Scopus, and Google Scholar - only 58.2 percent of these citations appeared in both Web of Science and Scopus.  

To see how widely the h-index can vary across citation tools, take a look at how Scopus, Web of Science, and HPoP calculate h-index for two anonymous scholars at different stages of their career.

 

 

You might also try: 

  • "In general, Scopus provides a higher citation count than ISI, both in the Sciences and the Social Sciences and Humanities.  In the Sciences, this increase is only marginal (except for Computer Science), whilst in the Social Science and Humanities, the increase is substantial" (Harzing 2013).  
  • A major weakness of Scopus, is that the database has only just begun updating references prior to 1996.  Scholars with iconic older works may find that Web of Science gives them better results. 

What's Special about Sociology?

As a discipline, sociology has several features that influence which citation metric are most useful for measuring scholarly impact:

  • Journal articles age well in this discipline.  "In sociology, it is not uncommon for papers to grow in influence for a decade or more after publication" (Jacobs 2016: 195).  
  • Books and articles are both important forms of scholarship in the field.  However, the emphasis placed on books versus articles depends on several factors.  Sociologists at private universities tend to "prefer" books, while those at public universities are more likely to publish articles (Clemens, Powell, McIlwaine, and Okamoto 1995).  "Books are much more likely to employ qualitative data, while journal articles more frequently utilize quantitative data" (Clemens et al. 1995: 459).  For this reason, it is not uncommon for sociologists to think of themselves as either "book people" or "article people".  
    • The distinction between articles and books matters because books in sociology are much more likely to be cited outside the discipline than articles are.  Indeed, "regardless of the total number of citations, books receive the majority of their citations from outside the discipline" (Clemens et al. 1995: 460-461).    
  • Sociology is frequently cited by non-sociologists.  "Over half the studies that draw on sociological research are published outside the field, and over 40 percent appear outside the social sciences" (Jacobs 2013: 81).  In comparison, only 30 percent of citations to chemistry appear in non-chemistry journals, and 40 percent of citations to biological research occur outside this discipline.     

This means that for sociology, the ideal index will consider: (1) the age of the article, and by extension, the age of the author, (2) both books and articles as sources and objects of citations, and (3) sources from a variety of disciplines. 

Recommended for Sociologists

While Web of Science (WoS) is often a go-to tool for measuring a scholar's impact, Harzing's Publish or Perish has several advantages over WoS for sociologists: 

HARZING'S PUBLISH OR PERISH (HPoP)

Image from http://harzing.com/resources/publish-or-perish​.

Strengths Weaknesses
HPoP is powered by Google Scholar, which has more comprehensive journal coverage for Sociology than Web of Science (Kousha and Thelwall 2007).     HPoP does not cover as many chemistry, biology, and physics journal than Web of Science.  But, this weakness is less important for sociologists, who are not regularly cited by these disciplines.
Coverage of books in Web of Science is limited - as of November 2014, the index covered only 38 percent of books published in the Social and Behavioral Sciences (Thomson Reuters 2016).  HPoP is therefore a stronger tool for a field where books are an important form of scholarship.

HPoP tends to overestimate the number of publications an author has because it includes book reviews, grey literature, notes, and articles in newsletters.  If the author appears in someone else's acknowledgments or in a list of reviewer for a particular journal, he/she also receives credit for these "citations".  Calculating an accurate h-index through HPoP may require more cleaning than indices calculated with other tools.

Who Should Use It?

  • Junior Scholars.  Google Scholar captures more citations of conference proceedings and theses - outputs that sociologist are likely to have more of earlier in their careers.  
    • For junior scholars, HPoP also makes it easy to calculated the m-index, which accounts for the length of a researcher's career (i.e. the h-index divided by the number of years since the researcher's first publication). 
  • Authors with one or two iconic publications. For these scholars, HPoP offers easy calculation of the g-index, which averages the total citations of articles above the author's h-index limit.  In other words, a g-score of 16 means that the top 16 articles an author has published have received on average g citations.  Authors with one or two publications that have garnered a lot of attention will have much higher g-scores than g-scores, as these standout books/articles will pull of the mean for their top publications.   
  • Authors who publish internationally. English-language journals are over-represented in Web of Science as compared to Google Scholar (Gonzalez-Alcaide, Valderrama-Zurian, and Aleixandre-Benavent 2012).     

Might be less useful for: 

  • Older scholars. The majority of citations from journals and conference papers in Google Scholar are from after 1994 (Meho and Yang 2007). However, given that publications age well in sociology, this may not be a great disadvantage.  Iconic articles from the 1980s, for example, will continue to be cited after 1993. 
© Emory University Libraries - 540 Asbury Circle, Atlanta, Georgia 30322