In several of his writings, Peter Jacso has indicated the inconsistencies of Google Scholar. One important flaw is the citation search. Both his web published paper "Google Scolar and The scientist" and the article "As we may search" published in Current Science 2005 (please see References to literature) discuss the problems.

My testings indicates less inconsistencies than before, but still they exist. The article "An Examination of Citation Counts in a New Scholarly Communication Environment" published in D-Lib magazine September 2005 Vol. 11, No. 9. by Kathleen Bauer et al at Yale University Library made some comparisions of the average number of times an article is cited. They checked the citation frequency of each article for a certain year, in this case both 1985 and 2000, in the Journal of the American Society for Information Science and Technology (JASIST). The search for 2000 showed Google Scholar had 4.5 more citations than Web of Science and 3.9 more citations than Scopus. But searching 1985 Web of Science had 8.7 more citations than Google Scholar and Google Scholar had "just" 2.9 more citations than Scopus. The major shortcoming in this article is that they never analyzed the inconsistencies with Google Scholar citation Search. The citation count doesn't always work properly. Here's an example of a record that indicates Cited by 15 (other sources):

When clicking this link Cited by 15 you will find only 14 citations:

Here's another example of an article by P Jacso himself. Cited by 3 sources according to Google Scholar:

When clicking this link Cited by 3 you will find only 2 citations:

This search on semiconductors is an example from Jacso. In this reference it seems like the article is published 2006, but checking the source shows it's published 1990 and 2006 is the starting page of the article:

Jacso has also pointed out the flaws of duplication in his article "As we may search" in Current Science. Google Scholar works hard with the ability to cluster duplicate articles together. If you look at the preceding screenshot after the title you see the link group of 3>>. Clicking that link shows you 3 duplicates. Because Google Scholar indexes not just peer-reviewed journal articles, but preprint archives, conference papers, master thesis, webpublished materials etc you understand they have a hard problem to discover duplicates.

Here's an example. Searching sojka modeling drop size distributions gives as the first hit an article by Bainsky and Sojka with title "Modeling drop size distributions". That article should be Cited by 7 other sources according to Google Scholar.

By clicking Cited by 7 you find 7 hits but two of them are duplicates. View the two titles "Modeling Spray Impingement using Linear Stability Theories for Droplet Shattering". Though the first title has a link to group of 2>>.

Conclusion: Don't trust the Google Scholar citation counting without manually checking it for inconsistencies in terms of counting and clustering duplicates.

Advertisements