Ranking the Research Productivity of Library and I

ThisstudyevaluatesthedatasourcesandresearchmethodsusedinearlierstudiestoranktheresearchproductivityofLibraryandInformationScience(LIS)facultyandschools.Indoingso,thestudyidentifiesbothtoolsandmethodsthatgeneratemoreaccuratepublica-tioncountrankingsaswellasdatabasesthatshouldbetakenintoconsiderationwhenconductingcomprehen-sivesearchesintheliteratureforresearchandcurricularneeds.Withalistof2,625itemspublishedbetween1982and2002by68facultymembersof18AmericanLibraryAssociation–(ALA-)accreditedLISschools,hundredsofdatabasesweresearched.Resultsshowthatthereareonly10databasesthatprovidesignificantcoverageoftheLISindexedliterature.Resultsalsoshowthatrestrict-ingthedatasourcestoone,two,oreventhreedatabasesleadstoinaccuraterankingsanderroneousconclusions.BecausenodatabaseprovidescomprehensivecoverageoftheLISliterature,researchersmustrelyonawiderangeofdisciplinaryandmultidisciplinarydatabasesforrankingandotherresearchpurposes.Thestudyanswerssuchquestionsasthefollowing:IstheAssociationofLibraryandInformationScienceEducation’s(ALISE’s)directoryofmembersareliabletooltoidentifyacompletelistoffacultymembersatLISschools?Howmanyandwhichdatabasesareneededinamultifilesearchtoarriveataccuratepublicationcountrankings?Whatcoveragewillbeachievedusingacertainnumberofdatabases?Whichresearchareasarewellcoveredbywhichdata-bases?Whatalternativemethodsandtoolsareavailabletosupplementgapsamongdatabases?Didcoverageperformanceofdatabaseschangeovertime?WhatcountingmethodshouldbeusedwhendeterminingwhatandhowmanyitemseachLISfacultyandschoolhaspub-lished?Theauthorsrecommendadvancedanalysisofresearchproductivitytoprovideamoredetailedassess-mentofresearchproductivityofauthorsandprograms.IntroductionAlthoughrankingsofacademicunitshadbeenapartoftheU.S.academiclandscapefornearly100yearsbeforethe1980s,theyreceivedlittleattentionfromconstituenciesotherthanadministrators,federalagencies,graduateschoolapplicants,andhighereducationresearchers(Stuart,1995).Overthelasttwodecades,however,nationalrankingshavebecomemorewidelyreadandmoreinfluentialamongstudents,parents,andtheacademiccommunitythaneverbefore,primarilybecauseofthepublicationofrankingsbymasscirculationmagazines,suchastheU.S.News&WorldReport’sAmerica’sBestCollege(1983–),Moneymaga-zine’sMoneyGuide:YourBestCollegeBuysNow(1990–),Time/ThePrincetonReview(1992–),andKaplan/Newsweek’sHowtoGetIntoCollege(1997–).Itisestimatedthatthesefourtitlesalonesellapproximately7millioncopiesofnewsmagazinecollegerankingsandguidesannually(McDonough,Antonio,Walpole,&Perez,1998).1Today,thereislittledoubtthatmasscirculationrankingsandthosepublishedinacademicjournalshavebecomeincreasinglyimportanttoschools,colleges,anduniversities,particularlybecausethestatusconveyedbytheminfluencesacademicinstitutions’abilitiesto(1)attracthigher-qualityfaculty,students,andadministrators;(2)generateincreasingsupportfromalumnianddonors;and(3)provideexcellentplacementopportunitiestograduates(Machung,1998;Roush,1995;Stock&Alston,2000).RankingshavebecomeJOURNALOFTHEAMERICANSOCIETYFORINFORMATIONSCIENCEANDTECHNOLOGY,56(12):1314–1331,2005RankingtheResearchProductivityofLibraryandInformationScienceFacultyandSchools:AnEvaluationofDataSourcesandResearchMethodsLokmanI.MehoSchoolofLibraryandInformationScience,IndianaUniversity,1320E.10thStreet,LI011,Bloomington,IN47405.E-mail:[e-mail]onandLibraryScience,UniversityofNorthCarolina,201ManningHall,CB#3360,ChapelHill,NC27599-3360.E-mail:[e-mail]3,2003;revisedMay24,2004,July20,2004;acceptedSeptember2,2004©2005WileyPeriodicals,Inc.•Publishedonline4August2005inWileyInterScience().DOI:10.1002/asi.202271Foradetailedhistoricalaccountofrankingsinnationalmagazines,reports,andscholarlypapers,seeHoward(2002).increasinglyimportantalsobecausetheycansupportand/orthreateninstitutions’coreorganizationalidentitiesandfunc-tions,inadditiontoaffectingadmissionspoliciesandthenumberofstudentapplicationsreceived(Elsbach&Kramer,1997;Machung,1998;Monks&Ehrenberg,1999).Univer-sitypresidentsanddeansarewellawareofthepublishedrankingsandoftencitepositiveonesintheirspeechesandprogramevaluations.Rankingimpliesbothevaluationandqualityandoftenisbasedon,orisinfluencedby,theresearchperformanceoffacultymembersandtheiracademicdepartments.Qualityinthissenseisgenerallydefinedasameasureoftheextenttowhichanidea,paper,author,orinstitutionhascontributedtotheprogressofknowledge(vanRaan,1996).Today,thegeneralpracticeforevaluatingandrankingtheresearchperformanceoffacultymembersandtheiracademicunitsinmanycollegesanduniversitiesistorelyonthreeinterrelatedcriteriaordata:theopinionsoffacultymembersandadministrators,listsofpublications,andcitationcounts.Rankingsbasedonopinionsoffacultymembersandadministratorsarealmostuniversalintermsofthemethodusedfordatacollection,namely,surveys.Therationalebehindperception-baseddataisthat,sincerankingpertainstothestatuspositionaffordedtoadepartmentoruniversitybyrelevantothers(e.g.,facultyandadministrators),themostdirectapproachwouldap