Opinion-mining-and-sentiment-analysis

FoundationsandTrendsinInformationRetrievalVol.2,No1-2(2008)1–135c2008BoPangandLillianLee.Thisisapre-publicationversion;thereareformattingandpotentiallysmallwordingdifferencesfromthefinalversion.DOI:xxxxxxOpinionminingandsentimentanalysisBoPang1andLillianLee21Yahoo!Research,701FirstAve.Sunnyvale,CA94089,U.S.A.,[e-mail]2ComputerScienceDepartment,CornellUniversity,Ithaca,NY14853,U.S.A.,[e-mail]rtofourinformation-gatheringbehaviorhasalwaysbeentofindoutwhatotherpeoplethink.Withthegrowingavailabilityandpopularityofopinion-richresourcessuchasonlinereviewsitesandpersonalblogs,newopportunitiesandchallengesariseaspeoplenowcan,anddo,activelyuseinformationtechnologiestoseekoutandunderstandtheopinionsofothers.Thesuddeneruptionofactivityintheareaofopinionminingandsentimentanalysis,whichdealswiththecomputationaltreatmentofopinion,sentiment,andsubjectivityintext,hasthusoccurredatleastinpartasadirectresponsetothesurgeofinterestinnewsystemsthatdealdirectlywithopinionsasafirst-classobject.Thissurveycoverstechniquesandapproachesthatpromisetodirectlyenableopinion-orientedinformation-seekingsystems.Ourfocusisonmethodsthatseektoaddressthenewchallengesraisedbysentiment-awareapplications,ascomparedtothosethatarealreadypresentinmoretraditionalfact-basedanalysis.Weincludematerialonsummarizationofevaluativetextandonbroaderissuesregardingprivacy,manipulation,andeconomicimpactthatthedevelopmentofopinion-orientedinformation-accessservicesgivesriseto.Tofacilitatefuturework,adiscussionofavailableresources,benchmarkdatasets,andevaluationcampaignsisalsoprovided.ContentsTableofContentsi1Introduction11.1Thedemandforinformationonopinionsandsentiment11.2Whatmightbeinvolved?Anexampleexaminationoftheconstructionofanopinion/reviewsearchengine31.3Ourchargeandapproach41.4Earlyhistory41.5Anoteonterminology:Opinionmining,sentimentanalysis,subjectivity,andallthat52Applications72.1Applicationstoreview-relatedwebsites72.2Applicationsasasub-componenttechnology72.3Applicationsinbusinessandgovernmentintelligence82.4Applicationsacrossdifferentdomains93Generalchallenges103.1Contrastswithstandardfact-basedtextualanalysis103.2Factorsthatmakeopinionminingdifficult114Classificationandextraction15PartOne:Fundamentals164.1Problemformulationsandkeyconcepts164.1.1Sentimentpolarityanddegreesofpositivity164.1.2Subjectivitydetectionandopinionidentification18i4.1.3Jointtopic-sentimentanalysis194.1.4Viewpointsandperspectives194.1.5Othernon-factualinformationintext204.2Features204.2.1Termpresencevs.frequency214.2.2Term-basedfeaturesbeyondtermunigrams214.2.3Partsofspeech214.2.4Syntax224.2.5Negation224.2.6Topic-orientedfeatures23PartTwo:Approaches234.3Theimpactoflabeleddata244.4Domainadaptationandtopic-sentimentinteraction254.4.1Domainconsiderations254.4.2Topic(andsub-topicorfeature)considerations264.5Unsupervisedapproaches274.5.1Unsupervisedlexiconinduction274.5.2Otherunsupervisedapproaches284.6Classificationbasedonrelationshipinformation294.6.1Relationshipsbetweensentencesandbetweendocuments294.6.2Relationshipsbetweendiscourseparticipants294.6.3Relationshipsbetweenproductfeatures304.6.4Relationshipsbetweenclasses314.7Incorporatingdiscoursestructure324.8Languagemodels324.9Specialconsiderationsforextraction334.9.1Identifyingproductfeaturesandopinionsinreviews354.9.2Problemsinvolvingopinionholders355Summarization375.1Single-documentopinion-orientedsummarization375.2Multi-documentopinion-orientedsummarization385.2.1Someproblemconsiderations395.2.2Textualsummaries415.2.3Non-textualsummaries435.2.4Review(er)quality496Broaderimplications556.1Economicimpactofreviews566.1.1Surveyssummarizingrelevanteconomicliterature586.1.2Economic-impactstudiesemployingautomatedtextanalysis586.1.3Interactionswithwordofmouth(WOM)59ii6.2Implicationsformanipulation597Publiclyavailableresources617.1Datasets617.1.1Acquiringlabelsfordata617.1.2Anannotatedlistofdatasets627.2Evaluationcampaigns657.2.1TRECopinion-relatedcompetitions657.2.2NTCIRopinion-relatedcompetitions667.3Lexicalresources667.4Tutorials,bibliographies,andotherreferences678Concludingremarks69References71iii1IntroductionRomanceshouldneverbeginwithsentiment.Itshouldbeginwithscienceandendwithasettlement.—OscarWilde,AnIdealHusband1.1Thedemandforinformationonopinionsandsentiment“Whatotherpeoplethink”hasalwaysbeenanimportantpieceofinformationformostofusduringthedecision-makingprocess.LongbeforeawarenessoftheWorldWideWebbecamewidespread,manyofusaskedourfriendstorecommendanautomechanicortoexplainwhotheywereplanningtovoteforinlocalelections,requestedreferencelettersregardingjobapplicantsfromcolleagues,orconsultedConsumerReportstodecidewhatdishwashertobuy.ButtheInternetandtheWebhavenow(amongotherthings)madeitpossibletofindoutabouttheopinionsandexperiencesofthoseinthevastpoolofpeoplethatareneitherourpersonalacquaintancesnorwell-knownprofessionalcritics—thatis,peoplewehaveneverheardof.Andconversely,moreandmorepeoplearemakingtheiropinionsavailabletostrangersviatheInternet.Indeed,accordingtotwosurveysofmorethan2000Ame