数据采集和营销工具（英文版）

Knowledgediscovery&dataminingTools,methods,andexperiencesFoscaGiannottiandDinoPedreschiPisaKDDLabCNUCE-CNR&Univ.Pisa@EDBT2000Konstanz,27-28.3.2000EDBT2000tutorial-Intro2ContributorsandacknowledgementsThepeople@PisaKDDLab:FrancescoBONCHI,GiuseppeMANCO,MircoNANNI,ChiaraRENSO,SalvatoreRUGGIERI,FrancoTURINIandmanystudentsThemanyKDDtutorialistsandteacherswhichmadetheirslidesavailableontheweb(allofthemlistedinbibliography);-)Inparticular:JiaweiHAN,SimonFraserUniversity,whoseforthcomingbookDatamining:conceptsandtechniqueshasinfluencedthewholetutorialRajeevRASTOGIandKyuseokSHIM,LucentBellLabsDanielA.KEIM,UniversityofHalleDanielSilver,CogNovaTechnologiesTheEDBT2000boardwhoacceptedourtutorialproposalKonstanz,27-28.3.2000EDBT2000tutorial-Intro3TutorialgoalsIntroduceyoutomajoraspectsoftheKnowledgeDiscoveryProcess,andtheoryandapplicationsofDataMiningtechnologyProvideasystematizationtothemanymanyconceptsaroundthisarea,accordingthefollowinglinestheprocessthemethodsappliedtoparadigmaticcasesthesupportenvironmenttheresearchchallengesImportantissuesthatwillbenotcoveredinthistutorial:methods:timeseries,exceptiondetection,neuralnetssystems:parallelimplementationsKonstanz,27-28.3.2000EDBT2000tutorial-Intro4TutorialOutline1.Introductionandbasicconcepts1.Motivations,applications,theKDDprocess,thetechniques2.DeeperintoDMtechnology1.DecisionTreesandFraudDetection2.AssociationRulesandMarketBasketAnalysis3.ClusteringandCustomerSegmentation3.Trendsintechnology1.KnowledgeDiscoverySupportEnvironment2.Tools,LanguagesandSystems4.ResearchchallengesKonstanz,27-28.3.2000EDBT2000tutorial-Intro5Introduction-moduleoutlineMotivationsApplicationAreasKDDDecisionalContextKDDProcessArchitectureofaKDDsystemTheKDDstepsinshortKonstanz,27-28.3.2000EDBT2000tutorial-Intro6EvolutionofDatabaseTechnology:fromdatamanagementtodataanalysis1960s:Datacollection,databasecreation,IMSandnetworkDBMS.1970s:Relationaldatamodel,relationalDBMSimplementation.1980s:RDBMS,advanceddatamodels(extended-relational,OO,deductive,etc.)andapplication-orientedDBMS(spatial,scientific,engineering,etc.).1990s:Datamininganddatawarehousing,multimediadatabases,andWebtechnology.Konstanz,27-28.3.2000EDBT2000tutorial-Intro7Motivations“NecessityistheMotherofInvention”Dataexplosionproblem:Automateddatacollectiontools,maturedatabasetechnologyandinternetleadtotremendousamountsofdatastoredindatabases,datawarehousesandotherinformationrepositories.Wearedrowningininformation,butstarvingforknowledge!(JohnNaisbett)Datawarehousinganddatamining:On-lineanalyticalprocessingExtractionofinterestingknowledge(rules,regularities,patterns,constraints)fromdatainlargedatabases.Konstanz,27-28.3.2000EDBT2000tutorial-Intro8Alsoreferredtoas:Datadredging,Dataharvesting,DataarcheologyAmultidisciplinaryfield:DatabaseStatisticsArtificialintelligenceMachinelearning,ExpertsystemsandKnowledgeAcquisitionVisualizationmethodsArapidlyemergingfieldArapidlyemergingfieldKonstanz,27-28.3.2000EDBT2000tutorial-Intro9MotivationsforDMAbundanceofbusinessandindustrydataCompetitivefocus-KnowledgeManagementInexpensive,powerfulcomputingenginesStrongtheoretical/mathematicalfoundationsmachinelearning&logicstatisticsdatabasemanagementsystemsKonstanz,27-28.3.2000EDBT2000tutorial-Intro10WhatisDMusefulfor?MarketingDatabaseMarketingDataWarehousingKDD&DataMiningIncreaseknowledgetobasedecisionupon.E.g.,impactonmarketingKonstanz,27-28.3.2000EDBT2000tutorial-Intro11TheValueChainData•Customerdata•Storedata•DemographicalData•GeographicaldataInformation•XlivesinZ•SisYyearsold•XandSmoved•WhasmoneyinZKnowledge•AquantityYofproductAisusedinregionZ•CustomersofclassYusex%ofCduringperiodDDecision•PromoteproductAinregionZ.•MailadstofamiliesofprofileP•Cross-sellserviceBtoclientsCKonstanz,27-28.3.2000EDBT2000tutorial-Intro12ApplicationAreasandOpportunitiesMarketing:segmentation,customertargeting,...Finance:investmentsupport,portfoliomanagementBanking&Insurance:creditandpolicyapprovalSecurity:frauddetectionScienceandmedicine:hypothesisdiscovery,prediction,classification,diagnosisManufacturing:processmodeling,qualitycontrol,resourceallocationEngineering:simulationandanalysis,patternrecognition,signalprocessingInternet:smartsearchengines,webmarketingKonstanz,27-28.3.2000EDBT2000tutorial-Intro13ClassesofapplicationsMarketanalysistargetmarketing,customerrelationmanagement,marketbasketanalysis,crossselling,marketsegmentation.RiskanalysisForecasting,customerretention,improvedunderwriting,qualitycontrol,competitiveanalysis.FrauddetectionText(newsgroup,email,documents)andWebanalysis.Konstanz,27-28.3.2000EDBT2000tutorial-IntroMarketAnalysisWherearethedatasourcesforanalysis?Creditcardtransactions,loyaltycards,discountcoupons,customercomplaintcalls,plus(public)lifestylestudies.TargetmarketingFindclustersof“model”customerswhosharethesamecharacteristics:interest,incomelevel,spendinghabits,etc.DeterminecustomerpurchasingpatternsovertimeConversionofsingletoajointbankaccount: