Machine Learning and Data Mining

GerstnerLaboratoryforIntelligentDecisionMakingandControlCzechTechnicalUniversityinPragueSeriesofResearchReportsReportNo:GL157/02MachineLearningandDataMiningJir´Palou[e-mail]a2,16627Prague6,CzechRepublictel.([phone],fax:[phone]{Instance-BAsedREasoningTool113.1iBARETstructure.................................113.2CQLServer.....................................133.3Consultation....................................153.4TestingSetEvaluation...............................153.4.1ClassicationTask.............................163.4.2RegressionTask..............................193.5IBRModelTuning.................................203.5.1SequentialAlgorithm............................203.5.2GeneticAlgorithm.............................213.6FutureWork....................................234Experiments255FutureResearch286Conclusion28References291ListofFigures1ROCcurveforexampleadTable1(ROCarea0:8)...............102TheiBARETblokstructure............................123Exampleofpredictionvaluesofsinusfunction..................204PMMLutilization.................................245Procedure6-iBARET'strainingerror......................266Procedure6-iBARET'sperformanceontestingdata.............267Procedure6-iBARET'sperformancewithreducednumberofpatientgroups27ListofTables1Example{calculatingROCcurve........................92Four-foldtable...................................93Exampleofasymboldistancetable........................144Exampleofthesecondtypeofasymboldistancetable.............145Example{RMSEofdierentLWRmethod...................1921IntroductionArticialIntelligence(AI)istheareaofcomputersciencefocusingoncreatingmachinesthatcanengageonbehaviorsthathumansconsiderintelligent.Theabilitytocreateintelligentmachineshasattractedhumanssinceancienttimes,andtodaywiththehugeexpansionofthecomputersand50yearsofresearchofAIprogrammingtechniques,thedreamofintelligentmachinesisbecomingareality.MachineLearning(ML)istheareaofArticialIntelligencethatfocusesondevelopingprinciplesandtechniquesforautomatingacquisitionofknowledge.Somemachinelearn-ingmethodscandramaticallyreducethecostofdevelopingknowledge-basedsoftwarebyextractingknowledgedirectlyfromexistingdatabases.Othermachinelearningmethodsenablesoftwaresystemstoimprovetheirperformanceovertimewithminimalhumaninter-vention.Theseapproachesareexpectedtoenablethedevelopmentofeectivesoftwareforautonomoussystemsthatcanoperateinpoorlyunderstoodenvironments.TheaimofthisworkistomakeshortoverviewofmostfrequentlyusedMachineLearningmethods,andtointroduceourresearchfocus.Thisreportcanbedividedintwomainparts.TherstoneconcentratesmainlyonusedMachineLearningmethods.EachimportantMLmethodisbrieydescribedandappraiseditssignicance.Thenweshowrelationtothenextbranchofarticialintelligence{DataMining.AfterthatwefocusonproblemsineachphaseinMachineLearningprocess.SomeopportunitiesofpreprocessingareshownandapowerfulpreprocessingtoolSumatraTTisintroduced.Thenwediscussproblemsingenerallearningphase,testingphaseandshowpossibilitiesinresultsevaluation.Attheendoftherstpart,thereismentionedpopularlanguageforpredictivemodelexchange{PMML.Thesecondpartdescribesoursolutionforclassicationandprediction{iBARET.AlltechniquesthatarecoveredbyiBARETareexplained,fromconsultationprocesstomodeltuningbygeneticalgorithm.Thenwetrytosketchfutureevaluationofthetool.AttheendweshowmostrecentexperimentonSPAdatamadebyiBARET.Thenalpartofthisworkincludesreectiononpossibledirectionsoffurtherresearch.TherewetrytondsomeinterestingtopicwewouldliketoworkoninframeofPhDthesis.32MachineLearningTherstpartofthissectioncontainstheoverviewofthemachinelearningmethods.Wefocusedonthebasic,frequentlyusedmethodsandseveral,atpresent,mostpopularmethods.AfterthisintroductiontoMachineLearning(ML)wewilltrytodescriberelationshipbetweenMLandDataMining(DM).ThenwefocusedonthewholeML/DMprocessfromthedatapreprocessing,throughthegenerallearning,tothetestinga