机器学习-试卷-finalf15

CS189Fall2015IntroductiontoMachineLearningFinal•Pleasedonotturnoverthepagebeforeyouareinstructedtodoso.•Youhave2hoursand50minutes.•Pleasewriteyourinitialsonthetop-rightofeachodd-numberedpage(e.g.,write\AEifyouareAlexeiEfros).Completethisbytheendofyour2hoursand50minutes.•Theexamisclosedbook,closednotesexceptyourone-pagecheatsheet.•Nocalculatorsorotherelectronicdevicesallowed.•MarkyouranswersONTHEEXAMITSELFINTHESPACEPROVIDED.Ifyouarenotsureofyouransweryoumaywishtoprovideabriefexplanation.DoNOTattachanyextrasheets.•Thetotalnumberofpointsis150.Thereare15true/falsequestionsworth2pointseach,10multiplechoicequestionsworth3pointseach,and6descriptivequestionswithunequalpointassignments.•Fortrue/falsequestions,llintheTrue/Falsebubble.•Formultiple-choicequestions,llinthebubblesforALLCORRECTCHOICES:Theremaybemorethanonecorrectchoice,buttherewillbeatleastonecorrectchoice.NOPARTIALCREDIT:thesetofallcorrectanswersmustbechecked.FirstnameLastnameSIDFirstandlastnameofstudenttoyourleftFirstandlastnameofstudenttoyourright1Q1.[30pts]TrueorFalse(1)[2pts]RandomforestsusuallyperformbetterthanAdaBoostwhenyourdatasethasmislabeleddatapoints.TrueFalse(2)[2pts]Thediscriminantfunctioncomputedbykernelmethodsarealinearfunctionofitsparameters,notnecessarilyalinearfunctionoftheinputs.TrueFalse(3)[2pts]TheXORoperatorcanbemodeledusinganeuralnetworkwithasinglehiddenlayer(i.e.3-layernetwork).TrueFalse(4)[2pts]Convolutionalneuralnetworksarerotationinvariant.TrueFalse(5)[2pts]Makingadecisiontreedeeperwillassurebettertbutreducerobustness.TrueFalse(6)[2pts]Baggingmakesuseofthebootstrapmethod.TrueFalse(7)[2pts]K-meansautomaticallyadjuststhenumberofclusters.TrueFalse(8)[2pts]Dimensionalityreductioncanbeusedaspre-processingformachinelearningalgorithmslikedecisiontrees,kd-trees,neuralnetworksetc.TrueFalse(9)[2pts]K-dtreesguaranteeanexponentialreductioninthetimeittakestondthenearestneighborofanexampleascomparedtothenaivemethodofcomparingthedistancestoeveryotherexample.TrueFalse(10)[2pts]Logisticregressionisequivalenttoaneuralnetworkwithouthiddenunitsandusingcross-entropyloss.TrueFalse(11)[2pts]Convolutionalneuralnetworksgenerallyhavefewerfreeparametersascomparedtofullyconnectedneuralnetworks.TrueFalse(12)[2pts]K-medoidsisakindofagglomerativeclustering.TrueFalse(13)[2pts]Whiteningthedatadoesn'tchangetherstprincipaldirection.TrueFalse(14)[2pts]PCAcanbekernelized.TrueFalse(15)[2pts]PerformingK-nearestneighborswithK=Nyieldsmorecomplexdecisionboundariesthan1-nearestneighbor.TrueFalse2Q2.[30pts]MultipleChoice(1)[3pts]Whichofthefollowingguidelinesisapplicabletoinitializationoftheweightvectorinafullyconnectedneuralnetwork.ShouldnotsetittozerosinceotherwiseitwillcauseoverttingShouldnotsetittozerosinceotherwise(stochastic)gradientdescentwillexploreaverysmallspaceShouldsetittozerosinceotherwiseitcausesabiasShouldsetittozeroinordertopreservesym-metryacrossallneurons(2)[3pts]DuplicatingafeatureinlinearregressionCanreducetheL2-PenalizedResidualSumofSquares.DoesnotreducetheResidualSumofSquares(RSS).CanreducetheL1-PenalizedResidualSumofSquares(RSS).Noneoftheabove(3)[3pts]Whichofthefollowingis/areformsofregularizationinneuralnetworks.WeightdecayL2regularizationL1regularizationDropout(4)[3pts]Wearegivenaclassierthatcomputesprobabilitiesfortwoclasses(positiveandnegative).ThefollowingisalwaystrueabouttheROCcurve,andtheareaundertheROCcurve(AUC):AnAUCof0.5representsaclassierthatperformsworsethanrandom.WegenerateanROCcurvebyvaryingthediscriminativethresholdofourclassier.TheROCcurveallowsustovisualizethetradeobetweentruepositiveandfalsepositiveclassications.TheROCcurvemonotonicallyincreases.(5)[3pts]TheK-meansalgorithm:RequiresthedimensionofthefeaturespacetobenobiggerthanthenumberofsamplesHasthesmallestvalueoftheobjectivefunc-tionwhenK=1MinimizesthewithinclassvarianceforagivennumberofclustersConvergestotheglobaloptimumifandonlyiftheinitialmeansarechosenassomeofthesam-plesthemselvesNoneoftheabove(6)[3pts]Supposewhenyouaretrainingyourconvolutionalneuralnetwork,youndthatthetraininglossjustdoesn'tgodownafterinitialization.Whatcouldyoutrytoxthisproblem?ChangethenetworkarchitectureChangelearningratesEnsuretrainingdataisbeingreadcorrectlyFindabettermodelNormalizetheinputstothenetworkAddaregularizationterm3(7)[3pts]Logisticregression:Minimizescross-entropylossHasasimple,closedformanalyticalsolutionModelsthelog-oddsasalinearfunctionIsaclassicationmethodtoestimateclassposteriorprobabilities(8)[3pts]Selectallthetruestatements.Therstprincipalcomponentisuniqueuptoasignchange.Thelastprincipalcomponentisuniqueuptoasignchange.Allprincipalcomponentsareuniqueuptoasignchange.Ifsomefeaturesarelinearlydependent,atleastonesingularvalueiszero.Ifsomefeaturesarecorrelated,atleastonesingularvalueiszero.(9)[3pts]Selectallthechoicesthatmakethefollowingstatementtrue:In(a),thetrainingerrordoesnotincreaseas(b)increases.a:K-means,b:numberofiterationsa:Trainingneuralnetswithbackpropagationusingbatchgradientdecent,b:numberofiterationsa:Trainingneuralnetswithbackpropagationusingstochasticgradient