18Multiview-and-Photometric-Stereo-计算机视觉-Berkeley课

C280ComputerVisionC280, Computer VisionProf. Trevor [e-mail] 18:  Multiviewand Photometric StereoToday•Multiviewstereo revisited•Shapefromlargeimagecollections•Shape from large image collections•VoxelColoring•Digital Forensics•Photometric StereoMulti-viewStereo[Seitz]Multi-viewStereoInput:calibratedimagesfromseveralviewpointsOutput:3DobjectmodelOutput:3DobjectmodelFiguresbyCarlosHernandez[Seitz]History8Numberofmulti-viewstereopapersinCVPR,ECCV,andICCVnumberofpapersinCVPR,ECCV,andICCV,byyear45671230199519961997199819992000200120022003200420052006F&KiFFurukawaHernandezTranFaugeras&KerivenLevelsetstereoFuaOrientedparticlesSeitz&DyerVoxelcoloringNarayanan&KanadeFromherz&Bichselshapefrommultiplecues,&SGoeseleVogiatzisPonsHornungKolmogorov&Zabihmulti-viewgraphcut[Seitz]Narayanan&KanadeVirtualizedRealitypp,ICYCS,(precursortoKutulakos&Seitz,spacecarving)Kutulakos&SeitzSpacecarvingFuaNarayanan,Rander,KanadeSeitz,Dyer199519971998Faugeras,Keriven1998[Seitz]Hernandez,SchmittPons,Keriven,FaugerasFurukawa,Ponce200420052006Goeseleetal.2007Stereo:basicideaerrordepthdepth[Seitz]ChoosingthestereobaselinellfthidthfallofthesepointsprojecttothesamepairofpixelswidthofapixelLargeBaselineLargeBaselineSmallBaselineSmallBaselineWhat’stheoptimalbaseline?•Toosmall:largedeptherror•Toolarge:difficultsearchproblem[Seitz]•Toolarge:difficultsearchproblemTheEffectofBaselineonDepthEstimation[Seitz]1/zwidthofapixelpixelmatchingscorewidthofapixel1/z[Seitz][Seitz]MultibaselineStereoBasicApproach•Chooseareferenceview•UseyourfavoritestereoalgorithmBUTreplacetwo-viewSSDwithSSSDoverallbaselinesLimitationsLimitations•Onlygivesadepthmap(notan“objectmodel”)•Won’tworkforwidelydistributedviews:[Seitz]Problem:visibilitySomeSolutions•Matchonlynearbyphotos[Narayanan98]•UseNCCinsteadofSSD[Seitz]•UseNCCinsteadofSSD,IgnoreNCCvaluesthreshold[Hernandez&Schmitt03]MergingDepthMapsvrip[CurlessandLevoy1996]•computeweightedaverageofdepthmapspggppsetofdepthmaps(oneperview)mergedsurfacemesh[Seitz](oneperview)meshMergingdepthmapsdepthmap1depthmap2UnionNaïvecombination(union)producesartifactsNaïvecombination(union)producesartifactsBettersolution:find“average”surface•Surfacethatminimizessum(ofsquared)distancestothedepthmaps(q)pp[Seitz]LeastsquaressolutionE(f)d2N(xf)dx[Seitz]E(f)dii1(x,f)dxVRIP[Curless&Levoy1996]depthmap1depthmap2combinationisosurfaceextractionsigneddistancedistancefunction[Seitz]MergingDepthMaps:TempleModel16images(ring)47images(ring)317images(hih)inputimagegroundtruthmodel(hemisphere)Goesele,Curless,Seitz,2006MichaelGoesele[Seitz]Multi-viewstereofromInternetCollections[Goesele, Snavely, Curless, Hoppe, Seitz, ICCV 2007] [Seitz]Challenges•appearancevariation•resolution•massivecollections[Seitz]LargeImageCollections[Seitz]206Flickrimagestakenby92photographers4btihbii4bestneighboringviewsreferenceviewLocalviewselection[Seitz]•Automaticallyselectneighboringviewsforeachpointintheimage•Desiderata:goodmatchesANDgoodbaselines4btihbii4bestneighboringviewsreferenceviewLocalviewselection[Seitz]•Automaticallyselectneighboringviewsforeachpointintheimage•Desiderata:goodmatchesANDgoodbaselines4btihbii4bestneighboringviewsreferenceviewLocalviewselection[Seitz]•Automaticallyselectneighboringviewsforeachpointintheimage•Desiderata:goodmatchesANDgoodbaselinesResultsStPeterTreiFontainMtRshmore[Seitz]St.Peter151images50photographersTreviFountain106images51photographersMt.Rushmore160images60photographersNotreDamedeParis653images313photographers[Seitz][Seitz][Seitz][Seitz]129Flickrimagestakenby98photographers[Seitz]mergedmodelofVenusdeMilo[Seitz]56Flickrimagestakenby8photographers[Seitz]mergedmodelofPisaCathedral[Seitz]Accuracycomparedtolaserscannedmodel:90%ofpointswithin0.25%ofgroundtruthProblem:visibilitySomeSolutions•Matchonlynearbyphotos[Narayanan98]•UseNCCinsteadofSSD[Seitz]•UseNCCinsteadofSSD,IgnoreNCCvaluesthreshold[Hernandez&Schmitt03]ThevisibilityproblemWhichpointsarevisibleinwhichimages?UnknownSceneUnknownSceneKnownSceneKnownSceneInverseVisibilityForwardVisibility[Seitz]knownimagesknownsceneVolumetricstereoSceneVolumeSceneVolumeVVVVInputImagesInputImagesInputImagesInputImages(Calibrated)(Calibrated)[Seitz]Goal:Goal:Determineoccupancy,“color”ofpointsinVDetermineoccupancy,“color”ofpointsinVDiscreteformulation:VoxelColoringDiscretizedDiscretizedSceneVolumeSceneVolumeInputImagesInputImagesInputImagesInputImages(Calibrated)(Calibrated)[Seitz]Goal:AssignRGBAvaluestovoxelsinVphoto-consistentwithimagesComplexityandcomputabilityDiscretizedDiscretizedSceneVolumeSceneVolumeNvoxelsNvoxelsCcolorsCcolors33CcolorsCcolorsAllS(CN3)TrueScene[Seitz]AllScenes(CN)Photo-ConsistentScenesIssuesTheoreticalQuestions•Identifyclassofallphoto-consistentscenesypPracticalQuestions•Howdowecomputephoto-consistentmodels?[Seitz]Voxelcoloringsolutions1.C=2(shapefromsilhouettes)•Volumeintersection[Baumgart1974][g]Formoreinfo:Rapidoctreeconstructionfromimagesequences.R.Szeliski,CVGIP:ImageUnderstanding,58(1):23-32,July1993.(thispaperisapparentlynotavailableonline)