A Survey of Augmented Reality

ASurveyofAugmentedRealityRonaldT.AzumaHughesResearchLaboratories3011MalibuCanyonRoad,MSRL96Malibu,[e-mail](310)317-51511.Introduction1.1GoalsThispapersurveysthecurrentstate-of-the-artinAugmentedReality.ItdescribesworkperformedatmanydifferentsitesandexplainstheissuesandproblemsencounteredwhenbuildingAugmentedRealitysystems.Itsummarizesthetradeoffsandapproachestakensofartoovercometheseproblemsandspeculatesonfuturedirectionsthatdeserveexploration.Thispaperdoesnotpresentnewresearchresults.Thecontributioncomesfromconsolidatingexistinginformationfrommanysourcesandpublishinganextensivebibliographyofpapersinthisfield.Whileseveralotherintroductorypapershavebeenwrittenonthissubject[Barfield95][Bowskill95][Caudell94][Drascic93b][Feiner94a][Feiner94b][Milgram94b][Rolland94],thissurveyismorecomprehensiveandup-to-date.Foranyoneinterestedinstartingresearchinthisarea,thissurveyshouldprovideagoodstartingpoint.Section1describeswhatAugmentedRealityisandthemotivationsfordevelopingthistechnology.FourclassesofpotentialapplicationsthathavebeenexploredaredescribedinSection2.ThenSection3discussestheissuesinvolvedinbuildinganAugmentedRealitysystem.Currently,twoofthebiggestproblemsareinregistrationandsensing,sothosearethesubjectsofSections4and5.Finally,Section6describessomeareasthatrequirefurtherworkandresearch.1.2DefinitionAugmentedReality(AR)isavariationofVirtualEnvironments(VE),orVirtualRealityasitismorecommonlycalled.VEtechnologiescompletelyimmerseauserinsideasyntheticenvironment.Whileimmersed,theusercannotseetherealworldaroundhim.Incontrast,ARallowstheusertoseetherealworld,withvirtualobjectssuperimposeduponorcompositedwiththerealworld.Therefore,ARsupplementsreality,ratherthancompletelyreplacingit.Ideally,itwouldappeartotheuserthatthevirtualandrealobjectscoexistedinthesamespace,similartotheeffectsachievedinthefilmWhoFramedRogerRabbit?Figure1showsanexampleofwhatthismightlooklike.Itshowsarealdeskwitharealphone.Insidethisroomarealsoavirtuallampandtwovirtualchairs.Notethattheobjectsare2combinedin3-D,sothatthevirtuallampcoverstherealtable,andtherealtablecoverspartsofthetwovirtualchairs.ARcanbethoughtofasthemiddlegroundbetweenVE(completelysynthetic)andtelepresence(completelyreal)[Milgram94a][Milgram94b].Figure1:Realdeskwithvirtuallampandtwovirtualchairs.(CourtesyDavidBreen,©1994ECRC.)SomeresearchersfeelthatARrequirestheuseofHead-MountedDisplays(HMDs).Forthepurposesofthispaper,IdefineARnotbyspecifictechnologiesbutbythreedesiredcharacteristics:1)Combinesrealandvirtual2)Interactiveinrealtime3)Registeredin3-DThisdefinitionallowsothertechnologiesbesidesHMDs,whileretainingtheessentialcomponentsofAR.Forexample,itdoesnotincludefilmor2-Doverlays.FilmslikeJurassicParkandTheMaskfeaturephotorealisticvirtualobjectsseamlesslyblendedwitharealenvironmentin3-D,buttheyarenotinteractivemedia.2-Dvirtualoverlaysontopoflivevideocanbedoneatinteractiverates,buttheoverlaysarenotcombinedwiththerealworldin3-D.However,thisdefinitiondoesallowmonitor-basedinterfaces,monocularsystems,see-throughHMDs,andvariouscombiningtechnologies.PotentialsystemconfigurationsarediscussedfurtherinSection3.1.3MotivationWhywouldsomeonebeinterestedinAugmentedReality?Whytrytocombinerealandvirtualobjectsin3-D?Basically,AugmentedRealityenhancesauser'sperceptionofandinteractionwiththerealworld.Thevirtualobjectsdisplayinformationthattheusercannotdirectlydetectwithhisownsenses.Sincemanyoperationsrequiremanipulationormodificationofarealenvironment,thevirtualobjectsprovideinformationthathelpsauserperformareal-worldtask.ARisaspecificexampleofwhatFredBrookscallsIntelligenceAmplification(IA):usingthecomputerasatoolthatmakesataskeasierforahumantoperform.AtleastsixclassesofpotentialARapplicationshavebeenexplored:medicalvisualization,maintenanceandrepair,annotation,robotpathplanning,entertainment,andmilitaryaircraftnavigationandtargeting.Thenextsectiondescribesworkthat3hasbeendoneineacharea.Whilethesedonotcovereverypotentialapplicationareaofthistechnology,theydocovertheareasexploredsofar.2.Applications2.1MedicalDoctorscoulduseAugmentedRealityasavisualizationandtrainingaidforsurgery.Itmaybepossibletocollect3-Ddatasetsofapatientinrealtime,usingnon-invasivesensorslikeMagneticResonanceImaging(MRI),ComputedTomographyscans(CT),orultrasoundimaging.Thesedatasetscouldthenberenderedandcombinedinrealtimewithaviewoftherealpatient.Ineffect,thiswouldgiveadoctorX-rayvisioninsideapatient.Thiswouldbeveryusefulduringminimally-invasivesurgery,whichreducesthetraumaofanoperationbyusingsmallincisionsornoincisionsatall.Aproblemwithminimally-invasivetechniquesisthattheyreducethedoctor'sabilitytoseeinsidethepatient,makingsurgerymoredifficult.ARtechnologycouldprovideaninternalviewwithouttheneedforlargerincisions.ARmightalsobehelpfulforgeneralmedicalvisualizationtasksinthesurgicalroom.SurgeonscandetectsomefeatureswiththenakedeyethattheycannotseeinMRIorCTscans,andvice-versa.ARwouldgivesurgeonsaccesstobothtypesofdatasimultaneously.Thismightalsoguideprecisiontasks,suchasdisplayingwheretodrillaholeintotheskull