1 KANTRA Human-Machine Interaction for Intelligent

1KANTRAHuman-MachineInteractionforIntelligentRobotsusingNaturalLanguageKANTRAHuman-MachineInteractionforIntelligentRobotsusingNaturalLanguageT.C.Lueth,Th.Laengle,*G.Herzog,*E.Stopp,U.RemboldInstituteforReal-TimeComputerSystemsandRoboticsUniversityofKarlsruhe,D-76128KarlsruheF.R.Germany,email:t.lueth@*ProjectVITRA-FB14InformatikUniversityoftheSaarland,D-66041SaarbrueckenF.R.Germany,email:[e-mail]:Inthispaper,anewnaturallanguageinterfaceispresentedthatcanbeappliedtomaketheuseofintelligentrobotsmoreflexible.Thisinterfacewasdevelopedfortheautonomousmobiletwo-armrobotKAMRO,whichusesseveralcamerasystemstogenerateanenvironmentmodelandtoperformassemblytasks.Afundamentalrequirementinhuman-machineinteractionforintelligentrobotsistheabilitytorefertoobjectsintherobot'senvironment.Hence,theinterfaceandtheintelligentsystemneedsimilarenvironmentmodelsanditisnecessarytoprovidecurrentsensorinformation.Additionalflexibilitycanbeachievedbyintegratingtheman-machineinterfaceintothecontrolarchitectureoftherobotandtogiveitaccesstoallinternalinformationandtothemodelsthattherobotusesforanautonomousbehaviour.Inordertofullyexploitthecapabilitiesofanaturallanguageaccess,wefavouradialogue-basedapproach,i.e.,fortheinterface,KANTRA,presentedhere,thehuman-machineinteractionisnotrestrictedtounidirectionalcommunication.1.INTRODUCTIONThefutureuseofintelligentrobots[1]inmanufacturingorasservicerobotsfordifferentapplicationswillleadtohighdemandsforman-machineinteractionandcommunication.Themorerobotsworkautonomouslythemorecooperationofman-machineandmachine-machine[2]willbenecessary.Distributedmulti-robotsystemsshowthewayfutureintelligentmanufacturingsystemscanoperateindependently[3,4].Insuchanenvironment,allmachinescan'talwaysbeprogrammedorcontrolledinthesameway.Onthataccount,naturallanguageasacommunicationmediumforhumansisanefficientmeanstomakeatechnicalsystemeasieraccessibletoitsusers[5].Apracticaladvantageofanaturallanguageaccessisthepossibilitytoconveyinformationinavaryingdegreeofcondensationandtocommunicateondifferentlevelsofabstractioninanapplication-specificway.Inthisarticle,wewanttoreportaboutthejointeffortsoftheUniversityofKarlsruheandtheUniversityoftheSaarlandatprovidinganaturallanguageaccesstotheautonomousmobiletwo-armrobotKAMROwhichisbeingdevelopedatIPR.Inthesecondsection,thestateoftheartinnaturallanguageaccesstointelligenttechnicalsystemsispresented.Insectionthree,ourmobilerobotKAMROisbrieflydescribed.Insectionfourandfive,themainaspectsofnaturallanguageaccessaresummarized,andinsectionsix,ourconceptionforadialogue-basednaturallanguageaccesssystemispresented.Thearticleendswithanevaluationandourconclusionforfuturework.2.STATEOFTHEARTAlthoughnaturallanguageprocessingandroboticsconstitutetwomajorareasofAI,theyhavebeenstudiedratherindependently.Onlyafewworksareconcernedwithnaturallanguageaccessforhuman-machine-interactionandcommunication.Sondheimer[6]focusesontheproblemofspatialreferenceinnaturallanguagemachinecontrol.ThewellknownSHAKEYsystem[7],amobilerobotwithoutmanipulators,isabletounderstandsimplecommandsgiveninnaturallanguage.Theworkdescribedin[8]concentratesonlanguage-aidedinstructionforteleoperationalcontrol.Specificwordscanbeutilizedtosimplifythespecificationofteleoperationalfunctionsfortheinstructionofaremoterobotsystem.Torrance[9]presentsanaturallanguageinterfaceforanavigatingindooroffice-basedmobilerobot.Inadditiontogivingcommandsandaskingquestionsabouttherobot'splanstheusercanassociatearbitrarynameswithspecificlocationsintheenvironment.Sometheoreticalaspectsofnaturallanguagecommunicationwithrobotsystemsfromtheperspectiveofcomputerlinguisticsarediscussedin[10].Otherapproacheshavebeenconcernedwithnaturallanguagecontrolofautonomousagentswithinsimulated2Dor3Denvironments[11-13].Onesalientaspectfornaturallanguageaccesstorobotsystemsistherelationshipbetweensensoryinformationandverbaldescriptions.Suchissueshavealreadybeeninvestigatedinthefieldofintegratednaturallanguageandvisionprocessing[14-17].3.THEINTELLIGENTMOBILEROBOTKAMROHigherintelligenceandgreaterautonomyofmoreadvancedrobotsystemsincreasetherequirementsforthedesignofaflexibleinterfacetocontrolthesystemondifferentlevelsofabstraction.IntheKAMRO(KarlsruheAutonomousMobileRobot)project,forexample,anautonomousmobilerobot(Fig.1)forassemblytasksisbeingdevelopedwhichalsohasthecapabilityofrecoveringfromerrorsituations[18].TheautonomousmobilerobotKAMROisatwo-armrobot-systemthatconsistsofamobileplatformwithanomnidirectionaldrivesystem,twoPuma260manipulators,anddifferentsensorsfornavigation,dockingandmanipulation.Fig.1:ThemobilerobotKAMROKAMROiscapableofperformingassemblytasks(Fig.2)autonomously.Thetasksorrobotoperationscanbedescribedondifferentlevels:assemblyprecedencegraphs,implicitelementaryoperations(pick,place)andexplicitelementaryoperations(grasp,transfer,finemotion,join,exchange,etc.).Givenacomplextask,itistransformedbythecontrolarchitecture(Fig.3)fromassemblyprecedencegraphleveltoexplicitelementaryoperat