Realizing distributed TTCN-3 test systems with TCI

RealizingdistributedTTCN-3testsystemswithTCIInaSchieferdecker1,TheofanisVassiliou-Gioles21FraunhoferFOKUS,CompetenceCenterforTesting,InteroperabilityandPerformance,Kaiserin-Augusta-Allee31,D-10589Berlin,Germany,[e-mail]2TestingTechnologiesISTGmbH,OranienburgerStr.65,D-10117Berlin,Germany,[e-mail]efficientload,performance,scalability,interworking,andend-to-endtestsaregainingimportancefortheassessmentofdistributedcommunicatingsystems.TheTestingandTestControlNotationTTCN-3providesconceptsforcomponent-baseddistributedtestsystemsindynamictestconfigurations,wheretestcomponentsmayresideonvariousnetworknodestobeneartheinterfacesofthetestedsystem.TherealizationofexecutableTTCN-3testsonconcretetestplatformsinvolvesTTCN-3compilation/interpretationandadaptationstothetestplatform.TheTTCN-3ControlInterfacesTCIdefineentities,interfaces,typesandoperationsneededtoflexiblymanageanddistributeTTCN-3basedtestsystems.ItcomplementsandcompletestheTTCN-3RuntimeInterfaceTRI.ThispaperdiscussestheunderlyingconceptsofTCIanddemonstratesitsusefortherealizationofadistributedtestfortheSessionInitiationProtocolSIP.1IntroductionTheTestingandTestControlNotationTTCN-3isatestspecificationandimplementationlanguagetodefinetestproceduresforblack-boxtestingofdistributedsystems.TTCN-3allowsaneasyandefficientdescriptionofcomplexdistributedtestbehaviorintermsofsequences,alternatives,loopsandparallelstimuliandresponses.Stimuliandresponsesareexchangedattheinterfacesofthesystemundertest,whicharedefinedasacollectionofportsbeingeithermessage-basedforasynchronouscommunicationorsignature-basedforsynchronouscommunication.Thetestsystemcanuseanynumberoftestcomponentstoperformtestproceduresinparallel.Likewisetotheinterfacesofthesystemundertest,theinterfacesofthetestcomponentsaredescribedasports.ThedevelopmentofTTCN-3wasforcedbykeyplayersofthetelecommunicationindustriesandsciencetogetasingletestnotationfornearlyallblack-boxtestingneeds.Especiallythenewlyintroducedsupportofdynamicdistributedtestsetups,i.e.dynamiccreationandterminationoftestcomponentsincludingdynamicconnectionsbetweentestcomponentsandtothesystemundertest(theSUT),enablesnewapplicationsofTTCNwhilekeepingthematureandstabletestconcepts.TTCN-3testspecificationsarenotonlyabasisforfunctionalandconformancetesting,butalsoforperformance,loadandscalabilitytests.SuchtestsrequirevaryingloadconditionsfortheSUT,whichcanberealizedbyanensembleofparalleltestcomponents.Sincethetestsystemhastobeasperformantasthesystemundertest,anyrealisticloadfortheSUTcanberealizedinadistributedenvironmentonly:theparalleltestcomponenthavetobedistributedandlocatedonremotenodesinanetworkconstitutingadistributedtestsystem.OneessentialbenefitofTTCN-3isthatitenablesthespecificationoftestsinaplatformindependentmanner.Hence,TTCN-3providestheconceptsoftestcomponents,theircreation,theircommunicationlinkstoeachotherandtotheSUT,theirexecutionandterminationassuchonabstractlevelonly.MeanstocontrolthedistributedexecutionoftestcomponentsandcoordinationbetweenthemareoutsideTTCN-3.However,theapplicationofexecutableteststoaSUTwithintestcampaignsrequirestherealizationandimplementationofsuchdistributedtestsystemsinanetworkoftestnodes–atbestinawell-definedmannertoenableastandardizedadaptationforthemanagement,componenthandling,communicationandloggingbetweendistributedtestnodes.Anotheraspectofthisadaptationistheabilitytoreuseexternalencoders/decoders,whicharealsooutsideTTCN-3andjustreferencedwithinatestspecification.Well-definedinterfacesasasetofoperationsindependentofthetarget,i.e.independentoftheSUT,processingplatform,implementationlanguage,etcwillenablethatcodefromanyTTCN-3compilerorinterpretersupportingandusingtheseinterfacescanbeexecutedonanytestplatform/testdevice,whichsupportsandusestheseinterfaces.AfirststeptowardsthiscodeindependencewasdonewiththeTTCN-3RuntimeInterfaceTRI24:TRIprovidesaninterfacetoadaptaTTCN-3testsystemtotheSUTbyprovidingmeanstoadaptthecommunicationwiththeSUTaswellastoadaptthetimerhandling.Assuch,TRIdefinesalocaladaptationtotheSUTonly.Theaspectsoftestmanagement,componenthandling(bothonlocalandremotenodes)aswellasthetypeandvaluehandlinghavenotbeenconsideredbyTRI.Theseaspectscanbesummarizedasbeingtheadaptationtothetestsystembeingeitherasingletestdeviceoratestplatformconsistingofseveraltestnodes1.TRIhastobesupplementedbyinterfacestoenableawell-definedadaptationtothetestplatform.TheseinterfacesarecalledtheTTCN-3ControlInterfacesTCI.TCItogetherwithTRIprovideacompletesolutionforawell-definedadaptationtothetestsystemandtotheSUTprovidingmaximalflexibilityinrealizingTTCN-3testsystems.Onlyrecently–attheETSIMTSmeeting,October2002–theimportanceofTRIandTCIforTTCN-3hasbeenreflected:TRI4andTCI5weremadeintegralpartsofthemulti-partstandardforTTCN-3.TCIiscurrentlydevelopedatETSIandisexpectedtobecometogetherwithTRIthefuturestandardinterfacesetforallTTCN-3testsystemimplementations.ItconsiderspreviousworkinGCI8ontypeandvalueinterfacesandinTSP1+7anditsimplementation6ondistributedtestsys