A multimedia programming model based on timed conc

—1—AMultimediaProgrammingModelBasedonTimedConcurrentConstraintProgrammingGeorgeA.PapadopoulosMultimediaResearch&DevelopmentLaboratoryDepartmentofComputerScienceUniversityofCyprus75KallipoleosStr.Nicosia,CY-1678,P.O.Box537CYPRUSe-mail:[e-mail]velopmentofmultimediaprogrammingframeworksbasedonthedeclarativelogicprogrammingsettingandinparticulartheframeworkofobject-orientedtimedconcurrentconstraintprogramming(OO-TCCP).Thereal-timeextensionsthathavebeenproposedfortheconcurrentconstraintprogrammingframeworkarecoupledwiththeobject-orientedandinheritancemechanismsthathavebeendevelopedforlogicprogramsyieldinganintegrateddeclarativeenvironmentformultimediaobjectsmodelling,compositionandsynchronisation.TheexpressivenessandabilityofOO-TCCPtoactasabasisforbuildinghigh-level,user-friendlyauthoringenvironmentsisillustratedbypresentingasimpleobjectmodelformultimediacompositionandsynchronisationfollowedbyananalysisonhowitcanbeusedtomodelthetemporalbehaviourandrelationshipsofmultimediaobjects.Wethendefinea“universal”setofmultimediaprogrammingprimitivesandweshowhowtheycanbeimplementedinOO-TCCP.Finally,weargueaboutthebenefitsofourapproachwrtbotheasinessofprogrammingandimplementationoftheproposedframework.Tothebestofourknowledgethisisthefirsttimelogicprogramming(intheformofconcurrentconstraintAMultimediaProgrammingModelBasedonTCCPGeorgeA.Papadopoulos—2—programmingorotherwise)isusedasabasisfordesigningandimplementingmultimediaprogrammingframeworks.Keywords:MultimediaProgramming;DeclarativeProgramming;ModellingandSynchronisationofMultimediaObjects;(Applicationsof)ConcurrentConstraintProgramming.1.IntroductionThedevelopmentofmultimediaapplicationsinvolvesthemanagingof,often,complexissuessuchasprogrammingthebehaviourofavarietyofmediaobjects(stillandmotionvideoframes,audiosamples,text),expressingthespatialandtemporalrelationshipsbetweenandwithinmultimediaobjects,andusingspecialhardware.Thisleadstoanumberofproblems,amongothersthelackofknowledgeregardingnovelprogrammingconceptsrequiredinthehandlingof,say,audiorecordingorvideoproduction,portabilityissues,etc.Awayofhandlingtheseproblemsisthedevelopmentofhigh-leveluser-friendlymultimediaprogrammingframeworksthathideawayhardwaredependenciesandmediacharacteristicsandprovideabstractionssuitableforexpressingeasilythe,often,complexmodellingandsynchronisationprogrammingparadigms.Inaddition,themodelusedmustpreferablybeamenabletoformalanalysiswhosepurposeistoreasonaboutthe“correctness”ofaprogram;inthiscasebycorrectnesswerefertotheabilityofthemodeltoprovethattheinteractionbetweenthemediacomponentsofanapplication(suchasinthecaseofsynchronisation)isasexpected.Suchaformalreasoningcapabilityisveryusefulin,say,derivingthequalityofservice(QOS)requirementsofsomeapplicationframework.Thepurposeofthispaperistoaddresstheaboveissues;however,unlikemostoftheotherapproachesthatareprimarilybasedonimperativeprogrammingtechniquesandtheuseoflanguagessuchasC++([21,22])orreal-timeones([2])suchasESTEREL([8,15]),ourmodelisbasedondeclarativeprogrammingand,inparticular,thatofconcurrentconstraintprogramming.Moretothepoint,weshowhowthetimedversion([18])ofconcurrentconstraintprogramming([17])canbeusedtomodelandsupportthesynchronisationAMultimediaProgrammingModelBasedonTCCPGeorgeA.Papadopoulos—3—requirementsofmultimediaobjects;thisisthencombinedwithalreadyexistingtechniquessupportingobject-orientedprogramming([4,6,12]).Thisworkshouldbeseenasasteptowardsthedesignandimplementationofahigh-levelmultimediaprogrammingframeworkbasedondeclarativeprogramming.Tothebestofourknowledgethisisthefirsttimethatanattemptismadetousedeclarativeprogramming(and,inparticular,concurrentconstraintprogramming)inthedevelopmentofmultimediaprogrammingframeworks.Therestofthepaperisorganisedasfollows:Thenexttwosectionsdiscusssomeoftheissuesrelatedtomultimediaobjectmodellingandsynchronisationandargueaboutthebenefitsofourapproach.ThisisfollowedbyabriefintroductiontoOO-TCCP,acombinationoftimedconcurrentconstraintprogramming([18])withtheobject-oriented([4,12])andinheritancefacilities([6])thathavebeendevelopedforlogicprograms.Thefifthsectionpresentsa“universal”setofprimitivesformultimediaprogramminganddiscussestheirimplementationinOO-TCCP,thusshowingthecapabilitiesofthemodelinhandlingtherequirementsimposedbymultimediaobjectmodellingandsynchronisation.Thefollowingsectiondiscussesbrieflyimplementationtechniquesfortheproposedframework.Thelastsectionconcludesthepaperwithadiscussionofcurrentandfuturework.2.MultimediaObjectModellingandCompositionBymodellinginthecontextofprogrammingmultimediaapplicationsweusuallyrefertotheneedfordevelopingsuitableabstractions,abletohideawayconceptsparticulartothebehaviourandcharacteristicsofamediaobject,withwhichallprogrammersarenotnecessarilyfamiliar.Suchcomplexissuesinclude,amongothers,dataencoding,compressiontechniques,qualityfactors,timing,etc.Inaddition,theseabstractionsattempttoproducemodelswhichareplatform-independent.Inourresearchprojectweareparticularlyconcernedwiththesocalledtime-base