A Distribution Network for Free Software

ADistributionNetworkforFreeSoftwareArnoBakkerMaartenvanSteenAndrewS.TanenbaumInternalreportIR-485February2001AbstractTheGlobeDistributionNetwork(GDN)isanapplicationfortheefficient,worldwidedistri-butionoffreelyredistributablesoftwarepackages.Distributionismadeefficientbyencapsulatingthesoftwareintospecialdistributedobjectswhichefficientlyreplicatethemselvesneartothedownloadingclients.TheGlobeDistributionNetworktakesanovel,optimisticapproachtostoptheillegaldistributionofcopyrightedandillicitmaterialviathenetwork.Insteadofhavingmod-eratorscheckthesoftwarearchivesatuploadtime,illegalcontentisremovedanditsuploader’saccesstothenetworkpermanentlyrevokedonlywhenthecontentisdiscovered.AnimportantfeatureoftheGDNisthatthedistributedobjectscontainingthesoftwarecanrunonuntrustwor-thyservers.Byexploitingthereplicationofthesoftwareandusingfault-tolerantserversoftware,theGlobeDistributionNetworkachieveshighavailability.AfirstversionoftheGDNhasbeenimplementedandhasbeenrunningsinceOctober2000acrossfourEuropeansites.ThisarticledescribesthedesignandimplementationoftheGDN.Keywords:softwaredistribution;wide-areanetworks;distributedobjects;traceablecontentvrijeUniversiteitFacultyofMathematicsandComputerScienceINTRODUCTIONDevelopingalargeInternetapplicationisadifficulttaskduetothecomplexnonfunctionalaspectsthathavetobetakenintoaccount.Adeveloperhastodealwithapotentiallylargenumberofusers,highcommunicationdelays,securitythreats,andmachineandnetworkfailures.Thekeytomakinglarge-scaleapplicationdevelopmenteasieristhereforeprovidingthedeveloperwiththemeansfordealingwiththesecomplexaspects.Currentmiddlewareplatforms,suchasCORBAandDCOM,however,donotprovideadequatesupportinthisarea,astheyaremainlyaimedatlocalareanetworks.Wearedesigningandbuildinganewmiddlewareplatformthatwillprovidethedeveloperwiththesupportneededtobuildworldwidedistributedapplicationsmoreeasily.ThismiddlewareplatformiscalledGlobe[1].TodemonstratethefeasibilityofourideasandthedesignoftheGlobemiddlewarewehavebeenbuildinganewlarge-scaleInternetapplication,calledtheGlobeDistributionNetwork.Thisarticledescribesthedesignandimplementationofthisapplication.TheGlobeDistributionNetwork,orGDNforshort,isanapplicationfortheefficient,world-widedistributionoffreelyredistributablesoftwarepackages,suchastheGNUCcompiler,theGIMPgraphicspackage,Linuxdistributions,andshareware.Efficiencyisachievedbyefficientlyreplicatingthesoftwareneartotheusers.TheGDNdoesnotrequireservershostingreplicastobetrustworthy.Theservercapacityrequiredtohostthereplicasofthesoftwarecanthereforebedonatedbyuntrustedvolunteers.ToprotectthesevolunteersagainstlegalactiontheGDNtakesspecialmeasurestopre-ventthedistributionofillicitcontent.TheGDNhashighavailabilityandwell-definedsemanticswhenfailurescannolongerbemasked.Wechosethedistributionoffreelyredistributablesoftware(henceforthfreesoftware)asanex-ampleapplicationforanumberofreasons.Themostimportantreasonisthattheapplicationitselfhasmanyinterestingaspects.Manypeopleareinterestedinfreesoftware,andmanypeoplearecre-atingfreesoftware,resultinginanapplicationthatislargebothintermsofnumbersofusersandintheamountofdatathatneedstobehandled.Theapplicationalsohasinterestingsecurityaspects.UnauthorizedmodificationofthesoftwarebeingdistributedmustbeimpossibleandneithershouldmaliciouspersonsbeabletousetheGDNtoillegallydistributecopyrightedorillicitmaterial.WhatmakesthesecurityaspectsparticularlyinterestingisourintentiontolettheGDNusespareserverca-pacityprovidedbyanyonewhowishestocontribute,implyingthatthemajorityofmachinesusedarenottobetrusted.Thisdesigngoalalsomakesmakingtheapplicationfaulttolerantmorechallenging.AsecondreasonforchoosingsoftwaredistributionisthatthecurrentInternetapplicationsfordistributinginformationareinneedofanupdate.FTPandHTTPhaveproventoscalequitewell,butreplicationandsecurityhavebeenaddedonto,insteadofintegratedintotheapplications.Asaresult,alotofthingsinparticularwithrespecttoreplicationstillhavetobedonebytheuser.Theseincludefindingoutwhichmirrorsitesexist,dealingwithsitefailuresandhandlinginconsistenciesbetweenmirrors(causedbytheperiodicpullmodelappliedinmanymirroringsolutions).TheGlobeDistributionNetworkprovidesanintegratedsolutionwherefailuresonlyveryrarelyrequirehumanintervention.Theremainderofthearticleisstructuredasfollows.WestartwithadescriptionofthebasicarchitectureoftheGlobeDistributionNetworkandprovidethenecessarybackgroundontheGlobemiddlewareplatform.Thisdescriptionwillexplainthebasicoperationoftheapplication.Thenexttwosectionspresentoursecurityandfaulttolerancemeasures,respectively.Wethencontinuewithadescriptionofourcurrentimplementationandconcludewithfutureenhancementstothedesignpresentedhere.1Server2ObjectServer1Server3ObjectGDNNISGIDSGLSGNSObjectClientFigure1:ThebasicarchitectureoftheGlobeDistributionNetwork(squaresrepresentprocesses,ovalsrepresent(distributed)services,arrowsrepresentcommunication).Clientsdownloadsoftwarepackagesfromacollectionofobjectservers.BeingaGlobeapplication,theGDNdependsonthethreestandardGlobemiddlewareservices:theGlobeNamingServi