Software Development People, Process, Technology

1/25SoftwareDevelopment:People,Process,TechnologyArashKhodabandeh,PaoloPalazziCERN-ECPDivision-ProgrammingTechniquesGroupAbstractTheproductionofsoftwareisalabourintensiveactivity.GiventhescaleofsoftwareprojectsrelatedtocurrentandfutureHEPexperiments,itisworthtryingtoimprovetheknowledgeofthePEOPLEinvolved,theorganizationofthesoftwaredevelopmentPROCESSandtheTECHNOLOGYusedinthevariousaspectsofthisactivity.Thegoalisbettersystemsatlowercostandhappierusersofthesoftware.Thisdocumentcanbeaccessedthrough=:[[e-mail]]and[[e-mail]]PeopleProcessTechnologyPublishedintheproceedingsofthe1994CERNSchoolofComputing,Sopron,Hungary,andasCERNECPReport95/5.2/251IntroductionSoftwareisexpensive.Buildingitandmaintainingitarelabourintensiveactivities,butdelaysindeliverycanbeverycostlyandanyundetectedproblemsmaycauselossofperformanceandfrustrateusers.ThisalsoappliestoHighEnergyPhysics,eventhoughphysicistshavebeenattheforefrontofcomputingintheearlydaysand,stillarenowinseveralareas.Todaytheuseofcomputersforphysicsismuchmorepervasive,andthesize,complexityandlifetimeofthesoftwarearemuchlarger.Formostofthephysicistsandengineersworkinginexperiments,thebusinessisphysicsorengineering,notcomputing,andmanytechnicalmanagersarenotcomputerliterateenough.Theuseofsoftwareisgrowing,andoftenitseemsthattheprocessofproducingitisoutofcontrol.Thereisthefeelingthatnotenoughpeoplearoundhavetheadequatesoftwareengineeringbackgroundtoimprovethesituation.Thesoftwaredevelopmentteamsaroundexperimentsaretraditionallyveryindependent.Aspectsofthesoftwaredevelopmentculturearelocaltovariouscenters,althoughthesameapplicationpackagesarewidelyusedinthecommunityforsomeaspectsofthework.Threekeycomponentsdriveallimprovementsinsoftwaredevelopmentproductivity:thePEOPLEinvolved,theorganizationofthedevelopmentPROCESS,andtheTECHNOLOGYused.Talentedpeopleareaveryimportantelementinanysoftwareorganization,buteventhebestprofessionalsneedanorganizedenvironmentinwhichtodocooperativework.Thesameappliestoadvancedtechnology:withoutanorganizationalframework,itcannotbefullyeffective,andthereisevidencethatgoingfornewtechnologyinsteadofimprovingtheprocesscanmakethingsworst.2PeopleItisobviousthatanysoftwaresystemisdependentonthequalityofthepeoplewhobuildit.PeopleworkinginParticlePhysicshavebeenselectedfortheirintellectualabilities,sonobodycouldarguethatoursoftwareproblemsareduetothelackoftalentedpeople.Ourpeoplehaveproducedmanyexcellentprogramsandpackages,andsuperiorprogramsderivefromsuperiordesign.Superiordesign,ontheotherhand,isalwaystheworkofpeoplewhounderstandtheapplicationdomain.Thisisanecessaryconditionofsuccess,anybodywhodoesnotunderstandstatisticscannotbuildagoodstatisticalsystem.Ofcoursehemayprofitfromhelpbythosewhoarefamiliarwithtechnicalaspectsofsoftwareproduction.Wecanconfidentlyassumethatinourfieldwehavebrightpeople,whoknowtheirapplicationdomain.Letusseewhattheydointhescientificdiscoveryprocess,andwhatistheirinvolvementwithsoftware.2.1SoftwareforHEPProgressintheunderstandingofnatureistheresultofthecombinedworkoftheoreticalandexperimentalphysicists.Theoreticiansputforwardnewtheoriescompatiblewithexistingexperimentalresults,whileexperimentalistsmeasurenewphysicalphenomenatodisprovethetheories.Tomeasureaphysicalprocess,largegroupsofscientists,engineersandtechniciansdesign,buildandoperateexperimentsandanalyzestatisticallythelargeamountofdataproduced.Theproductionofsoftwareisinvolvedatallstagesoftheprocess.Scientistscanbemoreproductiveinagivendomainoraspectofthedataprocessingbyusingspecializedsystemsorpackages.Suchsoftwareprovidestheappropriateabstractionsthroughauser-friendlyinterface.ExamplesofthatareMathematica3/25andsimilarsystems,CAVIAR,thehistogrammingpackageHBOOKanditsmorerecentinteractiveversionPAW,andsoon.2.2RolesofpeopleScientistswhoworkclosetothephysicsaspectsofthingswilltendtousesuchspecializedsystems.Theseproductswillallowthemtoworkeffectivelyattherightlevelofabstractioninaneasy-to-usecontext.Stillthereisaneedtoproducesuchprograms,andthisistheworkofaminorityofphysicists.Theyareexpertsinagiventechnicaldomain,andatthesametimeactasspecializeddevelopersandbuildsystemsfortheotherstouse.Werefertotheauthorsofwidelyusedscientificsoftware,aswellastothephysicists/programmersofacollaboration.Therepeopletakeresponsibilityforproducingthedataacquisition,simulation,reconstructionanddataanalysisprogramsusedbyalltheothers.Inordertobuildsuchnon-trivialsoftwaresystemswithalifetimeofseveralyearsordecades,specializedknowledgeofsoftwaretechnologyandadisciplinedorganizationarebothessential,andafewsoftwareengineeringspecialistsneedtobeinvolved,whocanadviseonappropriatechoicesoforganizationalpractices,methodsandtools.Backingfromthemanagementisessentialinenforcingsuchpractices.2.3Openproblems:specialization,training,recognitionSoftwareforparticlephysicsisacomplexmatter,wherespecializationisimportant,bothintheapplicationdomain,andinaspectsofsoftwareengineering.Thevariouspeopleinvolvedactincomplementary