DataStage入门培训

DataStage入门培训讲师：邱明伟日期：2010-03-01东南融通版权所有DataStage2AgendaDataStage介绍DataStage开发1.DataStage四个客户端的使用2.DataStage常用组件使用DataStage常用命令练习DataStage介绍DataStage4AscentialPlatformDataStage5WhatisDataStage?DesignjobsforExtraction,Transformation,andLoading(ETL)Idealtoolfordataintegrationprojects–suchas,datawarehouses,datamarts,andsystemmigrationsImport,export,create,andmanagedmetadataforusewithinjobsSchedule,run,andmonitorjobsallwithinDataStageAdministeryourDataStagedevelopmentandexecutionenvironmentsDataStage开发DataStage7DataStageServerandClientsDataStage8DataStageServerandClientsAdministratorAdministersDataStageprojectsandconductshousekeepingontheserverDesignerCreatesDataStagejobsthatarecompiledintoexecutableprogramsDirectorUsedtorunandmonitortheDataStagejobsManagerAllowsyoutoviewandeditthecontentsoftherepositoryDataStageAdministratorDataStage10DataStageAdministratorInDataStagealldevelopmentworkisdonewithinaproject.ProjectsarecreatedduringinstallationandafterinstallationusingAdministrator.Eachprojectisassociatedwithadirectory.Thedirectorystorestheobjects(jobs,metadata,customroutines,etc.)createdintheproject.Beforeyoucanworkinaprojectyoumustattachtoit(openit).YoucansetthedefaultpropertiesofaprojectusingDataStageAdministratorDataStage11DataStageAdministratorUsetheAdministratortospecifygeneralserverdefaults,addanddeleteprojects,andtosetprojectproperties.UsetheAdministratorProjectPropertieswindowto:·SetjobmonitoringlimitsandotherDirectordefaultsontheGeneraltab.·SetusergroupprivilegesonthePermissionstab.·Enableordisableserver-sidetracingontheTracingtab.·SpecifyausernameandpasswordforschedulingjobsontheScheduletab.·SpecifyhashedfilestagereadandwritecachesizesontheTunablestabDataStageManagerDataStage13DataStageManagerDataStageManagermanagestwodifferenttypesofobjects:·Metadatadescribingsourcesandtargets:-CalledtabledefinitionsinManager.Thesearenottobeconfusedwithrelationaltables.DataStagetabledefinitionsareusedtodescribetheformatandcolumndefinitionsofanytypeofsource:sequential,relational,hashedfile,etc.-TabledefinitionscanbecreatedinManagerorDesignerandtheycanalsobeimportedfromthesourcesortargetstheydescribe.DataStage14DataStageManager·DataStagecomponents-EveryobjectinDataStage(jobs,routines,tabledefinitions,etc.)isstoredintheDataStagerepository.Manageristheinterfacetothisrepository.-DataStagecomponents,includingwholeprojects,canbeexportedfromandimportedintoManager.DataStage15DataStageManagerAnyobjectinManagercanbeexportedtoafileCanexportwholeprojectsUseforbackupSometimesusedforversioncontrolCanbeusedtomoveDataStageobjectsfromoneprojecttoanotherUsetoshareDataStagejobsandprojectswithotherdevelopersDataStage16DataStageManagerImportProcedureInManager,click“ImportDataStageComponents”SelectDataStageobjectsforimportDataStage17DataStageManagerExportProcedureInManager,click“ExportDataStageComponents”SelectDataStageobjectsforexportSpecifiedtypeofexport:DSX,XMLSpecifyfilepathonclientmachineDataStageDirectorDataStage19DataStageDirectorCanschedule,validating,andrunjobsCanbeinvokedfromDataStageManagerorDesignerClearjoblogSetDirectoroptionsRowlimitsAbortafterxwarningsDataStage20DirectorLogViewClicktheLogbuttoninthetoolbartoviewthejoblog.Thejoblogrecordseventsthatoccurduringtheexecutionofajob.Theseeventsincludecontrolevents,suchasthestarting,finishing,andabortingofajob;informationalmessages;warningmessages;errormessages;andprogram-generatedmessages.DataStage21DataStageDirectorDataStageDesingerDataStage23WhatIsaJob?ExecutableDataStageprogramCreatedinDataStageDesigner,butcanusecomponentsfromManagerBuiltusingagraphicaluserinterfaceCompilesintoOrchestrateshelllanguage(OSH)DataStage24CreateNewJobSeveraltypesofDataStagejobs:Parallel–thiscoursewillconcentrateonparalleljobs.JobSequence–usedtocreatejobsthatcontrolexecutionofotherjobs.DataStage25CreateNewJobDataStage26ComponentsIntroduceSequentialfile功能特点：适用于一般顺序文件（定长或不定长），可识别文本文件或IBM大机ebcdic文件。使用要点：按照命名规范命名点住文件，双击鼠标，在general说明此文件内容，格式，存储目录等修改文件属性，文件名称，reject方式DataStage27SequentialfileDataStage28Sequentialfile修改文件格式，比如记录结束符是什么，字段分隔符，字符串是用什么区别等DataStage29SequentialfileDataStage30Sequentialfile输入此文件字段内容DataStage31Annotation功能特点：一般用于注释，可利用其背景颜色在job中分颜色区别不同功能块DataStage32AnnotationDataStage33CopyStage功能说明：CopyStage可以有一个输入，多个输出。它可以在输出时改变字段的顺序，但是不能改变字段类型。DataStage34CopyStageDataStage35FilterStage功能说明：FilterStage只有一个输入，可以有多个输出。根据不同的筛选条件，可以将数据输出到不同的outputlinkDataStage36FilterStageDataStage37SortStage功能说明：只能有一个输入及一个输出，按照指定的Key值进行排列。可以选择升序还是降序，是否去除重复的数据等等DataStage38SortStageDataStage39SortStageOption具体说明：AllowDuplicates：是否去除重复数据。为False时，只选取一条数据，当StableSort为True时，选取第一条数据。当SortUnility为UNIX时此选项无效。SortUtility：选择排序时执行应用程序，可以选择DataStage内建的命令或者Unix的Sort命令OutputStatistics：是否输出排序统计信息到job日志StableSort：是否对数据进行二次整理DataStage40SortStageCreateClusterKeyChangeColumn：是否为每条记录创建一个新的字段：clusterKeyChange。当SortKeyMode为Don’tSort(PreviouslySorted)或Don’tSort(PreviouslyGrouped)时，对于第一条记录该字段被设置为1，其余的记录设置为0。CreateKeyChangeColumn：是否为每一条记录创建一个新的字段KeyChangeDataStage41RemoveDuplicat