(更新1)基于Hadoop的分布式文件系统的研究

南阳理工学院云计算导论大作业学院(系)：软件学院专业：软件工程学生：李元池指导教师：陈可完成日期2016年04月基于Hadoop的分布式文件系统的研究StudyofthedistributedfilesystembasedonHadoop基于Hadoop的分布式文件系统的研究南阳理工学院云计算导论大作业基于Hadoop的分布式文件系统的研究StudyofthedistributedfilesystembasedonHadoop学院(系)：软件学院专业：软件工程学生姓名：李元池学号：1415925619指导教师(职称)：陈可讲师评阅教师：陈可完成日期：2016年04月南阳理工学院NanyangInstituteofTechnology基于Hadoop的分布式文件系统的研究基于Hadoop的分布式文件系统的研究云计算4班李元池[摘要]随着互联网的普及和快速发展，用户数据出现了迅速的增长。传统的通过网络通信访问非本地数据在面对当前大规模数据存储需求，在数据容量、可靠性、安全性、高可扩展等方面已难胜任。针对上述问题，提出一种新的大规模数据存储系统，即基于HDFS(HadoopDistributedFileSystem)的分布式存储系统，通过充分利用分布式存储的技术优势，以满足当前大规模数据存储和管理的要求。HDFS作为主流的分布式存储系统，因为其高可靠、高性能存储服务而被广泛使用。利用系统中的Master对所有的DataNode进行集中管理、监控和维护，同时，提供Client让用户通过其与整个系统交互，完成用户需要的系列功能，如包括大规模数据的读写、删改等。[关键词]Hadoop分布式系统HDFS文件系统分布式数据存储基于Hadoop的分布式文件系统的研究基于Hadoop的分布式文件系统的研究StudyofthedistributedfilesystembasedonHadoopCloudcomputingclass4LiYuanChiAbstract:WithAbstract:thepopularizationandrapiddevelopmentoftheInternet,usershaveexperiencedrapidgrowthindata.Traditionalcommunicationaccessthroughthenetworkofnon-localdatainthefaceoflarge-scaledatastoragerequirementscurrentlyindatacapacity,reliability,security,highscalability,etc.havebeendifficulttodothejob.Inresponsetotheseproblems,anewlarge-scaledatastoragesystems,namelyHDFS(HadoopDistributedFileSystem)-baseddistributedstoragesystem,bymakingfulluseoftheadvantagesofdistributedstoragetechnologytomeetthecurrentlarge-scaledatastorageandmanagementrequirements.Abstract:Asamainstreamamongdistributedstoragesystemsduetoitshighreliabilityandsatisfyingperformanceinstorageservice,HDFSiswidelyused.AndwecanmakeuseofMaster,thusconductingcentralizedmanagement,monitoringandmaintenanceoveralltheDataNodeinthesystem.Meanwhile,weprovidetheuserswithclienttoallowthemtointeractwiththewholesystemandcompeletetheneedysystematicalfunctions,includingthereading,writinganddeletingoflarge-scaledata.Abstract:HadoopHDFSDistributedDataStorage基于Hadoop的分布式文件系统的研究目录1绪论........................................................................................................................................11.1研究背景及意义..........................................................................................................11.2国内外研究现状..........................................................................................................21.3论文结构......................................................................................................................22相关键技术分析....................................................................................................................32.1Hadoop介绍.................................................................................................................32.2Hadoop分布式文件系统HDFS..................................................................................32.2.1HDFS相关概念..................................................................................................42.3本章小结......................................................................................................................43实验环境构建........................................................................................................................43.1运行环境......................................................................................................................43.2VMwareWorkstation12Pro的安装............................................................................53.2.1准备条件............................................................................................................53.2.2安装截图............................................................................................................53.3基于Ubuntu15.10的Hadoop集群安装与配置........................................................63.3.1安装JDK............................................................................................................73.3.2安装SSH免密登录...........................................................................................83.3.3进行Hadoop集群完全分布式的安装配置.....................................................93.4测试是否安装成功....................................................................................................123.5本章小结....................................................................................................................144Hadoop中HDFS的应用.....................................................................................................144.1HDFS体系架构.........................................................................................................144.2基本操作....................................................................................................................154.3HDFS的文件存储原理..............................................................................................164.4基本文件命令............................................................................................................184.5本章小结....................................................................................................................215结束语...........................................................................................