Greenplum的安装

Greenplum的安装Geenplum是一款基于postgresql开发的mpp数据库，适合于数据仓库数据存储，是对hadoop无法有效处理结构化数据的一个很好的补充。五年前曾经使用过greenplum，那时的版本还是3.6，现在已经更新到4.3.5了，有消息称今年将开源。和以前的相比，安装方式也有了一些小变化，简记如下，如要更详细的说明，还是得看官方文档。一、测试环境CentOS6.58台虚机Greenplum4.3.5二、环境准备1.修改系统参数编辑/etc/sysctl.conf，以下是最小配置kernel.shmmax=500000000kernel.shmmni=4096kernel.shmall=4000000000kernel.sem=2505120001002048kernel.sysrq=1kernel.core_uses_pid=1kernel.msgmnb=65536kernel.msgmax=65536kernel.msgmni=2048net.ipv4.tcp_syncookies=1net.ipv4.ip_forward=0net.ipv4.conf.default.accept_source_route=0net.ipv4.tcp_tw_recycle=1net.ipv4.tcp_max_syn_backlog=4096net.ipv4.conf.all.arp_filter=1net.ipv4.ip_local_port_range=102565535net.core.netdev_max_backlog=10000net.core.rmem_max=2097152net.core.wmem_max=2097152vm.overcommit_memory=22.修改文件打开数等限制编辑/etc/security/limits.conf*softnofile65536*hardnofile65536*softnproc131072*hardnproc131072编辑/etc/security/limits.d/90-nproc.conf*softnofile65536*hardnofile65536*softnproc131072*hardnproc1310723.修改scheduler编辑scheduler，原先默认是cfqechodeadline>/sys/block/vda/queue/scheduler[root@bigdata03~]#more/sys/block/vda/queue/schedulernoopanticipatory[deadline]cfq4.编辑/boot/grub/grub.conf新增elevator=deadline[root@bigdata03~]#more/boot/grub/grub.conf...elevator=deadlinecrashkernel=auto...5.编辑设备块大小[root@bigdata03~]#/sbin/blockdev--setra16384/dev/vda[root@bigdata03~]#/sbin/blockdev--getra/dev/vda163846.配置hosts文件编辑/etc/hosts，添加，选中203作为master，其他为segment,偷懒没改，直接加了层别名172.30.13.203bigdata03.rdchzmdw172.30.13.204bigdata04.rdchzsdw1172.30.13.205bigdata05.rdchzsdw2172.30.13.206bigdata06.rdchzsdw3172.30.13.207bigdata07.rdchzsdw4172.30.13.208bigdata08.rdchzsdw5172.30.13.209bigdata09.rdchzsdw6172.30.13.210bigdata10.rdchzsdw77.分别创建各数据节点文件(一个含master，一个不含)[root@bigdata03~]#more/tmp/hostfile_exkeysmdwsdw1sdw2sdw3sdw4sdw5sdw6sdw7[root@bigdata03data]#more/tmp/hostfile_segmentssdw1sdw2sdw3sdw4sdw5sdw6sdw7三、集群安装1.软件下载，上传、解压缩下载地址：输入yes同意aggrement输入安装路径，比如/usr/local/gpmaster最后安装完后，结果是这样的：[root@bigdata03gpmaster]#lltotal284drwxr-xr-x3gpadmingpadmin4096Jun2412:29bindrwxr-xr-x2gpadmingoadmin4096Jun2412:09demodrwxr-xr-x5gpadmingpadmin4096Jun2412:09docsdrwxr-xr-x2gpadmingpadmin4096Jun2412:09etcdrwxr-xr-x3gpadmingpadmin4096Jun2412:09ext-rw-r--r--1gpadmingpadmin43025Jun2412:29GPDB-LICENSE.txt-rw-r--r--1gpadmingpadmin664Jul2814:51greenplum_path.shdrwxr-xr-x6gpadmingpadmin4096Jun2412:09includedrwxr-xr-x9gpadmingpadmin12288Jun2412:09lib-rw-r--r--1gpadmingpadmin193083Jun2412:29LICENSE.thirdpartydrwxr-xr-x2gpadmingpadmin4096Jun2412:29sbindrwxr-xr-x3gpadmingpadmin4096Jun2411:57share2.建立master与各节点的信任关系[root@bigdata03~]#source/usr/local/greenplum-db/greenplum_path.sh[root@bigdata03~]#gpssh-exkeys-f/tmp/hostfile_exkeys[STEP1of5]createlocalIDandauthorizeonlocalhost[STEP2of5]keyscanallhostsandupdateknown_hostsfile[STEP3of5]authorizecurrentuseronremotehosts...sendtosdw1******Enterpasswordforsdw1:...sendtosdw2...sendtosdw3...sendtosdw4...sendtosdw5...sendtosdw6...sendtosdw7[STEP4of5]determinecommonauthenticationfilecontent[STEP5of5]copyauthenticationfilestoallremotehosts...finishedkeyexchangewithsdw1...finishedkeyexchangewithsdw2...finishedkeyexchangewithsdw3...finishedkeyexchangewithsdw4...finishedkeyexchangewithsdw5...finishedkeyexchangewithsdw6...finishedkeyexchangewithsdw7[INFO]completedsuccessfully实质就是master与各个节点创建了个ssh的信任关系，信任的key存放在/root/.ssh/authorized_keys里--测试是否无需密码登陆，ok[root@bigdata03~]#sshroot@172.30.13.204Lastlogin:WedJul2908:45:582015from172.30.11.31[root@bigdata04~]#exitlogoutConnectionto172.30.13.204closed.[root@bigdata03~]#[root@bigdata03~]#sshroot@sdw7Lastlogin:WedJul2908:49:202015from172.30.11.31[root@bigdata10~]#exitlogoutConnectiontosdw7closed.[root@bigdata03~]#3.安装各个节点准备首先把包传送到各个节点上，并解压缩以及创建用户，组等[root@bigdata03~]#gpseginstall-f/tmp/hostfile_exkeys-ugpadmin-p12345620150729:09:20:20:029326gpseginstall:bigdata03:root-[INFO]:-InstallationInfo:link_namegreenplum-dbbinary_path/usr/local/gpmasterbinary_dir_location/usr/localbinary_dir_namegpmaster20150729:09:20:20:029326gpseginstall:bigdata03:root-[INFO]:-checkclusterpasswordaccess20150729:09:20:22:029326gpseginstall:bigdata03:root-[INFO]:-de-duplicatehostnames20150729:09:20:22:029326gpseginstall:bigdata03:root-[INFO]:-masterhostname:bigdata03.rdchz20150729:09:20:23:029326gpseginstall:bigdata03:root-[INFO]:-checkforusergpadminoncluster20150729:09:20:24:029326gpseginstall:bigdata03:root-[INFO]:-addusergpadminonmaster20150729:09:20:24:029326gpseginstall:bigdata03:root-[INFO]:-addusergpadminoncluster20150729:09:20:24:029326gpseginstall:bigdata03:root-[INFO]:-chown-Rgpadmin:gpadmin/usr/local/greenplum-db20150729:09:20:24:029326gpseginstall:bigdata03:root-[INFO]:-chown-Rgpadmin:gpadmin/usr/local/gpmaster20150729:09:20:25:029326gpseginstall:bigdata03:root-[INFO]:-rm-f/usr/local/gpmaster.tar;rm-f/usr/local/gpmaster.tar.gz20150729:09:20:25:029326gpseginstall:bigdata03:root-[INFO]:-cd/usr/local;tarcfgpmas