运维基础设施之数字化运营徐巍

运维基础设施之数字化运营饿了么徐巍CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGAgenda基础设施的定义努力的方向遇到的问题HowtodoTodo一些感想CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCG基础设施的定义IDC网络CGITCGITOCSGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCG基础服务•负载均衡/防火墙(VPN)•私有云/公有云/CDN•分布式文件系统•日志平台•Monitor/DNS/YUM/NTP/LDAP努力的方向稳定快低成本CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCG遇到的问题1网络设备性能不足：核心/接入/F5/防火墙带宽收敛比过低网络结构复杂网络单点不同型号混用监控不全CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCG遇到的问题2服务器不够标准化：机型混乱，且硬盘/内存存在插拔OS版本混乱外网ip/bond有的有，有的没有单点：部分设备单电/单网卡CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCG遇到的问题3xfs坑mode0省电模式网卡中断/网卡百兆CPU温度过高磁盘io虚拟化超分CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-标准化让用户做选择题，而不是问答题CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-标准化-network高收敛比大二层无单点关于防攻击CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-标准化-IDC机柜标准化CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-标准化-服务器机型标准化服务器定制化采购周期化采购框架协议OS标准化CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-标准化-应用统一公司层面运维支持技术平台控制研发技术使用谨慎引入新技术，要经过archreview讨论简化技术平台适当允许引入新技术维护一个稳定版，一个推进版‘用好’技术，比‘用’好技术更重要CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-自动化+流程标准化是自动化实现的前提自动化促进标准化流程才能保证运营数据流转的闭环不能为了流程而流程简单即可用CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-自动化-装机装机和交付分离接电自动化安装CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-自动化-私有云zstackzstack和EWF结合和PRO完全解耦快速部署和恢复CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-自动化-分布式文件系统基于ceph的分布式文件系统数据库备份图片源站对象存储服务CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-自动化-故障检测和报修Disk/mem/raid卡/cpu/网卡百兆监控处理记录CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-自动化-监控网络(traffic+连通性)硬件OSAppmetricCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-自动化-监控各机房调用关系CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-自动化-监控管理报警阀值自定义SnoozeCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-自动化-监控数据分析报警量谁负责报警topCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-自动化-日志平台how实时+离线展示&告警whataccesslogCdnlog大禹logServerlogNetworklogCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-自动化和流程结合自动化流程整合CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-架构review架构评审架构资源技术选型部署模式监控埋点熔断/降级容量预估CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-数据驱动资产成本CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-数据驱动利用率交付CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGHow-数据驱动CDN日志CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGTodo多数据中心高可用，支持多活全万兆网络/DPDKDocker容器化实践资源利用率提高机器学习混合云架构精细化运营数据分析(瓶颈/容量)数据挖掘CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCG一些感想基础设施永远要超前投资业务导向，不能为了技术而技术服务意识很重要简单及可用不要重复造轮子拥抱变化CGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGQ&ACGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGThankyouCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCGITCG欢迎入伙Wechat:xw2014