互联网公司技术架构资料新浪微博构建可扩展微博架构

TimYang!• – – – • ContentManagerSystem!• ,LAMP!– MySQLmaster/slave– Memcached– PHP– CDN• – Real-time– – • – Push()– Pull()Push• – Inbox:– Outbox:• inbox()!• Inbox()!Push(Figure)UserAUpdateActionInbox(Appendto1shometimeline)Inbox(Appendto2shometimeline)Inbox(Appendto3shometimeline)FollowersofUserA=1,2,3Push• • Pull• outbox()!• Inbox()!PullUserIGethome_timelineOutbox(statusessentbyA)Outbox(StatusessentbyB)Outbox(StatusessentbyC)UserIsFollowingList=A,B,CPull• • • • -• 5,000/• 578/• 2,000?!IO• 5,000–10,000?!• 100,000?!• Latency!• DBreadtimeout!• timeout(503error)!• • • (MessageQueue)!• MQproducts• Kestrelbytwitter• RabbitMQ,anErlangQueueServer• Memcacheq!– Memcacheq• Berkeleydb,• Memcachedprotocol,clientlibrary!• (statsqueue)!• 2get/set!1. Memcacheq2. Get:3. Set:MQ• Offlinework!• • • • MQThevalueofthetweetdecreasesexponentiallywithtime!JohnKalucki,Twitter!• LocalCache!• Memcached!• Databasebuffer/cache!• LAMPcache=• Cache• TB• – QuickLZ– LZO– gzip• ,SIGSEGV!• – 1.Consistenthash– 2.Read-throughcacheConsistenthash• • – Read-throughcacheRead-throughandWrite-through• Productsorprojects!– MySQLmemcachedUDF– CachemoneyforRubyonRails• Orwrapaproxyforthedbdriver,inanylanguage!Evictions• Evections:cache• • Latencyevictions• cache• • key!MultigetWhenmemcachedserversareCPUbound,addingmorememcachedserversdoesn'thelpservemorerequests.!!-JeffRothschild,VicePresidentofTechnologyatFacebookCachemultigetholeMemcached(MaxRPS:A)Memcached(MaxRPSC)Memcached(MaxRPS:B)Multiget(keys…)MultigetMultigetMultigetApplicationMaxRPSofapplication(AandBandC)Multiget• Memcachedreplication!• – – IDC– • • Master-slave!• Master-master• 2PC/3PC!• Paxos!• !• MySQLmaster/slave!• Dynamo/Cassandra!• PNUTS!API• RESTAPI!– library• curl,javascriptclient– • SinaAppEngine• SinaAppEngineAPI• hostapp!• ,Web2.0• • Q&A!@TimYang!Twitter:@xmpp!Email:[e-mail]!!