小型搜索引擎的设计与实现

大连民族学院2005届信息与计算科学专业本科毕业论文1摘要互联网上的信息每天都以指数量级的速度爆炸性增长，面对如此浩瀚的资源，搜索引擎为所有网上冲浪的用户提供了一个入口，所有的用户都可以从搜索引擎出发到达自己想去的网上任何一个地方。因此它也成为除了电子邮件以外最多人使用的网上服务。但是，随着信息多元化的增长，千篇一律的给所有用户提供同一个入口显然已经不能满足特定用户更深入的查询需求。本文首先详细介绍了基于英特网的搜索引擎的系统结构，然后从网络机器人、索引引擎、Web服务器三个方面进行详细的说明。这不仅对政府、企业、院校的发展极为不利，还在宏观上成为制约我国信息化建设健康良性发展的一大障碍。搜索引擎不是单纯的技术问题。在互联网时代，哪一个公司掌握了包括搜索引擎、信息传递在内的基础软件，它就能在竞争中傲视群雄；哪一个国家掌握和普及了这些技术，她就能在运用互联网的商业竞争占尽先机。关键词：搜索引擎，网络机器人，优化策略，索引小型搜索引擎的设计与实现曹成2AbstractInformationontheInternetgrowsexplosivelyeveryday.Searchengineprovidesallthesurfersonitwithanentrance,fromwhichtheycanreacheverycorneroftheweb.Therefore,searchenginebecomesthemostpopularnetworkservicesecondtoemail.Withinformationcontinuingtoexplodeinalldirections,however,somespecifickindsofusersarenotsatisfiedwithonlyoneentrance.Thisarticlefistintroducesthesystemstructureofsearchenginebasedontheinternetindetail,thengivesaminuteexplanationformSpidersearch,engineandwebserver.Thisnotonlyisextremelydisadvantageoustothedevelopmentofthegovernment,businessenterprise,college,butalsobecometomakeonthemacroviewtheroughlyourcountryinformationturnsabigobstacleofthepositivedevelopmentinhealthindevelopments.Searchingfortheengineisnotapuretechniqueproblem.LookdownuponthegroupofheroesinInternetages,whichcompaniescontrolincludemanhuntengine,informationdeliverfoundationininsidesoftware,itcaninthecompetition;Whichnationscontroledwithmadewidelyavailablethesetechniqueses,shecanoccupyatthebusinessthatmakeuseoftheInternetcompetitionexhaustedfirsttiming.Keywords:SearchEngine，Robot，OptimizeStrategies，Index大连民族学院2005届信息与计算科学专业本科毕业论文3目录摘要................................................................................................................1Abstract................................................................................................................2第一章概述...........................................................................................................51.1引言..................................................................................................................51.2课题的基本内容...............................................................................................61.2.1搜索引擎三段式工作流程：....................................................................61.3开发环境...........................................................................................................7第二章搜索引擎的技术概要....................................................................................92.1搜索引擎简述...................................................................................................92.2基于INTERNET的搜索引擎的构成的...............................................................102.2.1网络蜘蛛.................................................................................................102.2.2索引.........................................................................................................102.2.3搜索引擎软件.........................................................................................102.3搜索引擎的主要指标及其分析......................................................................11第三章网络机器人..................................................................................................123.1什么是网络机器人.........................................................................................123.2网络机器人的结构.........................................................................................123.2.1网络机器人（SPIDER）的组成..............................................................123.3HTML语言.......................................................................................................133.4网络机器人的实现及代码分析......................................................................143.4.2结点的结构体.........................................................................................153.5关于机器人礼节及ROBOT.TXT.........................................................................213.5.1机器人礼节.............................................................................................213.5.2robot.txt..............................................................................................213.6特点和存在问题.........................................................................................22第四章搜索引擎优化策略......................................................................................244.1站点角度.........................................................................................................244.2用户角度........................................................................................................26第五章系统实现及结论..........................................................................................275．2搜索引擎的完成情况...................................................................................275．3存在的问题..................................................................................................275．4心得体会......................................................................................................27小型搜索引擎的设计与实现曹成4致谢...................................................................................