全球14家机构专家发百页报告：警惕人工智能(附全文)

FutureofHumanityInstituteUniversityofOxfordCentrefortheStudyofExistentialRiskUniversityofCambridgeCenterforaNewAmericanSecurityElectronicFrontierFoundationOpenAIFebruary2018TheMaliciousUseofArtificialIntelligence:Forecasting,Prevention,andMitigationTheMaliciousUseofArtificialIntelligence:Forecasting,Prevention,andMitigationAuthorsarelistedinorderofcontributionDesignDirectionbySankalpBhatn[e-mail]ute,UniversityofOxford;ArizonaStateUniversity2Correspondingauthor,[e-mail]stentialRisk,UniversityofCambridge3OpenAI4OpenPhilanthropyProject5ElectronicFrontierFoundation6FutureofHumanityInstitute,UniversityofOxford7FutureofHumanityInstitute,UniversityofOxford;YaleUniversity8CenterforaNewAmericanSecurity9AmericanUniversity10Endgame11Endgame12UniversityofOxford/ArizonaStateUniversity/NewAmericaFoundation13CenterforaNewAmericanSecurity14StanfordUniversity15FutureofHumanityInstitute,UniversityofOxford16CentrefortheStudyofExistentialRiskandCentrefortheFutureofIntelligence,UniversityofCambridge17CentrefortheStudyofExistentialRisk,UniversityofCambridge18CentrefortheStudyofExistentialRisk,UniversityofCambridge19FutureofHumanityInstitute,UniversityofOxford20FutureofHumanityInstitute,UniversityofOxford21InformationSocietyProject,YaleUniversity22FutureofHumanityInstitute,UniversityofOxford23OpenAI24UniversityofBath25UniversityofLouisville26OpenAIMilesBrundageShaharAvinJackClarkHelenTonerPeterEckersleyBenGarfinkelAllanDafoePaulScharreThomasZeitzoffBobbyFilarHyrumAndersonHeatherRoffGregoryC.AllenJacobSteinhardtCarrickFlynnSeánÓhÉigeartaighSimonBeardHaydnBelfieldSebastianFarquharClareLyleRebeccaCrootofOwainEvansMichaelPageJoannaBrysonRomanYampolskiyDarioAmodeiArtificialintelligenceandmachinelearningcapabilitiesaregrowingatanunprecedentedrate.Thesetechnologieshavemanywidelybeneficialapplications,rangingfrommachinetranslationtomedicalimageanalysis.Countlessmoresuchapplicationsarebeingdevelopedandcanbeexpectedoverthelongterm.Lessattentionhashistoricallybeenpaidtothewaysinwhichartificialintelligencecanbeusedmaliciously.Thisreportsurveysthelandscapeofpotentialsecuritythreatsfrommalicioususesofartificialintelligencetechnologies,andproposeswaystobetterforecast,prevent,andmitigatethesethreats.Weanalyze,butdonotconclusivelyresolve,thequestionofwhatthelong-termequilibriumbetweenattackersanddefenderswillbe.Wefocusinsteadonwhatsortsofattackswearelikelytoseesoonifadequatedefensesarenotdeveloped.p.3ExecutiveSummaryp.4ExecutiveSummaryTheMaliciousUseofArtificialIntelligenceInresponsetothechangingthreatlandscapewemakefourhigh-levelrecommendations:1.Policymakersshouldcollaboratecloselywithtechnicalresearcherstoinvestigate,prevent,andmitigatepotentialmalicioususesofAI.2.Researchersandengineersinartificialintelligenceshouldtakethedual-usenatureoftheirworkseriously,allowingmisuse-relatedconsiderationstoinfluenceresearchprioritiesandnorms,andproactivelyreachingouttorelevantactorswhenharmfulapplicationsareforeseeable.3.Bestpracticesshouldbeidentifiedinresearchareaswithmorematuremethodsforaddressingdual-useconcerns,suchascomputersecurity,andimportedwhereapplicabletothecaseofAI.4.Activelyseektoexpandtherangeofstakeholdersanddomainexpertsinvolvedindiscussionsofthesechallenges.p.5ExecutiveSummaryTheMaliciousUseofArtificialIntelligenceAsAIcapabilitiesbecomemorepowerfulandwidespread,weexpectthegrowinguseofAIsystemstoleadtothefollowingchangesinthelandscapeofthreats:•Expansionofexistingthreats.ThecostsofattacksmaybeloweredbythescalableuseofAIsystemstocompletetasksthatwouldordinarilyrequirehumanlabor,intelligenceandexpertise.Anaturaleffectwouldbetoexpandthesetofactorswhocancarryoutparticularattacks,therateatwhichtheycancarryouttheseattacks,andthesetofpotentialtargets.•Introductionofnewthreats.NewattacksmayarisethroughtheuseofAIsystemstocompletetasksthatwouldbeotherwiseimpracticalforhumans.Inaddition,maliciousactorsmayexploitthevulnerabilitiesofAIsystemsdeployedbydefenders.•Changetothetypicalcharacterofthreats.WebelievethereisreasontoexpectattacksenabledbythegrowinguseofAItobeespeciallyeffective,finelytargeted,difficulttoattribute,andlikelytoexploitvulnerabilitiesinAIsystems.p.6ExecutiveSummaryTheMaliciousUseofArtificialIntelligenceWestructureouranalysisbyseparatelyconsideringthreesecuritydomains,andillustratepossiblechangestothreatswithinthesedomainsthroughrepresentativeexamples:•Digitalsecurity.TheuseofAItoautomatetasksinvolvedincarryingoutcyberattackswillalleviatetheexistingtradeoffbetweenthescaleandefficacyofattacks.Thismayexpandthethreatassociatedwithlabor-intensivecyberattacks(suchasspearphishing).Wealsoexpectnovelattacksthatexploithumanvulnerabilities(e.g.throughtheuseofspeechsynthesisforimpersonation),existingsoftwarevulnerabilities(e.g.throughautomatedhacking),orthevulnerabilitiesofAIsystems(e.g.throughadversarialexamplesanddatapoisoning).•Physicalsecurity.TheuseofAItoautomatetasksinvolvedincarryingoutattackswithdronesandotherphy