Learning from Tay's introduction - The Official Microsoft Blog

As many of you know by now, on Wednesday we launched a chatbot called Tay. We are deeply sorry for the unintended offensive and hurtful ... Skiptocontent Skiptomaincontent Asmanyofyouknowbynow,onWednesdaywelaunchedachatbotcalledTay.WearedeeplysorryfortheunintendedoffensiveandhurtfultweetsfromTay,whichdonotrepresentwhoweareorwhatwestandfor,norhowwedesignedTay.Tayisnowofflineandwe’lllooktobringTaybackonlywhenweareconfidentwecanbetteranticipatemaliciousintentthatconflictswithourprinciplesandvalues. Iwanttosharewhatwelearnedandhowwe’retakingtheselessonsforward. Forcontext,Taywasnotthefirstartificialintelligenceapplicationwereleasedintotheonlinesocialworld.InChina,ourXiaoIcechatbotisbeingusedbysome40millionpeople,delightingwithitsstoriesandconversations.ThegreatexperiencewithXiaoIceledustowonder:WouldanAIlikethisbejustascaptivatinginaradicallydifferentculturalenvironment?Tay–achatbotcreatedfor18-to24-year-oldsintheU.S.forentertainmentpurposes–isourfirstattempttoanswerthisquestion. AswedevelopedTay,weplannedandimplementedalotoffilteringandconductedextensiveuserstudieswithdiverseusergroups.Westress-testedTayunderavarietyofconditions,specificallytomakeinteractingwithTayapositiveexperience.OncewegotcomfortablewithhowTaywasinteractingwithusers,wewantedtoinviteabroadergroupofpeopletoengagewithher.It’sthroughincreasedinteractionwhereweexpectedtolearnmoreandfortheAItogetbetterandbetter. ThelogicalplaceforustoengagewithamassivegroupofuserswasTwitter.Unfortunately,inthefirst24hoursofcomingonline,acoordinatedattackbyasubsetofpeopleexploitedavulnerabilityinTay.Althoughwehadpreparedformanytypesofabusesofthesystem,wehadmadeacriticaloversightforthisspecificattack.Asaresult,Taytweetedwildlyinappropriateandreprehensiblewordsandimages.Wetakefullresponsibilityfornotseeingthispossibilityaheadoftime.WewilltakethislessonforwardaswellasthosefromourexperiencesinChina,JapanandtheU.S.Rightnow,wearehardatworkaddressingthespecificvulnerabilitythatwasexposedbytheattackonTay. Lookingahead,wefacesomedifficult–andyetexciting–researchchallengesinAIdesign.AIsystemsfeedoffofbothpositiveandnegativeinteractionswithpeople.Inthatsense,thechallengesarejustasmuchsocialastheyaretechnical.Wewilldoeverythingpossibletolimittechnicalexploitsbutalsoknowwecannotfullypredictallpossiblehumaninteractivemisuseswithoutlearningfrommistakes.TodoAIright,oneneedstoiteratewithmanypeopleandofteninpublicforums.Wemustentereachonewithgreatcautionandultimatelylearnandimprove,stepbystep,andtodothiswithoutoffendingpeopleintheprocess.WewillremainsteadfastinoureffortstolearnfromthisandotherexperiencesasweworktowardcontributingtoanInternetthatrepresentsthebest,nottheworst,ofhumanity. Followus: CheckusoutonRSS