机器学习的大牛Pedro Domingos在ACM的最新文章,小编将题目翻译成“机器学习的那点儿事儿”,文中给出关于机器学习十二条真知灼见的认识.

AFewUsefulThingstoKnowaboutMachineLearning

DepartmentofComputerScienceandEngineering

UniversityofWashingtonSeattle,WA98195-2350,U.S.A.

PedroDomingos

pedrod@cs.washington.edu

ABSTRACT

Machinelearningalgorithmscan gureouthowtoperformimportanttasksbygeneralizingfromexamples.Thisisof-tenfeasibleandcost-e ectivewheremanualprogrammingisnot.Asmoredatabecomesavailable,moreambitiousproblemscanbetackled.Asaresult,machinelearningiswidelyusedincomputerscienceandother elds.However,developingsuccessfulmachinelearningapplicationsrequiresasubstantialamountof“blackart”thatishardto ndintextbooks.Thisarticlesummarizestwelvekeylessonsthatmachinelearningresearchersandpractitionershavelearned.Theseincludepitfallstoavoid,importantissuestofocuson,andanswerstocommonquestions.

1.INTRODUCTION

Machinelearningsystemsautomaticallylearnprogramsfromdata.Thisisoftenaveryattractivealternativetomanuallyconstructingthem,andinthelastdecadetheuseofmachinelearninghasspreadrapidlythroughoutcomputerscienceandbeyond.MachinelearningisusedinWebsearch,spam lters,recommendersystems,adplacement,creditscoring,frauddetection,stocktrading,drugdesign,andmanyotherapplications.ArecentreportfromtheMcKinseyGlobalIn-stituteassertsthatmachinelearning(a.k.a.dataminingorpredictiveanalytics)willbethedriverofthenextbigwaveofinnovation[15].Several netextbooksareavailabletointer-estedpractitionersandresearchers(e.g,[16,24]).However,muchofthe“folkknowledge”thatisneededtosuccessfullydevelopmachinelearningapplicationsisnotreadilyavail-ableinthem.Asaresult,manymachinelearningprojectstakemuchlongerthannecessaryorwindupproducingless-than-idealresults.Yetmuchofthisfolkknowledgeisfairlyeasytocommunicate.Thisisthepurposeofthisarticle.Manydi erenttypesofmachinelearningexist,butforil-lustrationpurposesIwillfocusonthemostmatureandwidelyusedone:classi cation.Nevertheless,theissuesIwilldiscussapplyacrossallofmachinelearning.Aclassi- erisasystemthatinputs(typically)avectorofdiscreteand/orcontinuousfeaturevaluesandoutputsasingledis-cretevalue,theclass.Forexample,aspam lterclassi esemailmessagesinto“spam”or“notspam,”anditsinputmaybeaBooleanvectorx=(x1,...,xj,...,xd),wherexj=1ifthejthwordinthedictionaryappearsintheemailandxj=0otherwise.Alearnerinputsatrainingsetofexam-ples(xi,yi),wherexi=(xi,1,...,xi,d)isanobservedinputandyiisthecorrespondingoutput,andoutputsaclassi er.Thetestofthelearneriswhetherthisclassi erproducesthe

correctoutputytforfutureexamplesxt(e.g.,whetherthespam ltercorrectlyclassi espreviouslyunseenemailsasspamornotspam).

2.

LEARNING=REPRESENTATION+EVALUATION+OPTIMIZATION

Supposeyouhaveanapplicationthatyouthinkmachinelearningmightbegoodfor.The rstproblemfacingyouisthebewilderingvarietyoflearningalgorithmsavailable.Whichonetouse?Thereareliterallythousandsavailable,andhundredsmorearepublishedeachyear.Thekeytonotgettinglostinthishugespaceistorealizethatitconsistsofcombinationsofjustthreecomponents.Thecomponentsare:

Representation.Aclassi ermustberepresentedinsome

formallanguagethatthecomputercanhandle.Con-versely,choosingarepresentationforalearneristan-tamounttochoosingthesetofclassi ersthatitcanpossiblylearn.Thissetiscalledthehypothesisspaceofthelearner.Ifaclassi erisnotinthehypothesisspace,itcannotbelearned.Arelatedquestion,whichwewilladdressinalatersection,ishowtorepresenttheinput,i.e.,whatfeaturestouse.

Evaluation.Anevaluationfunction(alsocalledobjective

functionorscoringfunction)isneededtodistinguishgoodclassi ersfrombadones.Theevaluationfunctionusedinternallybythealgorithmmaydi erfromtheexternalonethatwewanttheclassi ertooptimize,foreaseofoptimization(seebelow)andduetotheissuesdiscussedinthenextsection.

Optimization.Finally,weneedamethodtosearchamong

theclassi ersinthelanguageforthehighest-scoringone.Thechoiceofoptimizationtechniqueiskeytothee ciencyofthelearner,andalsohelpsdeterminetheclassi erproducediftheevaluationfunctionhasmorethanoneoptimum.Itiscommonfornewlearnerstostartoutusingo -the-shelfoptimizers,whicharelaterreplacedbycustom-designedones.Table1showscommonexamplesofeachofthesethreecom-ponents.Forexample,k-nearestneighborclassi esatestexampleby ndingthekmostsimilartrainingexamplesandpredictingthemajorityclassamongthem.Hyperplane-basedmethodsformalinearcombinationofthefeaturesperclassandpredicttheclasswiththehighest-valuedcombina-tion.Decisiontreestestonefeatureateachinternalnode,