Abstract. Visual tracking, in essence, deals with non-stationary data streams that change over time. While most existing algorithms are able to track objects well in controlled environments, they usually fail if there is a significant change in object appe

Inthispaper,wepresentane cientandadaptivealgorithmthatincremen-tallyupdatesthemodeloftheobjectbeingtracked.Insteadofusingasimplecontourtoencloseanimageregionorcolorpixelstorepresentmoving“stu ”[9]fortargettracking,weuseaneigenbasistorepresentthe“thing”beingtracked.Inadditiontoprovidingacompactrepresentationofthemodelbasedonthere-constructionprinciple,theeigenbasisapproachalsorendersaprobabilisticinter-pretationandfacilitatese cientcomputation.Furthermore,ithasbeenshownthatforaLambertianobjectthesetofallimagestakenunderalllightingcondi-tionsformsaconvexpolyhedralconeintheimagespace[10],andthispolyhedralconecanbeapproximatedwellbyalow-dimensionallinearsubspaceusinganeigenbasis[11][12].

Givenanobservedimage,wetracktheobjectbasedonamaximumaposte-rioriestimateoflocation-theimageregionnearthepredictedpositionthatcanbebestapproximatedbythecurrenteigenbasis.Wecomputethisestimateusingsamplesofa neparameters,describingtheframetoframemotionofthetarget,drawnfromtheirpriordistributions,andcombiningthemwiththelikelihoodoftheobservedimageregionunderourmodel.

Theremainingpartofthispaperisorganizedasfollows.We rstreviewthemostrelevanttrackingworkinthenextsection.Thedetailsofourtrack-ingalgorithmispresentedinSection3,followedbynumerousexperimentstodemonstrateitsrobustnessunderlargeposeandlightingvariation.Weconcludethispaperwithremarksonpossibleextensionsforfuturework.

2ContextandPreviousWork

Thereisanabundanceofvisualtrackingworkintheliterature,fromasimpletwo-viewtemplatematchingapproach[13]toa3Dmodel-basedalgorithm[6].Thesealgorithmsdi ermainlyintherepresentationscheme–rangingfromcolorpixels,blobs,texture,features,imagepatches,templates,activecontours,snakes,wavelets,eigenspace,to3Dgeometricmodels–andinthepredictionapproach,suchascorrelation,sumofsquaredistance,particle lter,Kalman lter,EMalgorithm,Bayesianinference,statisticalmodels,mixturemodels,andoptimizationformulations.Athoroughdiscussionofthistopicisbeyondthescopeofthispaper,thusinthissectionwereviewonlythemostrelevantobjecttrackingworkandfocusonthealgorithmsthatoperatedirectlyongrayscaleimages.

In[1]BlackandJepsonadvocatedaview-basedeigenbasisrepresentationforobjecttrackingandformulatedthetwo-viewmatchingprocessasanoptimizationproblem.Blacketal.laterextendedtheeigentrackingalgorithmtoamixturemodeltoaccountforchangesinobjectappearance[14].Themajoradvantagesofusinganeigenbasisrepresentationarethatitallowsthetrackertohavethenotionofthe“thing”beingtracked,andthetrackingalgorithmoperatesonthesubspaceconstancyassumptionasopposedtothebrightnessconstancyassumptionofoptical owestimation.Onedisadvantageoftheabovementionedalgorithmsistheuseofviewed-basedrepresentation.Inotherwords,oneneedstolearntheeigenbasisofanobjectateachviewpointbeforetracking,andthesesubspaces