Visual Recognition With Humans in the LoopSteve BransonCatherine WahFlorian Schr

Visual Recognition With Humans in the LoopSteve BransonCatherine WahFlorian Schr www.phwiki.com

Visual Recognition With Humans in the LoopSteve BransonCatherine WahFlorian Schr

Tabback, Tom, On-Air Personality and General Manager has reference to this Academic Journal, PHwiki organized this Journal Visual Recognition With Humans in the LoopSteve BransonCatherine WahFlorian SchroffBoris BabenkoSerge Belongie Peter WelinderPietro PeronaECCV 2010, Crete, Greece1What type of bird is this2What type of bird is this3 Field Guide

Johnson & Wales University, Charleston US www.phwiki.com

This Particular University is Related to this Particular Journal

What type of bird is this4Computer VisionWhat type of bird is this5BirdComputer VisionWhat type of bird is this6ChairBottleComputer Vision

Parakeet Auklet7Field guides difficult as long as average usersComputer vision doesn’t work perfectly (yet)Research mostly on basic-level categoriesVisual Recognition With Humans in the LoopParakeet AukletWhat kind of bird is this8Levels of CategorizationBasic-Level Categories9[Griffin et al. ‘07, Lazebnik et al. ‘06, Grauman et al. ‘06, Everingham et al. ‘06, Felzenzwalb et al. ‘08, Viola et al. ‘01, ]

Levels of CategorizationSubordinate Categories10[Belhumeur et al. ‘08 , Nilsback et al. ’08, ]Levels of CategorizationYellow Belly Blue Belly Parts in addition to Attributes11[Farhadi et al. ‘09, Lampert et al. ’09, Kumar et al. ‘09]Visual 20 Questions GameBlue Belly noCone-shaped Beak yesStriped Wing yesAmerican Goldfinch yesHard classification problems can be turned into a sequence of easy ones12

Recognition With Humans in the LoopComputer VisionCone-shaped Beak yesAmerican Goldfinch yesComputer VisionComputers: reduce number of required questionsHumans: drive up accuracy of vision algorithms13Research AgendaHeavy Reliance on Human AssistanceMore AutomatedComputer Vision ImprovesBlue belly noCone-shaped beak yesStriped Wing yesAmerican Goldfinch yesStriped Wing yesAmerican Goldfinch yesFully AutomaticAmerican Goldfinch yes20102015202514Field Guideswww.whatbird.com15

Field Guideswww.whatbird.com16Example Questions17Example Questions18

Example Questions19Example Questions20Example Questions21

Example Questions22Basic AlgorithmInput Image ( )Question 1:Is the belly blackQuestion 2:Is the bill hookedComputer VisionA: NOA: YESMax Expected In as long as mation GainMax Expected In as long as mation Gain 23Without Computer VisionInput Image ( )Question 1:Is the belly blackQuestion 2:Is the bill hookedClass PriorA: NOA: YESMax Expected In as long as mation GainMax Expected In as long as mation Gain 24

Tabback, Tom Duo - KAZM-AM, The On-Air Personality and General Manager www.phwiki.com

Basic AlgorithmSelect the next question that maximizes expected in as long as mation gain:Easy to compute if we can to estimate probabilities of the as long as m:Object ClassImageSequence of user responses25Basic AlgorithmModel of user responsesComputer vision estimateNormalization factor26Basic AlgorithmModel of user responsesComputer vision estimateNormalization factor27

Modeling User ResponsesAssume:Estimate using Mechanical TurkgreyredblackwhitebrownbluegreyredblackwhitebrownblueDefinitelyProbablyGuessingWhat is the color of the bellyPine Grosbeak28Incorporating Computer VisionUse any recognition algorithm that can estimate: p(cx)We experimented with two simple methods:1-vs-all SVMAttribute-based classification[Lampert et al. ’09, Farhadi et al. ‘09]29Incorporating Computer Vision[Vedaldi et al. ’08, Vedaldi et al. ’09]Used VLFeat in addition to MKL code + color features30

Research Agenda58This paperResearch Agenda59Computer vision improvesFewer questions requiredResearch Agenda60Computer vision solvedFully automatic system

Tabback, Tom On-Air Personality and General Manager

Tabback, Tom is from United States and they belong to Duo – KAZM-AM, The and they are from  Sedona, United States got related to this Particular Journal. and Tabback, Tom deal with the subjects like Entertainment; Human Interest; Local Government and Politics; Local News; Public Affairs/Issues

Journal Ratings by Johnson & Wales University, Charleston

This Particular Journal got reviewed and rated by Johnson & Wales University, Charleston and short form of this particular Institution is US and gave this Journal an Excellent Rating.