Vision Sensor Helps Make Robot a Guitar Hero

Oct. 5, 2009
Banner Engineering vision sensor helps make robot a guitar hero.

Banner Engineering Corp.,

Minnesota West Community and Technical College,

Watch a video of this robotic guitar hero,

Video of another Guitar-Hero-playing system from Cyth Systems,

Engineering student Peter Nikrin got a bit upset when his friend bested him at Guitar Hero, a video game where players score points by correctly hitting keys on a plastic guitar as “notes” to popular rock tunes scroll by on a TV. Rather than buckle down and practice so he could beat his friend’s score, Nikrin decided to build a robot that could.

The robot depends to a large degree on a vision sensor able to quickly detect the notes as they scroll by. Nikrin’s first attempt used five fiber-optic photoelectric sensors, one for each key/note color. But the sensors could not detect the notes, which were just moving spots of color on a video screen. It turned out the fiber-optic sensors didn’t match up well with video; they worked best at detecting light reflecting from a solid object.

A PresencePlus P4 Omni sensor from Banner Engineering, Minneapolis, worked better. His school had purchased one as a start-up educational kit. Nikrin also used a right-angle lens from Banner so that the sensor would fit inside the robot’s head. It took some programming effort before the robot could read signals from the monochromatic sensor to detect and identify notes by position rather than color. A bit more programming went into recognizing the white circle surrounding a note that indicates the right instant to play that note.

It takes the robot about 9 msec to recognize that it’s time to play a note. That’s when it sends a signal to one of five solenoids that activate the robot’s five fingers.

The robot, named Roxanne, averages 98% accuracy on the game’s medium setting. “The higher the difficulty level, the less accurate the robot is, but this is mostly a mechanical issue,” says Nikrin. “I spent so much time on programming that I didn’t have time to focus on the solenoids, so the robot can’t press buttons fast enough to beat the most difficult levels. But the vision system itself hits the notes correctly 100% of the time at all difficulty levels.”

Sponsored Recommendations

How BASF turns data into savings

May 7, 2024
BASF continuously monitors the health of 63 substation assets — with Schneider’s Service Bureau and EcoStruxure™ Asset Advisor. ►Learn More: https://www.schn...

Agile design thinking: A key to operation-level digital transformation acceleration

May 7, 2024
Digital transformation, aided by agile design thinking, can reduce obstacles to change. Learn about 3 steps that can guide success.

Can new digital medium voltage circuit breakers help facilities reduce their carbon footprint?

May 7, 2024
Find out how facility managers can easily monitor energy usage to create a sustainable, decarbonized environment using digital MV circuit breakers.

The Digital Thread: End-to-End Data-Driven Manufacturing

May 1, 2024
Creating a Digital Thread by harnessing end-to-end manufacturing data is providing unprecedented opportunities to create efficiencies in the world of manufacturing.

Voice your opinion!

To join the conversation, and become an exclusive member of Machine Design, create an account today!