Voice-commanded robotic arm that sees its workspace and acts on natural-language instructions.
Problem
Hobby and lab robot arms run pre-programmed routines; they cannot understand a spoken request or reason about a cluttered table.
Solution
A 5-DOF servo arm driven by a vision + language pipeline: a webcam view is warped to a bird's-eye frame with homography, segmented, and sent with coordinates to a fast multimodal LLM that chooses actions; a Raspberry Pi relays them to an Arduino running inverse kinematics, with position feedback closing the loop.
Demo moment
Say a command out loud and the arm sorts parts on a messy table, holds a tool, or waves back.
Why it stood out
- curator inferenceNamed and framed after Iron Man's robot helper, so the pitch needs no explanation.
- team statedChose homography + inverse kinematics + fast LLM inference over research-stage vision-language-action models, trading novelty for a demo that reliably works.Source ↗
- team statedVisible engineering on the table: six servos, ~9A of current, full-rotation wiring and a closed feedback loop.Source ↗
- organizer statedOrganizer featured it on the next year's homepage as a showcase project.Source ↗
Lessons for your next hackathon
- Pick the boring, reliable technique that makes the live demo work; save the research-grade approach for 'what's next'.
Projects that share domains, mechanisms or judging lenses — often with a different problem and stack.Tune the comparison.
Grand winner · 2024
A Furby-shaped desk assistant that follows your face, reads your mood and talks back.
Shared: domains productivity, robotics hardware · mechanisms cultural reference hook, physical artifact, demo first wow, one sentence clarity · lenses wow factor · ai llm api, computer vision, voice ai · stack python, arduino, servos
Grand winner · 2025
A turret that finds each plant with a camera and shoots it exactly the water it needs, controlled from a web app.
Shared: domains robotics hardware · mechanisms physical artifact, demo first wow, visible engineering depth, one sentence clarity · lenses technical depth, wow factor · ai computer vision · stack arduino, servos, opencv, python
Finalist · 2025
A robot that draws paintings onto a blind person's palm with warm and cold brushes, with narration.
Shared: domains robotics hardware · mechanisms physical artifact, cultural reference hook, visible engineering depth, demo first wow · lenses wow factor, technical depth · ai computer vision, voice ai · stack python, arduino, servos
Award winner · 2025
A swerve-drive chair you steer by leaning, running on a real-time OS.
Shared: domains robotics hardware · mechanisms physical artifact, demo first wow · lenses technical depth, wow factor · stack python
Award winner · 2026
A robot desk lamp reworked into a companion you can talk to.
Shared: domains robotics hardware · mechanisms physical artifact · lenses wow factor · ai voice ai, computer vision, llm api · stack raspberry pi, servos, opencv, python
Award winner · 2025
Reach into the screen to grab, scale and rotate 3D models with your hands, and ask a voice assistant about them.
Shared: mechanisms cultural reference hook, demo first wow · lenses technical depth, wow factor · ai computer vision, voice ai · stack python, opencv, flask