Voice-commanded robotic arm that sees its workspace and acts on natural-language instructions.
Problem
Hobby and lab robot arms run pre-programmed routines; they cannot understand a spoken request or reason about a cluttered table.
Solution
A 5-DOF servo arm driven by a vision + language pipeline: a webcam view is warped to a bird's-eye frame with homography, segmented, and sent with coordinates to a fast multimodal LLM that chooses actions; a Raspberry Pi relays them to an Arduino running inverse kinematics, with position feedback closing the loop.
Demo moment
Say a command out loud and the arm sorts parts on a messy table, holds a tool, or waves back.
Why it stood out
- curator inferenceNamed and framed after Iron Man's robot helper, so the pitch needs no explanation.
- team statedChose homography + inverse kinematics + fast LLM inference over research-stage vision-language-action models, trading novelty for a demo that reliably works.Source ↗
- team statedVisible engineering on the table: six servos, ~9A of current, full-rotation wiring and a closed feedback loop.Source ↗
- organizer statedOrganizer featured it on the next year's homepage as a showcase project.Source ↗
Lessons for your next hackathon
- Pick the boring, reliable technique that makes the live demo work; save the research-grade approach for 'what's next'.
Projects that share domains, mechanisms or judging lenses — often with a different problem and stack.Tune the comparison.
Grand winner · 2024
A Furby-shaped desk assistant that follows your face, reads your mood and talks back.
Shared: domains productivity, robotics hardware · mechanisms cultural reference hook, physical artifact, demo first wow, one sentence clarity · lenses wow factor · ai llm api, computer vision, voice ai · stack python, arduino, servos
Finalist · 2025
A robot that draws paintings onto a blind person's palm with warm and cold brushes, with narration.
Shared: domains robotics hardware · mechanisms physical artifact, cultural reference hook, visible engineering depth, demo first wow · lenses wow factor, technical depth · ai computer vision, voice ai · stack python, arduino, servos
Finalist · 2025
A Nerf blaster rebuilt as a first-person-shooter controller.
Shared: domains robotics hardware · mechanisms physical artifact, demo first wow, one sentence clarity, cultural reference hook · lenses wow factor · stack python
Submitted · 2026
Ask for help and coordinate rescues without internet, with remote visibility once the network returns.
Shared: domains robotics hardware · mechanisms physical artifact, visible engineering depth, one sentence clarity · lenses technical depth, wow factor · stack raspberry pi, python
Finalist · 2025
Behavioral-interview practice that scores what you say and how you say it from video.
Shared: mechanisms visible engineering depth, one sentence clarity · lenses technical depth · ai llm api, computer vision, voice ai
Award winner · 2026
Nine substantial pull requests turning agentgateway from a model proxy into an identity-aware control plane for agents.
Shared: mechanisms visible engineering depth · lenses technical depth · ai llm api