Full Description:
This project uses a TinkerKit Braccio robotic arm, driven by an Arduino UNO over USB serial, to detect coloured cubes with an overhead camera and sort them.
- Learning to pick: The arm is taught by driving it onto cubes a few times. A small learned model (kNN) then maps what the camera sees directly to servo angles, so no camera or arm calibration is needed. It can also collect many of its own examples. Roughly 25–40 examples cover a desk-sized area.
- Web dashboard: The arm is watched and controlled from any phone or laptop. It has a live 3D model, the camera view, colour setup, training, auto-collection, a live log and role-based access (owner, operator, viewer).
- Exhibition mode: The arm keeps arranging cubes into patterns (lines, circles, colour zones, swaps) for hours, with rest periods and a big-screen view.
- Simulation: A Gazebo simulation uses inverse kinematics.
- Hardware and software: Raspberry Pi 5 (Ubuntu Server 24.04), ROS 2 Jazzy, Arduino UNO with the Braccio shield, and a Grove Vision AI V2 or USB camera. Edge Impulse is used for cube detection in simulation.