muse-qc-cell

Results (real hardware, measured)

All numbers come from human-run tests, recorded in docs/DEVLOG.md.

Sorting runs with 10 parts

Run	What changed	Gripped on the first try	Sorted correctly	Time	Meta API cost
30 Oct, run 1	first 10-part run, close-up right after the lift	10/10	not checked: 5 parts on the left half read 0 levers (light), so they were sorted bad without a count	7 min 20 s, about 44 s per part	0.22 USD (about 2 cents per part, 14 Muse Spark calls)
31 Oct, run 3	lever direction from the slot axis, rescan after every part, fixed inspection station	9/9	10/10 (9 in this agent run plus 1 single pick just before it; boxes checked by hand)	about 6.5 min, about 43 s per part	not measured (the credits page did not load)
2 Nov, 10 parts, arm session	one session for all 10 parts (--plan-once --scan-at-station --arm-session), new empty-table image after turning the lamp away	10/10	10/10 (7 good, 3 bad; 3 bad placed on purpose; boxes checked by hand, all correct)	4 min 39 s first scan to report, about 28 s per part (24 to 28 s count to count)	not measured (5 Muse Spark calls)

Run 2 (31 Oct, first run with the inspection station) gripped 10/10, but several parts were gripped along the wrong axis because the lamp shadow fooled the lever direction. That led to the slot axis fix used in run 3.

Building blocks

Check	Result
Table camera calibration	RMS reprojection error 0.199 px (26 images)
Lever direction from the table camera (7 parts vs. photo)	7/7 within about 10 deg with the slot axis (before: 3/7)
Control loop	100 Hz, cycle period 10.09 ms mean, 10.20 ms max
Grip	touch contact on both fingers before the lift, every grip of run 3 on the first try
Offline test suite	976 tests, no hardware, camera or network needed

How it works

Order: I type an order in plain words. The Muse Spark agent decides which tool to call next: scan_table, inspect_and_sort(part_id), write_report or finish.
Scan: the table camera finds four ArUco markers and maps pixels to table millimeters with a checkerboard calibration. An empty-table reference shows what changed, SAM 3.1 confirms each candidate, and the lever direction is read from the dark slot at one end of the part.
Pick: the arm moves above the part, turns the gripper so the fingers close across the levers, lowers, closes until the touch sensors report contact on both sides, and lifts.
Inspect: the arm carries the part to a fixed inspection station. The arm camera takes a close-up, SAM 3.1 finds the part, the crop is enlarged and SAM 3.1 counts the levers. Exactly 3 levers = good. A doubtful count is retried on fresh frames; a part is only good when the count is confirmed.
Sort: the part is dropped into the good or bad box, the arm returns to rest, and the table is scanned again, so a part that moved is found again.
Report: every step is saved as JSON, and a live page in the browser shows the table, each part's close-up, its lever count and its box.

Where Meta AI is used

Meta tool	Where	What it does
Muse Spark (`muse-spark-1.3`, Meta Model API)	`muse_qc/agent/agent.py`	The agent: reads the order, picks the next tool, reacts to failures and rescans, writes the summary
SAM 3.1 (`sam-3.1`, Meta Model API)	`muse_qc/vision/find_parts.py`, `muse_qc/vision/vision_sam.py`	Confirms parts on the table image
SAM 3.1	`muse_qc/inspection/count_clips.py`, `muse_qc/inspection/count_retry.py`	Finds the held part and counts its levers in the close-up
Muse Code (powered by Muse Spark)	the whole repository	Wrote all the code, logged task by task in `docs/MUSE_CODE_LOG.md`

A real agent run (excerpt)

From run 3 on 31 Oct (docs/DEVLOG.md, output shortened):

agent -> scan_table()
tool: 8 parts found
grip yaw: lever 119.3 deg, offset -19.9 deg
inspect: levers 3 verdict good
tool: rescan found 7 parts
grip yaw: lever 102.3 deg, offset +9.9 deg
inspect: levers 2 verdict bad
...
tool: rescan found no parts, call scan_table
agent -> scan_table()
tool: no parts found
agent -> write_report()
agent finished: Sorted 9 parts: 6 good (3 levers), 3 bad (<3 levers), 0 failures/missed grips. Final rescan found no parts.

Safety

In short: every waypoint is checked before motion, every command is re-checked each control cycle, only a human starts hardware runs, speed is capped, tracking errors abort the move, touch sensing stops the descent, only grasp waypoints may go low, the model passes part IDs only, and devices are found by USB ID.

Not certified

muse-qc-cell is a hackathon research prototype. Its safety layers are software checks only. They have not been assessed or certified by any testing body (for example TÜV), and the cell does not comply with the EU Machinery Regulation (EU) 2023/1230 or robot safety standards such as ISO 10218. There is no certified emergency stop, no guarding and no risk assessment. Do not use it in production or near people without a proper safety concept, risk assessment and certification.

What is new in this hackathon

I started this repository from scratch during the hackathon. Everything in muse_qc/, tests/, docs/ and calibration/ was written during the event.

Prior work, declared and not part of this submission: my Touch-Sensing Fin-Ray Gripper (hardware and firmware). I built it before the hackathon, and it is listed in the official Seeed reBot-DevArm repository under "DIY Soft Finger". New in this hackathon: muse_qc/gripper/gripper.py, which reads the gripper sensors over USB. See docs/THIRD_PARTY.md.

Demo video

Replay of a real run