Train an ACT Policy for the SO-101 Robot with LeRobot

Опубликовано: 16 Июль 2025
на канале: Trelis Research

403

📜Get repo access at Trelis.com/ADVANCED-robotics

🗝️ Get Trelis All Access (Trelis.com/All-Access)
1. Access all SEVEN Trelis Github Repos (-robotics, -vision, -evals, -fine-tuning, -inference, -voice, -time-series)
2. Support via Github Issues & Trelis’ Private Discord
3. Early access to Trelis videos via Discord

Tip: If you subscribe here on YouTube, click the bell to be notified of new vids

Credit to Rohan Sharma (https://x.com/rs545837) for assistance exploring ACT and GR00T.

💡 Need Technical or Market Assistance?
Book a Consult Here: https://forms.gle/wJXVZXwioKMktjyVA

🤝 Are You a Top Developer?
Work for Trelis: https://trelis.com/jobs/

💸 Starting a New Project/Venture?
Apply for a Trelis Grant: https://trelis.com/trelis-ai-grants/

📧 Get Trelis AI Tutorials by Email
Subscribe on Substack: https://trelis.substack.com

Video Links:
ACT Paper: https://arxiv.org/abs/2304.13705
LeRobot Library: https://github.com/huggingface/lerobot

TIMESTAMPS:
00:00 Introduction to Training the SO-101 Robot with ACT
00:21 Overview of the Video Series
01:16 Scripts and Repo Access: Trelis.com/ADVANCED-robotics
01:57 Cloning and Installing LeRobot Libraries
06:07 Connecting and Configuring the Robots
08:53 Calibrating the Motors and Arms
12:33 Teleoperation Setup
18:04 PID Controller Calibration
27:10 Recording and Managing Data
39:05 Training the ACT Model
44:33 Style settings and KL Weight (ADVANCED)
49:06 Running Training on a Mac (or cpu)
50:54 Setting Up Validation and Output Directories
53:44 Running Training on Mac and Handling Issues
55:31 Monitoring Training Progress
57:25 Calculating Training S teps and Epochs
58:28 Analyzing Training and Validation Loss
01:04:02 Setting Up Training on GPU
01:08:19 Connecting to Remote Host and Cloning Repo
01:12:48 Running Training on CUDA
01:14:48 Handling Issues Running on CUDA
01:23:28 Inspecting Results after Running on CUDA
01:27:04 Evaluating Model Performance
01:28:37 Replay and Evaluation of Training Examples
01:35:21 Challenges with Generalization and Data Requirements
01:36:28 Using Image Augmentations and Jitter
01:37:08 Deciding Number of Rollout Steps
01:40:06 Ensembling Predictions for Smoother Trajectories
01:43:43 Conclusion and Next Steps