aRLarm demonstration

Every so often, I like to strip away the problem-design bottleneck and focus on workflow.

If I’m stuck on the conceptual part of a problem, I’m not actually working with the tools needed to get the job done. If I’m not using tools, I’m not learning them. And if I’m not learning them, I get slowed down when I do get to implementation.

Not to mention, new tooling is moving on the scale of months, maybe even weeks now. Easy to spend time on an old workflow that might be getting lapped by a new one.

Visiting an old friend

Designing a neural network for a 2D planar actuator was actually one of the first machine learning projects I used when learning ML, so this one is a bit nostalgic. That one was a pretty simple FFNN to learn inverse kinematics. Except, once I tried to go higher than 4 DOF, I gave up.

So, jump forward to modern architectures. A bit of a statement for the scaling advocates too. Using a transformer model that can relate all the arm angles to the target at the same time, applying reinforcement learning, and throwing a bunch of compute at it, I got models working up to 6-DOF. The only catch is that at the far end (6-DOF, 4 million steps), the training started taking days.

Could I keep scaling? Probably, it’s just getting expensive and it’s too much waiting for this step of the project.

Scheduling/Training Workflows

Anyways, I’ve gotten what I wanted out of this, which is working out some HPC scheduling workflows for training that I like. I’ve got an idea for what I want to try next time for this arm, but that’ll be for later. For now, workflow practice is over and it’s time to get back to applying the workflows toward research problems again.

  • Tinkering here: https://github.com/dfu99/aRLarm
  • The old IK robot for comparison: https://github.com/dfu99/IK-FNN