Real physics for artificial intelligence
Machine learning
Access to volumes of high quality data is essential for machine learning. Physics simulation is the only practical way to generate such data. This is particularly obvious in the design phase, where machine hardware hasn’t even been built yet, but you still need to work on the basic machine learning controllers.
Machine intelligence
Our technologies enable simulation of a large variety of complex machines ranging from traditional industry robots to humanoids, as well as construction machines, cranes, ships and sub-sea robots. Complex modeling capability and high simulation fidelity enables high sim2real transfer when you deploy your trained AI in real machines.
Simulation platforms
Development of machine autonomy relies on efficient integration of physics simulation, 3D-graphics and AI-software. We support many platforms and tools, e.g. AGX Dynamics for Unity, AGX Dynamics for Unreal, web/cloud environments and many of the market leading AI-tools, such as Tensorflow, pyTorch, ML Agents, Matlab and more. You can also integrate AGX Dynamics into your own tools and platforms.
Deep reinforcement learning for robot control
This video shows an ABB robot with an end-to-end machine learning controller. Thus, we replace the entire control chain with machine learning including, programming, trajectory generation and mechanical control. The task for the robot is to align its tool tip with a goal position.
This is how we did it:
- Set up the simulation in AGX Dynamics for Unity.
- Connect sensors and control data from the simulation with Unity ML Agents
- Design a function that gives optimal reward when the tool tip is located at a given goal position (the colored box) and aligned with a goal vector (normal to the box surface)
- Design a deep neural network in ML Agents to be trained for robot control. In this case we use three hidden layers and a width of 200 neurons
- Run a large number of simulations and use deep reinforcement learning to train the network. Here we use the vanilla Proximal Policy Optimization (PPO) method
- Voila, you have an AI controlled robot!
- In industrial applications, end-to-end learning might not be optimal. Then you can instead re-use parts of your control chain for the robot and only replace crucial parts with machine learning
A machine learning controlled robot that has been trained from physics simulation data, trying to align its tool tip with the goal position.
Autonomous loading
So far, robotics has dominated the research on machine learning controllers. We’re expecting that most types of machines will become robots in the coming decade. This development is particularly important for machines operating in environments that are hazardous to humans, for example mines. The videos illustrate how simulation is used to explore optimal loading strategies in a mine shaft. This requires accurate modeling and simulation of the vehicle including engine, drive train and wheels, the hydraulic loading system and the bucket as well as interaction with the rock pile.
In this particular example, simulations are used to derive a surrogate model and Pareto optimal loading strategies. This work was done in a research project lead by Dr Martin Servin at Umeå university together with Daniel Lindmark from Algoryx. See the project page and the scientific article for further details. These results can be readily extended to train an autonomous control system, such as a deep reinforcement learning system.
AI and Machine Learning News
Algoryx in research collaboration to develop autonomous machines for risky mining environments
AGX Dynamics for Unity adds Linux support, with a view to expanding Machine Learning capabilities
