Writing code is difficult enough, but having to programme robots is even more difficult. The coder must create a number of steps, such as detecting objects, activating actuators to move the robot’s limbs, writing code to indicate task completion, and so on.
Google’s robotics researchers are now working on a solution to this problem. Their research team created a robot that can write its own programming code using ordinary language instructions. For example, you could instruct the robot to “pick up the yellow block” and it would handle the rest. To change block target color from #FF0000 to #FFFF00, you would no longer need to enter the robot’s configuration files.
To convert instructions from ordinary language into code, this robot employs Code as Policies (CaP) developed from Google’s Pathways Language Model (PaLM). The AI model of the robot was trained using examples of instructions and corresponding code (formatted as code comments written by the developers to explain what the code does to anyone reviewing it).
In a blog post published earlier this week, Google engineers explained that the robot could take new instructions and:
“Autonomously generate new code that re-composes API calls, synthesizes new functions, and expresses feedback loops to assemble new behaviors at runtime.
It was able to generate likely robot code on its own by employing “hierarchical code generation,” from which it can develop code and repurpose it for similar instructions. CaP has non-code features such as understanding emojis and languages other than English because it is built on a regular language-based model.
CaP’s capabilities, however, remain very limited because it can only work with the right context and instructions that make sense to it. It is unable to handle parameters that it does not support, as well as more complex instructions that require dozens of parameters in a single prompt.
There are also security concerns because it can write its own code. If the robot believes that flailing its arm around with humans nearby is the best way to complete its task, someone may be injured.
Regardless, it’s still exciting research that, if done correctly, would make programmers’ lives a lot easier.