“Merging diverse data sources does not lead to good results. The key to using them well is to add diverse context to the prompt.” This is the bet that Physical Intelligence is taking in their latest work π0.7: a Steerable Model with Emergent Capabilities. I want to explore whether this addition of diverse context is a trick that is solving a wrong problem or this is a long term solution to make capable VLA models for robotics. I present a few ideas around what can be done more in research as well as where do the opportunities lie for startups.
About Pi0.7
In this work, the PI team present a new general purpose model that exhibits compositional task generalization and cross-embodiment transfer. They achieve this by training on broad and diverse data i.e. data from different robots, human data and autonomous episodes collected by running various policies. But instead of naively using the data they add diverse context to the prompt: training the model with a variety of multimodal prompt structures that specify not only what the robot should do, but how it should do it. The prompt can include not just a textual description of the task, but a variety of other annotations and modalities. For example, providing the model with a visual subgoal defines a precise spatial layout of objects. Providing the desired length of the episode specifies how quickly the task should be done. All of these pieces of information disambiguate the behavior, enabling diverse data with different strategies, behaviors, and levels of proficiency to be included in training.
The episode metadata is the backbone of the work here. This aims at expanding the context provided to the model to train on a broader, more diverse data set of trajectories allowing the training dataset to leverage lower-quality demonstrations, including failures and even autonomous data from prior models instead of purely relying on high quality data.
Episode metadata includes three main things:
- Overall Speed: length of the episode in time steps discretized into interval of 500 steps
- Overall Quality: task execution quality expressed as a score between 1 and 5 with 5 being the highest quality
- Mistake: a binary of whether or not the robot made a mistake within a given action segment
The Potential Wrong Problem
PI’s focus can be broken down to “how do we best utilize the data that we have” instead of “we don’t have nearly enough data”. This may be the wrong problem to solve. Metadata conditioning makes existing pile of mediocre demos trainable. But if the binding constraint on robot foundation models is raw coverage i.e. enough distinct tasks, objects, environments, embodiments then squeezing more signal out of a small, narrow pile is optimizing the wrong axis. If the gains truly do not extend past the sampled distribution, metadata effectively becomes a way to overfit more efficiently.
The cost perspective of this is that they may be substituting labelling cost for data-collection cost. Moving the bottleneck from “collect good demos” to “label bad demos”.
The test that suggests that they are on the right track
In the past, effectively learning from diverse data has been a challenge. The PI team is attempting the same. The test is simple… break the data into 4 buckets
- The top 30% by quality and speed
- The top 50%
- The top 80%
- All of the data
And then they trained the models on all of them with and without the metadata.
The findings are that π0.7 (without metadata) can actually get worse when trained on larger, mixed-quality datasets. π0.7 (with metadata) continuously improves as they train on more data, even though the dataset size increases corresponds to a decrease in average data quality. This makes the model design more scalable.
The missing “seen” and “unseen”
The PI team discusses the limitation of the experiments in that it’s practically difficult when training on such large and diverse datasets to definitively determine which tasks are truly “seen” or “unseen”. They tested on tasks for which they had not deliberately collected data and it seemed to work. But the variety of scenes does not guarantee that they are not already present in the data. They can be present in the form of potentially related skill, under a different, or incidentally as part of some other tasks.
This raises two questions:
- Whether the compositional generalization is actually happening and the model can truly invent new behaviors or it is actually just remixing old tasks.
- Is it only learning what poor and good actually mean or is it learning different behaviors as different behaviors. If it’s only learning about “poor” and “good” quality, then the value is limited but if it is learning the “poor” quality trajectory as a different solution that is poor for performing specific task then very soon we’ll start seeing more creative ways of doing the tasks as well. The way humans learn from mistakes the models will too. The way humans use the failures in one task serendipitously as a success in another, models will start doing that too. This means the value of diverse quality data will multiply. It would be interesting to test for two things:
- What happens when we ask it to perform a task poorly
- What happens when we ask it to solve a task “creatively”
These tests are likely to bring a lot of information on what does the model actually learn.
The opportunities for the startups
There are a few opportunities where startups can contribute value to these VLA models:
- Data Collection at Scale: Generating more and higher quality data is crucial for all robotics foundational models. Better hardware, software, more machines deployments in real world performing tasks and capturing data both autonomous and teleoperation, business models that encourage gathering of data from users.
- Data Labelling: As of now, most of the data labelling tasks seems to be done by human annotators. This means high costs and high variability. There is a lot that can be done with the existing vision models to assist the labelling of the data.
- Embodiment testing: The models can operate with different arms embodiment. There is much to be gained by testing the same models on low cost, open-source arms. This testing data can enable the models to more generalized.
I started this conversation with whether PI is solving a wrong problem or building a long term solution for all VLAs. My bet is that this is a highly valuable long term approach. Extracting more information out of every minute of available data. However, PI’s work seems to be at similar levels as GPT2 when it came out i.e. there are lot of rough edges but still the utility is already high. With the right systems and good engineering a lot of value can be delivered in real world today. The data from real world applications can further educate the models. This is a great time to be building on top of and for the VLAs.
— Ankur Goel
I am brewing a few experiments. I might do some interesting things soon. You can reach me at:
You can check out other blogs I wrote at:
When AI hallucinates what it sees: notes from a robotics sim
Building a personal automation system on the graveyard of past attempts