AD-WM: Action-Discriminative World Models for Counterfactual Model Predictive Control

Problem

Latent world models traditionally trained for factual transitions lack the capability to effectively differentiate candidate actions in model predictive control (MPC) scenarios. This paper addresses this gap by proposing a new framework, AD-WM (Action-Discriminative World Model), which is designed to improve action discrimination in the context of MPC. The work is presented as a preprint and has not yet undergone peer review.

Method

The AD-WM architecture incorporates several key components:

• Residual Latent Dynamics: This allows the model to capture the dynamics of the environment more effectively.
• Predictor-Level Action-Recovery Regularization: This regularization technique ensures that the model retains action information during the planning phase.
• Inverse Dynamics: This component aids in understanding the relationship between actions and state transitions.
• Normalized Recovery Objective: Based on conditional mutual information, this objective helps in optimizing the action recovery process.

The training mechanism employs auxiliary heads that are responsible for preserving action information during planning, although these heads are discarded during testing to streamline the model's operation.

Results

The performance of AD-WM was evaluated against several benchmarks:

• In the hard-start success metric, AD-WM achieved a success rate of 52.0%, significantly outperforming the LeWM baseline, which recorded only 3.7%.
• For the mean success metric, AD-WM showed improvements in four out of five simulation environments compared to a reproduced baseline.
• In a basic pick-and-place task, AD-WM achieved a success rate of 71.1%, compared to 42.2% for zero-shot transfer to the Franka setup.

Limitations

The authors did not report any limitations in their work. However, as with many novel approaches, potential limitations could include generalizability across diverse environments and the computational efficiency of the model during real-time applications.

Why it matters

The introduction of AD-WM has significant implications for the field of model predictive control, particularly in robotics and autonomous systems. By enhancing the ability to discriminate between actions, this model could lead to more effective and efficient decision-making processes in complex environments. The advancements presented in this paper may pave the way for future research focused on improving action selection mechanisms in latent world models.