Skip to content

Incorrect Object Orientation in GRAB Environment Reconstruction in MuJoCo #23

@binbinbai

Description

@binbinbai

Hi,thanks for this great job.
I am reconstructing the GRAB environment in MuJoCo, ensuring that the world coordinate system aligns with that of GRAB. My understanding is that the global_orient parameter represents the axis-angle rotation of the object’s local coordinate system relative to the world coordinate system. Therefore, by aligning the MuJoCo world coordinate system with GRAB, I assumed I could directly use global_orient to set the object's orientation.
However, this approach often results in incorrect orientations. For example, as shown in the attached figure with the "elephant" object, directly applying the provided global_orient does not align the object with its true pose in GRAB.
image
image

To investigate, I examined the visualize_grab.py script provided in GRAB. The code responsible for transforming the object vertices from the local coordinate system to the world coordinate system is:
vertices = torch.matmul(v_template, rot_mats) + transl.unsqueeze(dim=1)
This corresponds to the transformation formula:
image
However, I think the correct transformation should be:
image
To verify, I negated the axis-angle rotation provided in global_orient and observed that the resulting object orientation matched the true GRAB pose, as shown in the next figure. This suggests that the global_orient should either be inverted (negated axis-angle) or its corresponding rotation matrix should be transposed for the transformation to be accurate.
image

Is my understanding correct that global_orient provided by GRAB needs to be negated or its rotation matrix transposed to achieve the correct object orientation in MuJoCo?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions