OakInk2: A Dataset of Bimanual Hands-Object Manipulation in
Complex Task Completion

Xinyu Zhan1,*, Lixin Yang1,*, Yifei Zhao1, Kangrui Mao1, Hanlin Xu1, Zenan Lin2, Kailin Li1, Cewu Lu1,✉
1Shanghai Jiao Tong University, 2South China University of Technology
* Equal contribution    ✉ Corresponding author   
CVPR 2024
Paper Code arXiv Video
overview

OakInk2 overview. The top row shows the data collection process. The second row shows the three levels of abstraction for complex tasks introduced by OakInk2.

Update

  • 2024-07-16:   Update Preview tools and program.tar to include PDGs and the initial conditions and task recipes of selected tasks.
  • 2024-06-19:   Preview tools & blender tools.
  • 2024-03-29:   Upload the videos and the preview version of annotations.

Download

Video

Image sequences.

Annotations

- 3D Human, Hands and Objects.
- Text descriptions of task targets, Primitives and the execution paths of the subjects.
- Primitive Dependency Graphs, and the initial conditions and task recipes of selected tasks.


Download at Hugging Face
OakInk2 is currently in preview. Annotations are subject to future changes.

Resources

Preview Tools

Preview tools for loading and visualizing the data and the annotations of OakInk2.


OakInk2-Preview-Tools

Blender Tools

Blender tools for rendering the images in the paper.


OakInk2-Blender-Tools

Simulation Environment & Challenge

set_pose
OakInk2-SimEnv-IsaacGym Challenge Overview