site stats
Understanding Tools

In this paper, we present a new framework for task-oriented object modeling, learning and recognition. The framework include: i) spatial decomposition of the object and 3D relations with the imagine human pose; ii) temporal pose sequence of human actions; iii) causal effects (physical quantities on the target object) produced by the object and action.

In this inferred representation, only the object is visible, and all other components are imagined "dark" matters. This framework subsumes other traditional problems, such as: (a) object recognition based on appearance and geometry; (b) action recognition based on poses; (c) object manipulation and affordance in robotics. We argue that objects, especially man-made objects, are designed for various tasks in a broad sense, and therefore it is natural to study them in a task-oriented framework.


Please cite our paper if you use our code or data.

    title={Understanding tools: {T}ask-oriented object modeling, learning and recognition},
    author={Zhu, Yixin and Zhao, Yibiao and Zhu, Song-Chun},
    booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},

Learning Inference Parsing

The authors thank Dr. Tao Gao and Prof. Josh Tenenbaum for constructive discussions and motivations. The authors also thank Dr. Brandon Rothrock and Siyuan Qi for their assistances in the experiments. This work is supported by DARPA MSEE FA 8650-11-1-7149, ONR MURI N00014-10-1-0933 and NSF IIS-1423305.