HOTR: End-to-End Human-Object Interaction Detection with Transformers

CVPR Oral (2021)


Human-Object Interaction (HOI) detection is a task of identifying "a set of interactions" in an image, which involves the localization of the subject (i.e., humans) and target (i.e., objects) of interaction, and the classification of the interaction labels. Most existing methods have addressed this task in an indirect way by detecting human and object instances and individually inferring every pair of the detected instances. In this paper, we present a novel framework, referred by HOTR, which directly predicts a set of <human, object, interaction> triplets from an image using a transformer-based encoder-decoder. Through the set-level prediction, our method effectively exploits the inherent semantic relationships in an image and does not require time-consuming post-processing which is the main bottleneck of existing methods. Our proposed algorithm achieves the state-of-the-art performance in two HOI detection benchmarks with an inference time under 1 ms after object detection.


김범수(카카오브레인), 이준현(고려대학교), 강재우(고려대학교), 김은솔(카카오브레인), 김현우(고려대학교)



발행 날짜