RESEARCH

COMPUTER VISION

End-to-end object detection with Transformers

May 26, 2020

Abstract

We present a new method that views object detection as a direct set prediction problem. Our approach streamlines the detection pipeline, effectively removing the need for many hand-designed components like a non-maximum suppression procedure or anchor generation that explicitly encode our prior knowledge about the task. The main ingredients of the new framework, called DEtection TRansformer or DETR, are a set-based global loss that forces unique predictions via bipartite matching, and a transformer encoder-decoder architecture. Given a fixed small set of learned object queries, DETR reasons about the relations of the objects and the global image context to directly output the final set of predictions in parallel. The new model is conceptually simple and does not require a specialized library, unlike many other modern detectors. DETR demonstrates accuracy and run-time performance on par with the well-established and highly-optimized Faster RCNN baseline on the challenging COCO object detection dataset. Moreover, DETR can be easily generalized to produce panoptic segmentation in a unified manner. We show that it significantly outperforms competitive baselines. Training code and pretrained models are available at https://github.com/facebookresearch/detr

Download the Paper

AUTHORS

Written by

Nicolas Carion

Alexander Kirillov

Francisco Massa

Gabriel Synnaeve

Nicolas Usunier

Sergey Zagoruyko

Publisher

ECCV

Research Topics

Computer Vision

Related Publications

March 09, 2023

COMPUTER VISION

The Casual Conversations v2 Dataset

Bilal Porgali, VĂ­tor Albiero, Jordan Ryda, Cristian Canton Ferrer, Caner Hazirbas

March 09, 2023

February 21, 2023

COMPUTER VISION

CORE MACHINE LEARNING

ArchRepair: Block-Level Architecture-Oriented Repairing for Deep Neural Networks

Felix Xu, Fuyuan Zhang, Hua Qi, Jianjun Zhao, Jianlang Chen, Lei Ma, Qing Guo, Zhijie Wang

February 21, 2023

January 10, 2023

COMPUTER VISION

CORE MACHINE LEARNING

Online Backfilling with No Regret for Large-Scale Image Retrieval

Gokhan Uzunbas, Joena Zhang, Sara Cao, Ser-Nam Lim, Taipeng Tian, Bohyung Han, Seonguk Seo

January 10, 2023

January 04, 2023

COMPUTER VISION

CORE MACHINE LEARNING

Learning from Few Samples: Transformation-Invariant SVMs with Composition and Locality at Multiple Scales

Xi Liu, Panganamala Kumar, Ruida Zhou, Tao Liu

January 04, 2023

Help Us Pioneer The Future of AI

We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment.