MedAI #32: Simple & Efficient Design for Semantic Segmentation with Transformers | Enze Xie

Title: SegFormer - Simple and Efficient Design for Semantic Segmentation with Transformers

Speaker: Enze Xie

Abstract:
We present SegFormer, a simple, efficient yet powerful semantic segmentation framework which unifies Transformers with lightweight multilayer perceptron (MLP) decoders. SegFormer has two appealing features- 1) SegFormer comprises a novel hierarchically structured Transformer encoder which outputs multiscale features. It does not need positional encoding, thereby avoiding the interpolation of positional codes which leads to decreased performance when the testing resolution differs from training. 2) SegFormer avoids complex decoders. The proposed MLP decoder aggregates information from different layers, and thus combining both local attention and global attention to render powerful representations. We show that this simple and lightweight design is the key to efficient segmentation on Transformers. We scale our approach up to obtain a series of models from SegFormer-B0 to SegFormer-B5, reaching significantly better performance and efficiency than previous counterparts. For example, SegFormer-B4 achieves 50.3% mIoU on ADE20K with 64M parameters, being 5x smaller and 2.2% better than the previous best method. Our best model, SegFormer-B5, achieves 84.0% mIoU on Cityscapes validation set and shows excellent zero-shot robustness on Cityscapes-C. Code is available at https://github.com/NVlabs/SegFormer.

Speaker Bio:
Enze Xie (https://xieenze.github.io/) is currently a PhD student in the Department of Computer Science, The University of Hong Kong. His research interest is computer vision in 2D and 3D. He has published 16 papers (including 10 first/co-first author) in top-tier conferences and journals such as TPAMI, NeurIPS, ICML and CVPR with 1400+ citations. His work PolarMask was selected as CVPR 2020 Top-10 Influential Papers. He was selected into NVIDIA Graduate Fellowship Finalist. He has won 1st place in Google OpenImages 2019 instance segmentation track.

------

The MedAI Group Exchange Sessions are a platform where we can critically examine key topics in AI and medicine, generate fresh ideas and discussion around their intersection and most importantly, learn from each other.

We will be having weekly sessions where invited speakers will give a talk presenting their work followed by an interactive discussion and Q&A.  Our sessions are held every Thursday from 1pm-2pm PST.

To get notifications about upcoming sessions, please join our mailing list: https://mailman.stanford.edu/mailman/...

For more details about MedAI, check out our website: https://medai.stanford.edu. You can follow us on Twitter @MedaiStanford

Organized by members of the Rubin Lab (http://rubinlab.stanford.edu)
- Nandita Bhaskhar (https://www.stanford.edu/~nanbhas)
- Siyi Tang (https://siyitang.me)