Linear Attention Mechanism: An Efficient Attention for Semantic Segmentation

Code

• github.com/lironui/Linear-Attention-Mechanism
  OfficialIn paperpytorch★ 52
• github.com/joigalcar3/LambdaNetworks
  pytorch★ 6

Abstract

In this paper, to remedy this deficiency, we propose a Linear Attention Mechanism which is approximate to dot-product attention with much less memory and computational costs. The efficient design makes the incorporation between attention mechanisms and neural networks more flexible and versatile. Experiments conducted on semantic segmentation demonstrated the effectiveness of linear attention mechanism. Code is available at https://github.com/lironui/Linear-Attention-Mechanism.

Tasks

Reproductions