Perceiver IO: A General Architecture for Structured Inputs & Outputs

Code

• github.com/deepmind/deepmind-research/tree/master/perceiver
  Officialjax★ 0
• github.com/huggingface/transformers
  pytorch★ 158,292
• github.com/krasserm/perceiver-io
  pytorch★ 519
• github.com/SforAiDl/vformer
  pytorch★ 164
• github.com/esceptico/perceiver-io
  pytorch★ 128
• github.com/MindSpore-scientific-2/code-7/tree/main/SISDTA
  mindspore★ 0
• github.com/2796gaurav/code_examples/tree/main/Perceiver
  none★ 0
• github.com/MindCode-4/code-2/tree/main/perceiver
  mindspore★ 0
• github.com/lucidrains/perceiver-pytorch
  pytorch★ 0

Abstract

A central goal of machine learning is the development of systems that can solve many problems in as many data domains as possible. Current architectures, however, cannot be applied beyond a small set of stereotyped settings, as they bake in domain & task assumptions or scale poorly to large inputs or outputs. In this work, we propose Perceiver IO, a general-purpose architecture that handles data from arbitrary settings while scaling linearly with the size of inputs and outputs. Our model augments the Perceiver with a flexible querying mechanism that enables outputs of various sizes and semantics, doing away with the need for task-specific architecture engineering. The same architecture achieves strong results on tasks spanning natural language and visual understanding, multi-task and multi-modal reasoning, and StarCraft II. As highlights, Perceiver IO outperforms a Transformer-based BERT baseline on the GLUE language benchmark despite removing input tokenization and achieves state-of-the-art performance on Sintel optical flow estimation with no explicit mechanisms for multiscale correspondence.

Tasks

Benchmark Results

Reproductions