Neuromodulation Gated Transformer
2023-05-05Code Available0· sign in to hype
Kobe Knowles, Joshua Bensemann, Diana Benavides-Prado, Vithya Yogarajan, Michael Witbrock, Gillian Dobbie, Yang Chen
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/kobeknowles/neuromodulation-gated-transformerOfficialIn papertf★ 0
Abstract
We introduce a novel architecture, the Neuromodulation Gated Transformer (NGT), which is a simple implementation of neuromodulation in transformers via a multiplicative effect. We compare it to baselines and show that it results in the best average performance on the SuperGLUE benchmark validation sets.