Recently, we talked to Dan Fu and Tri Dao – authors of “Hungry Hungry Hippos” (aka “H3”) – on our Deep Papers podcast. H3 is a proposed language modeling architecture that performs comparably to ...

For a while now, we’ve been talking about transformers, frontier neural network logic models, as a transformative technology, no pun intended. But now, these attention mechanisms have other competing ...