#flash-attention

Flash Attention doesn't make attention faster. It makes attention fit in memory. The speedup is a side effect of better memory access.