Why One Attention Head Is Never Enough: Multi-Head Attention Explained
A single attention pass can only ask one question at a time. Multi-head attention runs several in parallel โ each head specialising in a different type of relationship. Here's what that means in practice.
Johannes Hayer
johanneshayer