Question 1

What are Q, K, and V?

Accepted Answer

Query is what you search for, Key is what each token advertises, and Value is the information you retrieve. Dot products between Q and K produce relevance scores; softmax turns them into weights that mix V vectors into an updated token representation.

Question 2

Why softmax?

Accepted Answer

Softmax converts dot-product scores into weights between 0 and 1 that sum to 1 per query row. It concentrates attention on the most relevant tokens while keeping a probabilistic interpretation.

Question 3

What is a causal mask?

Accepted Answer

For left-to-right generation (GPT-style), future keys (j > i) get score −∞ so their weight becomes 0 after softmax. BERT uses bidirectional attention and can see all tokens at once.

Question 4

Why divide by √d_k?

Accepted Answer

As dimension d_k grows, dot products get larger and softmax can become too sharp. Scaling by √d_k stabilizes score magnitudes for training and interpretation.

Question 5

Is it free with no install?

Accepted Answer

Yes. It runs entirely in your browser—no PyTorch or TensorFlow required. A free educational playground to visualize QKV → softmax → weighted sum for Self-Attention.

Attention Playground

Attention Playground