Attention Softmax

Interactive visualization of the scaled dot-product attention mechanism.

Formula:Softmax(z_i) = e^z_i/T / 危e^z_j/T

Controls

Presets

Scale Factor (T)1.00

Large = Uniform. Small = Sharp.

Raw Logits (Q路K)

z₁3.5

z₂1.0

z₃-2.0

z₄0.5

Transformation Pipeline

1. Scaled Logits

z_i / T

3.5

z1

1.0

z2

-2.0

z3

0.5

z4

→

2. Exponentiate

e^(z_i/T)

33.12

e1

2.72

e2

0.14

e3

1.65

e4

→

3. Softmax

Normalized Weights

88%

w1

7%

w2

0%

w3

4%

w4

危 = 1.0