THE CODE READER / PYTHON

example-873288aad740.py

Your snippet, in context. Explore the file or visualize its recorded example.

PYTHON / LINE NUMBERS
 1import numpy as np
 2from numpy_core import attention
 3
 4q = np.array([[1., 0.]])
 5k = np.array([[1., 0.], [0., 1.], [1., 1.]])
 6v = np.array([[1., 0.], [0., 2.], [2., 1.]])
 7output, weights = attention(q, k, v)
 8
 9np.testing.assert_allclose(weights.sum(-1), 1)
10np.testing.assert_allclose(output, [[1.203336, 0.796664]], atol=1e-5)
11print(np.round(weights, 4))
12print(np.round(output, 4))