THE CODE READER / PYTHON

example-adfd4c0abdbe.py

Your snippet, in context. Explore the file or visualize its recorded example.

PYTHON / LINE NUMBERS
 1import numpy as np
 2from numpy_core import MultiHead
 3
 4rng = np.random.default_rng(4)
 5query = rng.normal(size=(2, 4, 8))
 6source = rng.normal(size=(2, 6, 8))
 7layer = MultiHead(width=8, heads=2)
 8output, weights = layer(query, source)
 9
10assert output.shape == (2, 4, 8)
11assert weights.shape == (2, 2, 4, 6)
12np.testing.assert_allclose(weights.sum(-1), 1)