Images are arrays. The contract matters.

You load a photograph, run a model, and get a poor prediction. Is the model weak, or did your loader supply blue where the model expected red? Both produce a tensor of the expected shape. A successful function call does not prove that the data means what you think it means.

Our first goal is modest and useful: describe an image precisely enough that another function can interpret it correctly. We will use four pixels, follow one through conversion, and build a small batch. These are the same boundaries that exist in a large inference service.

1. Six properties describe an image input

An array stores values arranged along numbered axes. Its shape gives the length of each axis. A color image commonly has shape (height, width, channels), abbreviated HWC. Height counts rows, width counts columns, and channels hold the measurements at one spatial location.

The dtype tells us how each value is stored. uint8 is an unsigned eight-bit integer: it represents integers from 0 to 255. float32 represents signed floating-point numbers, including fractions, with finite precision. Neither dtype tells us the color order or what numeric range the next function expects.

Write this contract where the array enters a pipeline. “Accepts an image” is insufficient documentation. An RGB float image in 0–1 and a BGR byte image in 0–255 are both reasonable representations; the conversion between them must be explicit.

If you need a slower introduction to axes and indexing, use What is actually inside a tensor? . Here we apply that vocabulary to spatial data.

2. Locate a pixel before naming its color

In a (2, 2, 3) array, image[1, 0] selects the bottom-left pixel and returns its three-channel vector. image[1, 0, 2] selects its final channel. image[:, :, 0] selects channel zero at every location and has shape (2, 2). The colon means all entries on that axis.

The top-left pixel has row 0, column 0. Rows increase downward; columns increase rightward. When an OpenCV drawing function asks for (x, y), x usually refers to the column and y to the row. NumPy indexing is [y, x]. A square test image can hide that swap because both coordinates remain valid. A non-square fixture is a better check.

For ordinary three-channel color loading, OpenCV uses BGR ordering. In our first pixel, [0, 0, 255] therefore means full red. An RGB display would interpret those same three numbers as blue. We use cv2.cvtColor(..., cv2.COLOR_BGR2RGB) at the boundary. This channel-order conversion does not rotate, resize, or move the image. See OpenCV’s image-loading contract ↗ and color conversion API ↗ .

def pixel_demo() -> tuple[ByteImage, ByteImage, ByteImage]:
    """Return a (2,2,3) BGR image, its RGB copy, and a copied one-pixel crop."""
    bgr: ByteImage = np.array(
        [[[0, 0, 255], [0, 255, 0]], [[255, 0, 0], [30, 60, 90]]],
        dtype=np.uint8,
    )
    rgb: ByteImage = cast(ByteImage, cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB))
    crop: ByteImage = rgb[1:2, 1:2, :].copy()
    crop[0, 0, 0] = 0
    return bgr, rgb, crop

The complete file defines ByteImage = NDArray[np.uint8]. This alias states the element type; the docstring states the shape. Python annotations do not automatically enforce either condition. The next function checks its public boundary explicitly.

Use Visualize on this block. All twelve input values fit in the recording. The last operation changes the red channel of a copied crop to zero. The original bottom-right RGB pixel remains [90, 60, 30]; the returned crop contains [0, 60, 30].

A channel is one component at every spatial location. A grayscale preview of the red channel draws high red-component values as white; it does not imply the original pixel was white. RGB and BGR specify channel order, while RGBA and BGRA add an opacity channel. The color and alpha lab separates all three planes and explains transparency, reductions, and compositing with exact examples.

3. A crop can be a view of the original

rgb[1:2, 1:2, :] keeps a one-row, one-column region. Slice stops are exclusive, so 1:2 selects only index 1. Slicing retains those axes and gives shape (1, 1, 3); scalar indexing rgb[1, 1] removes them and gives (3,).

Basic NumPy slices generally share the original storage. Editing the crop can edit the source. .copy() creates independent storage for our experiment. This is useful when an augmentation or visualization should not modify the array kept for evaluation. Copying every array is wasteful, but relying on accidental independence is unsafe. The NumPy copies and views guide ↗ describes which operations share data.

Keep coordinates alongside a crop. A mask predicted on a crop is located in crop coordinates. Before comparing it with a full-image target, you must place it back at the correct offset. Matching shape alone does not establish spatial alignment.

4. Convert before arithmetic that needs a wider range

A byte array is convenient for storage but unsuitable for arbitrary signed or fractional calculations. A subtraction such as dark minus bright needs a negative result; uint8 cannot represent it. Fixed-width array arithmetic can wrap around instead of giving the intended difference. Convert operands to a suitable type before subtracting, not after the damage has occurred. NumPy documents this behavior in its numerical types and overflow guide ↗ .

For our model input, we convert to float32 and scale each channel into the unit interval:

def to_unit_rgb(bgr: ByteImage) -> FloatImage:
    """Nonempty uint8 (H,W,3) BGR -> float32 (H,W,3) RGB in [0,1]."""
    if bgr.dtype != np.uint8 or bgr.ndim != 3 or bgr.shape[2] != 3:
        raise ValueError("Expected uint8 HWC with three BGR channels")
    if min(bgr.shape[:2]) == 0:
        raise ValueError("Image must have pixels")
    rgb: ByteImage = cast(ByteImage, cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB))
    unit_rgb: FloatImage = rgb.astype(np.float32) / np.float32(255.0)
    return unit_rgb

cast(ByteImage, ...) tells the type checker that this OpenCV operation preserves the validated byte dtype. It does not convert data; astype performs the real conversion.

This is range scaling, not a complete model preprocessing recipe. A checkpoint may also require a particular resize, crop, interpolation method, channel mean, or channel standard deviation. Those values belong to its training contract. Do not select them because another model used them.

Nor does dividing encoded RGB bytes by 255 convert them into physical light measurements. For example, ordinary sRGB encoding is nonlinear. If your task requires radiometric reasoning, the color space and transfer function become part of the contract too. Our synthetic mask task makes no claim about scene radiance.

5. Move axes; do not reinterpret their meaning

Many image models accept NCHW: batch, channels, height, width. N counts independent images. Our input (2, 2, 3) becomes (3, 2, 2) after transpose(2, 0, 1), then (1, 3, 2, 2) after inserting a batch axis.

The transpose says “take the old channel axis first, then old rows, then old columns.” It preserves which channel belongs to which pixel. A direct reshape(3, 2, 2) merely regroups the same flattened order and generally creates incorrect channel planes. Equal element count does not make a reshape an axis permutation.

def model_input(unit_rgb: FloatImage) -> FloatImage:
    """Finite float32 HWC RGB [0,1] -> contiguous NCHW batch of size one."""
    if unit_rgb.dtype != np.float32 or unit_rgb.ndim != 3 or unit_rgb.shape[2] != 3:
        raise ValueError("Expected float32 HWC with three RGB channels")
    if unit_rgb.size == 0 or not np.isfinite(unit_rgb).all():
        raise ValueError("Expected nonempty finite image")
    if np.any(unit_rgb < 0) or np.any(unit_rgb > 1):
        raise ValueError("Expected values in [0,1]")
    chw: FloatImage = unit_rgb.transpose(2, 0, 1)
    batch: FloatImage = np.ascontiguousarray(chw[None, ...])
    return batch

None inserts a length-one axis; the ellipsis keeps all existing axes. np.ascontiguousarray ensures values occupy a contiguous layout in the resulting order, copying when needed. A transpose can otherwise be represented through strides: metadata describing how far to move in memory along each axis. Logical shape and physical storage layout are related but distinct.

The verification script checks a meaningful coordinate: batch[0, :, 1, 1] must contain [90/255, 60/255, 30/255]. A shape-only test would let the wrong reshape pass. It also checks that editing the crop did not alter the original.

This prepares a NumPy tensor for a channel-first model boundary. It has not created a PyTorch tensor, transferred anything to a GPU, or applied checkpoint-specific standardization. Those are additional explicit operations.

6. Treat image data and label data differently

A color channel measures an intensity. A segmentation label may instead be a category ID: 0 for background, 1 for object, perhaps 2 for another class. Multiplying a class ID by 0.5 does not describe a half-class.

When resizing a hard label map, use a label-preserving rule such as nearest-neighbor interpolation; do not create intermediate IDs through ordinary image interpolation. When resizing probabilities, interpolation can be meaningful, but the threshold and coordinate conventions still need to be specified. We will keep the next lesson on one fixed grid so this additional choice does not obscure the baseline.

A mask will be a Boolean array with shape (height, width). True means foreground. It has no color-channel axis and no intensity scale. A visualization may draw it orange or encode it as 0 and 255 for an OpenCV operation; that display encoding is not its semantic definition.

7. Your first reproducible check

Run python verify_cv.py from the starter bundle . The image checks confirm channel conversion, copied-crop independence, range, layout, and coordinate identity. They reject incorrect dtypes instead of silently accepting a differently scaled image.

Then change the bottom-right BGR value to [10, 20, 200]. Before running anything, predict the corresponding RGB vector and the channel-first value at row 1, column 1. The answers are [200, 20, 10] and approximately [0.78431, 0.07843, 0.03922]. Update the expected test values deliberately; do not weaken the test to “the shape looks right.”

Next: make a decision for every pixel

You can now state what an image array means and verify that a transformation preserves its spatial identity. In From pixels to masks , we turn intensities into decisions. The difficult question changes: not “did I represent the pixel correctly?” but “does this rule identify the object I actually care about?”