RGB, BGR, and alpha: what a channel actually means.

A transparent logo looks correct in your browser, but develops black edges when you resize it for a model. A red object turns blue after loading. A preprocessing function returns three averages, yet you expected one value at every pixel. These failures can happen before a neural network sees a single example.

They share a cause: a shape tells us how many values exist, but not what those values represent. This lesson connects channel meaning to operations you can inspect. We will reuse the four pixels from lesson 1 , then introduce a separate four-pixel transparency experiment. Every diagram is generated from the same source used by the downloadable examples.

1. A channel is one measurement at each location

For an RGB image, a pixel contains red, green, and blue components. A channel plane collects one component at every row and column. An HWC array with shape (2,2,3) contains four pixels, three planes, and twelve scalar values. It does not contain twelve spatial locations.

RGB means the channel order is red, green, blue. BGR means blue, green, red. BGR, rather than GBR, is the usual OpenCV ordering for ordinary three-channel color loading. These names specify order; they do not, by themselves, specify the color primaries, transfer function, or numeric range. OpenCV documents its color conversions ↗ and image-loading flags ↗ .

Our top-left BGR pixel is [0,0,255]. Reading its first entry as red creates a blue display. Swapping the first and last components produces the RGB representation [255,0,0]. No pixel changes position. Green remains in the middle. A black, white, or gray test image cannot expose this bug because equal red and blue values survive the swap unchanged.

The three grayscale panels show the values in individual channels. White means 255 and black means zero, even in the red panel. A red-tinted preview would use red to visualize that same scalar intensity; it would not add information. Always label whether a plot is a grayscale plane, a tinted plane, or a full-color reconstruction. The orange border belongs to the site, not to the measured data.

At the bottom-right location, RGB is [90,60,30]. Read across the three planes at row 1, column 1: red 90, green 60, blue 30. Conversely, a bright pixel in the red plane alone does not prove the original image contains a red object: a white pixel has all three channels bright.

2. Split and reconstruct before processing

def channel_planes(rgb: ByteImage) -> tuple[ByteImage, ByteImage, ByteImage, ByteImage]:
    """Nonempty uint8 RGB (H,W,3) -> independent R/G/B (H,W), rebuilt RGB."""
    if rgb.dtype != np.uint8 or rgb.ndim != 3 or rgb.shape[2] != 3 or rgb.size == 0:
        raise ValueError("Expected nonempty uint8 RGB HWC")
    red: ByteImage = rgb[:, :, 0].copy()
    green: ByteImage = rgb[:, :, 1].copy()
    blue: ByteImage = rgb[:, :, 2].copy()
    rebuilt: ByteImage = cast(ByteImage, cv2.merge((red, green, blue)))
    return red, green, blue, rebuilt

rgb[:, :, 0] keeps all rows and columns and selects channel zero. Scalar channel indexing removes the channel axis, giving (H,W). Here H and W mean height and width. .copy() makes the plane independent: editing it will not silently edit the original image. See NumPy indexing ↗ and ndarray.copy ↗ .

cv2.merge ↗ assembles equal-sized planes in the order supplied. It does not know that the first plane is called red. Passing (blue, green, red) produces BGR storage. The companion cv2.split separates channels; slicing is often sufficient when you only need one. Our explicit copies serve the ownership contract, not a claim that copying is always necessary.

The full file defines ByteImage = NDArray[np.uint8]. That alias specifies the element type. The docstring and runtime checks specify shape and channel semantics. cast informs a type checker about the OpenCV result; it performs no color conversion at runtime.

Use Visualize to keep this source beside its recorded values. The three returned planes each contain four values. The fourth result is the reconstruction. The verification script checks exact equality with the original and confirms that modifying an extracted plane leaves the source unchanged.

3. Read axis as the dimension being reduced

A reduction combines several numbers into fewer numbers. A mean adds values and divides by their count. The axis argument says which dimensions to combine; the remaining dimensions tell you what the answer still distinguishes. In HWC, axis 0 is rows, axis 1 is columns, and axis 2 is channels. Negative axis -1 refers to the last axis.

For our unit-range red plane, only 255 and 90 are nonzero before division by 255. Its spatial mean is therefore:

def channel_statistics(rgb: FloatImage) -> tuple[FloatImage, FloatImage, BoolPlane, FloatImage]:
    """Unit-range RGB (H,W,3) -> channel means, pixel means, red mask, overlay."""
    check_unit(rgb, 3)
    channel_average: FloatImage = rgb.mean(axis=(0, 1), dtype=np.float32)
    pixel_average: FloatImage = rgb.mean(axis=2, dtype=np.float32)
    red_dominant: BoolPlane = (rgb[:, :, 0] > rgb[:, :, 1]) & (rgb[:, :, 0] > rgb[:, :, 2])
    selected: FloatImage = np.where(red_dominant[:, :, None], rgb, np.float32(0))
    return channel_average, pixel_average, red_dominant, selected

The first output contains those three channel means. The second contains one average per pixel: about 0.33333 at each pure primary and 0.23529 at the bottom-right pixel. It is not a perceptual grayscale conversion. Equal arithmetic contributions from R, G, and B are not a model of equal perceived brightness. A grayscale conversion has a stated color-space convention; use the appropriate documented conversion when that is your intent.

numpy.mean ↗ documents axis, dtype, and keepdims. With keepdims=True, reduced axes remain with length one. For example, spatial means become (1,1,3) rather than (3,); either can broadcast against HWC here, but the retained axes make the intended alignment explicit. PyTorch calls the corresponding argument dim ↗ and uses keepdim without an s. A CHW tensor requires different spatial axes from an HWC array: never copy axis numbers without checking the layout.

4. Where selects values; it does not move pixels

The Boolean mask in the same function asks whether red strictly exceeds both green and blue at a pixel. Parentheses group the comparisons, and & combines them element by element. Python’s scalar and is not the elementwise array operator. Ties fail our strict rule. The four decisions are [[True,False],[False,True]].

This is a deliberately simple color rule, not a reliable red-object detector. Shadows, white balance, sensor response, and colored illumination can change the measurements. A slightly red-biased gray pixel can pass even when you would not call it red. That failure is useful: the name of a rule does not make it a learned semantic category.

np.where(condition, x, y) ↗ chooses x where the condition is true and y elsewhere. All operands must broadcast to compatible shapes. Our (H,W) mask becomes (H,W,1) through [:, :, None]. That final singleton axis repeats one decision across all three color channels. Omitting it usually fails for an HWC image, and some coincidental shapes can broadcast in unintended ways.

The selected output keeps [255,0,0]/255 and [90,60,30]/255, and replaces the other two pixels with black. Shape and coordinates remain unchanged. This is different from Boolean indexing such as rgb[red_dominant], which gathers selected pixel vectors into a shorter array and loses the original two-dimensional grid.

Python evaluates function arguments before calling np.where. It is not a lazy branch that makes an invalid expression safe. For example, putting a division by zero in the unselected expression can still produce a warning while constructing that argument. Use an operation’s supported masking mechanism when computation itself must be guarded. Also, the one-argument form of where returns indices; it is a different use from the three-argument selection taught here.

5. Alpha is opacity, not a fourth color

RGBA stores RGB plus alpha. BGRA stores BGR plus alpha. Alpha zero means fully transparent; alpha one means fully opaque in our floating-point convention. Byte alpha commonly uses 0 and 255 for the same endpoints. Alpha 0.25 allows a 25% foreground contribution under the compositing rule below; it is not a 25% probability that an object exists.

A segmentation probability estimates a class under a model. An alpha matte describes how foreground covers or transmits through a pixel under a compositing model. A hard mask, a soft class probability, and an alpha matte can share a shape while meaning different things. Hair, motion blur, and antialiased edges are examples where fractional alpha can be useful.

With straight alpha, RGB stores the foreground color independently of opacity. Transparent red can still have RGB [1,0,0] with alpha zero. Dropping that alpha exposes the hidden red. It does not render the foreground over a background. A checkerboard in an editor is a background used to reveal transparency; it is not stored image content unless you deliberately flatten it.

Our example assumes an opaque background and linear-light RGB:

def composite_over(foreground: FloatImage, background: FloatImage, alpha: FloatImage) -> FloatImage:
    """Straight LINEAR RGB (H,W,3) over opaque background; alpha (H,W,1)."""
    check_unit(foreground, 3)
    check_unit(background, 3)
    check_unit(alpha, 1)
    if foreground.shape != background.shape or alpha.shape[:2] != foreground.shape[:2]:
        raise ValueError("Foreground, background and alpha must share a pixel grid")
    foreground_part: FloatImage = alpha * foreground
    background_part: FloatImage = (np.float32(1) - alpha) * background
    composite: FloatImage = foreground_part + background_part
    return composite

FloatImage means NDArray[np.float32]. The helper check_unit, visible through Explore file, checks dtype, shape, nonempty input, finiteness, and range. It cannot infer whether values really are linear RGB: that semantic promise belongs to the caller. The explicit shape checks prevent an accidentally shared row or column from broadcasting across the image. Alpha has shape (H,W,1); multiplying it broadcasts one weight over the three channels. No byte arithmetic takes place.

The function returns RGB because the background is opaque and therefore the result is opaque. Combining two partially transparent layers also requires computing the output alpha and interpreting whether output color is straight or premultiplied. The W3C compositing specification ↗ describes the general case. Our limited signature makes the simpler assumption visible.

6. Two subtle contracts: premultiplication and light

With premultiplied alpha, stored color already includes its alpha factor. Red at alpha 0.25 is stored as [0.25,0,0]. Multiplying that value by 0.25 again would darken it to [0.0625,0,0]: opacity was applied twice. Conversely, treating straight RGB as premultiplied can create bright fringes. At zero alpha, the original straight color cannot be recovered from premultiplied zero. Specify which convention a loader, resizer, compositor, and exporter expect.

Filtering transparent images needs this attention because invisible RGB values can leak into neighboring edge colors. A common approach is to filter premultiplied color and alpha consistently, then unpremultiply only where alpha is nonzero when a straight representation is required. This does not excuse mixing color spaces or coordinate conventions. Treat edge halos as evidence to inspect the pipeline, not automatically as a segmentation error.

There is a second, independent issue. Normalizing an ordinary sRGB byte value by 255 changes the numeric range but leaves its nonlinear encoding in place. Linear-light 0.5 is displayed at approximately sRGB 0.73536, or byte 188 after rounding, rather than byte 128. Thus averaging encoded black and white at 0.5 is darker than encoding their linear-light average.

For the linear-light workflow demonstrated here: convert sRGB RGB values to linear light, composite, then encode back to sRGB for display. Do not apply the RGB transfer function to alpha. The full source includes srgb_to_linear and linear_to_srgb, using the piecewise sRGB conversion documented in CSS Color 4’s conversion reference ↗ . The figure uses that encoder for its colored swatches; its printed numbers remain linear RGB. Other rendering systems may deliberately composite in another space, so matching a particular application’s output requires matching its working-space policy.

7. A debugging experiment you can reproduce

Run python verify_cv.py from the updated vision starter bundle . Its tests check the channel round trip, independent storage, exact selection mask, reduction values, alpha endpoints, an intermediate mixture, invalid shape/range rejection, and sRGB round trips. The tests compare compositing against a scalar loop, so a shape that merely looks plausible is insufficient.

Then change one thing at a time:

1. Swap red and blue before splitting. Predict which two planes swap and which stays fixed.
2. Replace axis=(0,1) with axis=2. Predict the output shape before examining values.
3. Change a pure green pixel to [0.4,0.4,0.1]. The strict red rule rejects the tie. Decide whether that matches your task.
4. Composite the same red foreground over white rather than blue. At alpha 0.25, expect linear RGB [1,0.75,0.75].
5. Set alpha to a negative value. Expect an error, not silent clipping that conceals the upstream bug.

8. Build an image-boundary checklist from failure cases

An image loader and a model preprocessor are separate interfaces. At the loader boundary, establish whether an alpha channel was preserved, whether the array is grayscale or color, its orientation and dimensions, and its dtype. At the model boundary, establish channel order, range, layout, resizing, and checkpoint-specific normalization. Avoid a function called normalize_image that silently does all of these with undocumented defaults.

For a transparent input, first decide what the downstream task means. A classifier trained on opaque photographs does not automatically know how to interpret a fourth channel. You might composite over a declared background, train a model that consumes alpha, or reject that input format. Those choices are task decisions, not interchangeable fixes for a shape error. Compositing over black during training and white during inference changes visible edge colors and can introduce a distribution mismatch.

Keep a few synthetic fixtures next to the ingestion code. Use a non-square image with distinctive corners to detect spatial swaps; unequal R and B values to detect channel swaps; alpha values of zero, one, and a fraction to detect dropped or doubled opacity; and a known midtone to detect confusion between range scaling and linear-light conversion. These fixtures complement real photographs because their exact answers are easy to predict.

A screenshot is useful evidence, but it is not a complete numeric check. A viewer may rescale floats, normalize each channel independently, or composite transparency over its own background. Record the display policy in the figure caption and assert the array values before rendering. In particular, separately stretching each channel to its own minimum and maximum can make a weak component look just as strong as a dominant component. Our channel figure uses the same fixed 0–255 range in every plane, so their intensities remain comparable.

Where this connects next

You now have three distinct tools: channel planes for inspecting representation, Boolean masks for selecting locations, and alpha for mixing foreground with a background. Keeping their meanings separate makes later segmentation, augmentation, and deployment pipelines easier to debug.

Return to the mask baseline and ask what its single grayscale channel measures. Then read IoU, Dice, and Hausdorff to see why a convincing colored overlay is not enough to establish segmentation quality. Representation, decisions, and evaluation are separate contracts that must agree.