x/image/draw: color space-correct interpolation

[aclements]

The BiLinear and CatmullRom interpolators assume the luminance of the source image's color space is linear. But the vast majority of images are in the sRGB color space, which has a highly non-linear luminance curve. As a result, scaled images can look quite different from the source image.

For example, on a reasonably calibrated monitor (and at 100% scaling in the browser), the left and right columns of the following image have the same average luminance:

But scaling this down by a factor of 2 using x/image/draw yields the following:

Here, the two columns appear very different.

For an in-depth (possibly too in-depth) discussion of this problem and many test images, including the one I used above, see http://www.4p8.com/eric.brasseur/gamma.html. My test program is here: https://gist.github.com/aclements/599107a2e3f187f8a2c0.

A lot of software (including browsers) gets this wrong, and that's really unfortunate. It would be fantastic if Go software using x/image got this right out of the box.

Probably the best solution to this would be to thread color space information throughout the image library. At the other extreme, given the general recommendation to assume sRGB in the absence of other information (since virtually every image created in the past two decades is sRGB), it may make sense to simply assume sRGB when interpolating. We could also do the latter first and then later add more complete color space support, with the default being sRGB. Another option is to add this information to the x/image/draw.Options structure, though I fear that may interfere with later adding proper color space support to image.

/cc @nigeltao. (We discussed this a few weeks ago in person, but I figured I should open an issue so it doesn't get lost.)

[ianlancetaylor]

CC @nigeltao

[rselph]

Just wanted to bump this issue. I'm working on creating test images from golang, and it's a constant headache having to do proper color space encoding myself. It's an issue that the vast majority of casual users are oblivious to, but one that affects image quality profoundly, as @aclements pointed out.

[nigeltao]

Yes, it's a valid feature request. No, I don't know anyone with all three of the time, expertise and inclination to work on it any time soon. Sorry.

[gopherbot]

Change https://golang.org/cl/253497 mentions this issue: x/image/draw: gamma corrected non linear interpolation

[eliasnaur]

Note that this issue doesn't just affect scaling, but also blending; operations that involve image/draw.Over should also operate in linear space.

Also, I haven't seen a suggestion to fix this issue by simply introducing new types: image.SRGBA (and/or image.SNRGBA?), both defined as using the sRGB encoding. The reasons seem compelling:

• Fully backwards compatible, in that image.RGBA/NRGBA retain their behaviour and performance characteristics.
• Side-steps the need for a fully-fledged color space API. Arguably, sRGB is not even a separate color space as much as it is a particular encoding for the RGB color space.
• sRGBA is only really useful for 8-bit color channels, so there won't be an explosion in image types.
• Package image already have separate types for separate color spaces (e.g. image.CMYK).

[pjanx]

• sRGB is a colour space of its own, given by its primaries and a white point, aside from being an encoding of it. There is no singular "RGB color space". Wikipedia suggests that the transfer function is also part of a colour space's definition.
• sRGB is definitely useful with higher bit depths, if only because 8 bits cause very visible banding. Also, Display P3 uses the same transfer function, and could use the same treatment.
• Note that CMYK is four-component and subtractive, major differences.

Another solution that hasn't been mentioned here yet is that you can create a simple adaptor around an arbitrary image.Image that un/applies the gamma-like transfer function in At/Set. Though it will presumably be very slow.

[eliasnaur]

Another solution that hasn't been mentioned here yet is that you can create a simple adaptor around an arbitrary image.Image that un/applies the gamma-like transfer function in At/Set. Though it will presumably be very slow.

Indeed. Explicit image.SRGBA/SNRGBA types may be faster because they can be special-cased in Scale/Draw et al.

[eliasnaur]

Another solution that hasn't been mentioned here yet is that you can create a simple adaptor around an arbitrary image.Image that un/applies the gamma-like transfer function in At/Set. Though it will presumably be very slow.

I implemented this as a proof of concept. Austin's program with sRGB wrappers.

[nigeltao]

Nice proof of concept.

I'd be hesitant to check SRGBA and SNRGBA types into the stdlib (and maintain them forever) until we also have a good idea of how to handle "image.RGBA with Display P3" or "image.NRGBA with an arbitrary ICC color profile" in general. We might be missing out on image/draw and golang.org/x/image/draw special-case optimization, but colorspace.SRGBA doesn't need to live in the stdlib to do its job, for now.

If you want to take it beyond a proof of concept, I'd also try to implement the image.RGBA64Image and draw.RGBA64Image interfaces (added to the stdlib this year in Go 1.17). This should speed things up (avoiding the per-pixel color.Color allocations) once golang.org/x/image/draw gets new code to recognize them (doing that is on my long TODO list, but it'd also be easier once golang.org/x can drop pre-Go-1.17 support), similar to what https://go-review.googlesource.com/c/go/+/340049 and https://go-review.googlesource.com/c/go/+/351852 did for the stdlib (not x) image/draw.