[Gamut mapping app] Scale LH improvements

[jamesnw]

This updates the Scale LH method, applying a few initial checks-

• If the color is already in gamut, it returns the input color immediately.
• If L=0, it returns black, and if L=1, it returns white.

[netlify]

✅ Deploy Preview for colorjs ready!

Name	Link
🔨 Latest commit	8bdeb25
🔍 Latest deploy log	https://app.netlify.com/sites/colorjs/deploys/65e0ffdf20152900091b116b
😎 Deploy Preview	https://deploy-preview-438--colorjs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[jamesnw]

@LeaVerou This is primarily a demonstration, so I'm fine with it being merged as is, or if you want me to incorporate into the existing method, or if you prefer this to just hang out on a branch for demonstration reasons. Feel free to touch base with me on Discord if that's easiest.

[jamesnw]

I've updated this to apply the 2 changes directly to the LH method, and for the sake of dedicated PRs, I have split out the delta display question into its own PR.

[LeaVerou]

Why not simply return color? There is no reason to convert it to anything, and any conversion is potentially a lossy operation.

[jamesnw]

I did this to match what was happening in the scale method, which returns the color converted to p3. This way, the method returns in a consistent space regardless of whether it starts in gamut or not. Otherwise, the serialized display with switch back and forth between p3 and oklch depending on whether it was in gamut.

(Perhaps the results from all methods should be serialized in a single format- currently some are oklch, and some are p3, making them harder to compare, but that's somewhat orthogonal to the question of whether the Scale LH method should return in a consistent space).

[LeaVerou]

I would prefer this to be a separate method (named something more descriptive than Scale LH 2).

1. The separate conversion hurts performance, and perf is a core advantage of Scale LH.
2. The discontinuity at 0 and 1 could affect quality.

Also, as I said in my comment, there is no reason to convert to P3 at any point. The objective is to be within P3 gamut, not to be in the P3 space.

[facelessuser]

2. The discontinuity at 0 and 1 could affect quality.

I think the reason this change was made is maybe due to a misunderstanding as to what Scale LH is trying to accomplish. I think I was also guilty of not quite understanding what Scale LH is either, and maybe I'm still wrong, please correct me if I'm wrong.

But I think Scale LH is more aimed at being a sane replacement for clip as opposed to a better overall gamut mapping approach. Like if a browser didn't want to opt into a more advanced gamut mapping due to speed concerns, this would be way better. Am I right with my understanding? It is trying to retain as much hue as possible and a reasonable amount of lightness (something clip doesn't really do at all).

[jamesnw]

2. The discontinuity at 0 and 1 could affect quality.

My thinking in applying this is that this is part of the CSS Color 4 spec for (ok)lab and (ok)lch.

If the lightness of an Oklch color is 0% or 0, or 100% or 1.0, the color will be displayed as black, or white, respectively due to gamut mapping to the display.

Because we are wanting to evaluate the method for gamut mapping to the display, I wanted to see what happens when this is applied.

It appears it isn't specified that this would also be the case if the lightness of an p3 color is 100% when converted to oklch, so perhaps I'm mistaken on that.

I'm fine creating a separate method, but my intention is to apply my understandings of the spec to Scale LH, and if those understandings are incorrect, then my alternate method wouldn't be needed anyways.

[LeaVerou]

@jamesnw

I did this to match what was happening in the scale method, which returns the color converted to p3. This way, the method returns in a consistent space regardless of whether it starts in gamut or not. Otherwise, the serialized display with switch back and forth between p3 and oklch depending on whether it was in gamut.

I don't have time to check the code rn, but if scale can return the original color when it's unchanged without doing extra work, it should.

(Perhaps the results from all methods should be serialized in a single format- currently some are oklch, and some are p3, making them harder to compare, but that's somewhat orthogonal to the question of whether the Scale LH method should return in a consistent space).

We should not be comparing GMAs by looking at coordinates (or eyeballing, which seems to be the primary method rn!). There are established metrics to evaluate color difference. Humans are notoriously bad in comparing sets of multiple variables (which is why decision making frameworks involve scorecards etc).

@facelessuser

But I think Scale LH is more aimed at being a sane replacement for clip as opposed to a better overall gamut mapping approach. Like if a browser didn't want to opt into a more advanced gamut mapping due to speed concerns, this would be way better. Am I right with my understanding? It is trying to retain as much hue as possible and a reasonable amount of lightness (something clip doesn't really do at all).

Yes and no. That was the original motivation behind Scale LH, but then when we saw the results in practice, I wondered if we could have our cake and eat it too.

[facelessuser]

We should not be comparing GMAs by looking at coordinates (or eyeballing, which seems to be the primary method rn!). There are established metrics to evaluate color difference. Humans are notoriously bad in comparing sets of multiple variables (which is why decision making frameworks involve scorecards etc).

I agree the coordinates are less meaningful. Near the end of my journey in finalizing the ray trace GMA concept, ∆L and ∆h specifically were key to honing in on what was absolutely necessary. Dialing those in, in turn, gave visually good results. Visually good results did not always yield low ∆L and ∆H values. Trying to tune things specifically to visuals is far more difficult.

I don't find the ∆E as meaningful except to see if there is a huge deviation between algorithms. I think it is good to keep though, just not necessarily THE metric to judge an algorithm on. When comparing at the distancing we are at times with GMAs, I think if they are roughly in the ballpark of each other (the different GMAs) it is a good sign you are close to where you need to be.

[LeaVerou]

I’m not arguing that ΔE is the metric at all. I’m only arguing that we need a metric that is a single number. Perhaps that single number could be a weighed sum of ΔL and ΔH (and even ΔC with a much reduced weight), maybe you have a sense of what that could look like now that you have all this empirical data?

[facelessuser]

I don't know if I have a minimum range yet. I do think certain lightness ranges can tolerate more hue shift. Like really dark colors can tolerate more, and maybe really really light. If anyone is interested here is a comparison of approaches, with maximum deviations broken up from about 25 increments of lightness. I went ahead and threw out white and black.

No one is beating the LUT-based one 🙃.

---- GMA clip ----
Lightness Range: 1 - 25
∆L = -0.5818861225325046
∆h = -85.70468709265549
Worst ∆h offender => oklch(1% 0.8 58)
---- GMA clip ----
Lightness Range: 25 - 50
∆L = -0.3264978106839201
∆h = 67.54186726919608
Worst ∆h offender => oklch(49% 0.8 96.5)
---- GMA clip ----
Lightness Range: 50 - 75
∆L = -0.15711774991465632
∆h = 67.68807682280118
Worst ∆h offender => oklch(51% 0.8 97)
---- GMA clip ----
Lightness Range: 75 - 100
∆L = 0.3414259248557022
∆h = 62.05787577722822
Worst ∆h offender => oklch(75% 0.8 91.5)


---- GMA css ----
Lightness Range: 1 - 25
∆L = -0.012835737752495913
∆h = 63.344331062322226
Worst ∆h offender => oklch(2% 0.8 92.5)
---- GMA css ----
Lightness Range: 25 - 50
∆L = -0.01295842629077093
∆h = 11.86376412962801
Worst ∆h offender => oklch(25% 0.8 63)
---- GMA css ----
Lightness Range: 50 - 75
∆L = 0.018031608064365656
∆h = 6.327892650317779
Worst ∆h offender => oklch(50% 0.8 58)
---- GMA css ----
Lightness Range: 75 - 100
∆L = 0.018899920612734822
∆h = -25.93857150413521
Worst ∆h offender => oklch(99% 0.8 53)


---- GMA scale-lh ----
Lightness Range: 1 - 25
∆L = -0.27090400380756646
∆h = -61.120359201229576
Worst ∆h offender => oklch(1% 0.8 44)
---- GMA scale-lh ----
Lightness Range: 25 - 50
∆L = -0.10270081995869595
∆h = 32.27708621939274
Worst ∆h offender => oklch(32% 0.8 50.5)
---- GMA scale-lh ----
Lightness Range: 50 - 75
∆L = 0.06660010637209557
∆h = 26.679923189280487
Worst ∆h offender => oklch(50% 0.8 72.5)
---- GMA scale-lh ----
Lightness Range: 75 - 100
∆L = 0.09960618640826902
∆h = -7.312037543819315
Worst ∆h offender => oklch(75% 0.8 41.5)


---- GMA scale ----
Lightness Range: 1 - 25
∆L = -0.7179863095519733
∆h = -67.85428810497132
Worst ∆h offender => oklch(1% 0.8 37.5)
---- GMA scale ----
Lightness Range: 25 - 50
∆L = -0.4273148682729661
∆h = 46.83453645865734
Worst ∆h offender => oklch(49% 0.8 61.5)
---- GMA scale ----
Lightness Range: 50 - 75
∆L = -0.2230610034656293
∆h = 61.249725477078414
Worst ∆h offender => oklch(72% 0.8 83.5)
---- GMA scale ----
Lightness Range: 75 - 100
∆L = 0.17098352357626057
∆h = 61.072541242219415
Worst ∆h offender => oklch(75% 0.8 84)


---- GMA raytrace ----
Lightness Range: 1 - 25
∆L = -0.00043705105815006196
∆h = 2.092343973841537
Worst ∆h offender => oklch(24% 0.8 40.5)
---- GMA raytrace ----
Lightness Range: 25 - 50
∆L = -0.0012679296769286807
∆h = 2.524405165629105
Worst ∆h offender => oklch(30% 0.8 48)
---- GMA raytrace ----
Lightness Range: 50 - 75
∆L = 0.0015509288153185974
∆h = 2.2250766481119513
Worst ∆h offender => oklch(50% 0.8 49.5)
---- GMA raytrace ----
Lightness Range: 75 - 100
∆L = 0.01166337674294049
∆h = -4.152382874428781
Worst ∆h offender => oklch(88% 0.8 85)


---- GMA edge-seeker ----
Lightness Range: 1 - 25
∆L = -0.0000013564776892849295
∆h = 0.001437609406309548
Worst ∆h offender => oklch(5% 0.8 38)
---- GMA edge-seeker ----
Lightness Range: 25 - 50
∆L = -0.000002769475282438094
∆h = 0.001437609406309548
Worst ∆h offender => oklch(26% 0.8 38)
---- GMA edge-seeker ----
Lightness Range: 50 - 75
∆L = 0.0000073318044458980935
∆h = 0.001437609406309548
Worst ∆h offender => oklch(52% 0.8 38)
---- GMA edge-seeker ----
Lightness Range: 75 - 100
∆L = 0.0021721492913789886
∆h = 1.3964618440284653
Worst ∆h offender => oklch(99% 0.8 109)

[jamesnw]

I don't have time to check the code rn, but if scale can return the original color when it's unchanged without doing extra work, it should.

scale creates a color in p3-linear, and is currently converting back to p3. To reduce the amount of conversions, I could have scale return p3-linear, and scale-lh return the original color space if already in gamut, and p3-linear if not in gamut. Does that sound good?

Separately, let me know if the white/black conversions are outside the scope for what these gamut mapping algorithms are intended to do, and I can remove that part as well.

[jamesnw]

I do think certain lightness ranges can tolerate more hue shift. Like really dark colors can tolerate more, and maybe really really light.

I was curious how the current algorithms were shifting hues, and so I ran some colors through them.

Pardon the unlabeled graphs, but the x-axis is lightness from 0 to 1, and the y-axis is the delta of H.

For css, it looks like there is a lot more hue shift around the edges, corroborating your theory.

Compare that to scale-lh, where there's more hue shift in the darker colors-

For raytrace- there's a similar dip to css at lighter colors, although it is not as strong a correlation, and I suspect the point sampling is creating some potential pattern-like noise.

edge-seeker seems to have little hue shift happening until lighter colors-

I do think this points to potentially making the acceptable delta for H a function of L.

[facelessuser]

it looks like there is a lot more hue shift around the edges, corroborating your theory.

Very interesting. I hadn't yet evaluated the data to this length yet, so it is interesting to see some of my thoughts more strongly confirmed.

The good news is that faster, more accurate approaches to the current CSS recommendation/example are possible. I also think specifying some minimum metric that should be met is probably not a bad idea as it seems there will be strong opinions as to the best way to approach it: complexity vs speed vs accuracy. Someone is always going to come along and say, "I can do it better!".

I am interested to see if other novel approaches surface as well. I've found this area to be interesting 🙂.

[LeaVerou]

@jamesnw wow, these are fascinating! What is the vertical axis?

@facelessuser While the ranges are useful, I was asking if you had a sense of what kind of weighted sum of deltas might give us a good single-dimension measure of proximity.

I.e. a * ΔL + b * ΔC + c * ΔH -> what should a, b, c be?

---

I’m also wondering if I should re-instate Scale LH+L, where there's an extra step where it sets lightness to the original color and scales again, perhaps conditionally if the shift is above a certain level. It had worse ΔΕs so I rejected it early, but it might not have this issue?

[facelessuser]

@LeaVerou Yeah, I realized what you asked, I just don't have an answer yet. I probably didn't make that clear. I had not yet considered deriving such a metric, though I admit such a metric would be useful. I would need more time to consider the question. I had not tried to see how much slack in the system I could get away with, but more how can I get as close as possible as quick as possible in the least complex way. So I shared some interesting musings instead :).

[facelessuser]

I’m also wondering if I should re-instate Scale LH+L, where there's an extra step where it sets lightness to the original color and scales again, perhaps conditionally if the shift is above a certain level. It had worse ΔΕs so I rejected it early, but it might not have this issue?

@LeaVerou Your intuitions were right, the extra step of correcting only L increases ∆h more significantly in the upper lightness range.

Results

### Scale LH + L
---- GMA scale-lh ----
Lightness Range: 1 - 25
∆L = -0.07503574050278539
∆h = -74.45715049173208
Worst ∆h offender => oklch(1% 0.8 39)


---- GMA scale-lh ----
Lightness Range: 25 - 50
∆L = -0.035192086262099465
∆h = 34.64706531106583
Worst ∆h offender => oklch(32% 0.8 50.5)


---- GMA scale-lh ----
Lightness Range: 50 - 75
∆L = -0.028146477184406216
∆h = 28.464790372272375
Worst ∆h offender => oklch(50% 0.8 72.5)


---- GMA scale-lh ----
Lightness Range: 75 - 100
∆L = 0.06848855048073144
∆h = -6.724383793889388
Worst ∆h offender => oklch(75% 0.8 46.5)
### Scale LH + L

---- GMA scale-lh ----
Lightness Range: 1 - 25
∆L = -0.27090400380756646
∆h = -61.120359201229576
Worst ∆h offender => oklch(1% 0.8 44)


---- GMA scale-lh ----
Lightness Range: 25 - 50
∆L = -0.10270081995869595
∆h = 32.27708621939274
Worst ∆h offender => oklch(32% 0.8 50.5)


---- GMA scale-lh ----
Lightness Range: 50 - 75
∆L = 0.06660010637209557
∆h = 26.679923189280487
Worst ∆h offender => oklch(50% 0.8 72.5)


---- GMA scale-lh ----
Lightness Range: 75 - 100
∆L = 0.09960618640826902
∆h = -7.312037543819315
Worst ∆h offender => oklch(75% 0.8 41.5)

I think the real issue is the approach, it was never going to net you much lower than what you got. I did try. The way in which Scale LH is scaling the colors just can't correct as well, but the idea of what it is doing is what sparked something in my brain.

Make no mistake, the fancy "raytrace" name I gave my method and even the new function is no different than what I started out with, functionally it is the same, so much so that you can replace the current function with the inverse interpolation approach and get the same results. And, the whole idea was inspired by Scale LH. Force max saturation of the color in RGB, but correct it in OkLCh.

The original idea was to use inverse interpolation to figure out the value needed to force the channel with the greatest out of gamut magnitude to either 0 or 1 (whichever is closest) with interpolation instead of scaling to the midpoint. This is basically finding the intersection of the line through the achromatic color to the color of interest and the RGB cube surface. We were just calculating what interpolation value is needed to get us there, and then calculating the color at that point. The new method does the same thing, just in a different way. I'm not even sure which approach is actually faster. Using this approach and keeping the same flow of Scale LH immediately produced better deltas. The approach of scaling the saturation is what made all the difference.

Results

---- GMA Max Sat ----
Lightness Range: 1 - 25
∆L = -0.0003019608152752262
∆h = 6.388265750000983
Worst ∆h offender => oklch(24% 0.8 40.5)


---- GMA Max Sat ----
Lightness Range: 25 - 50
∆L = -0.0005956533026971744
∆h = 17.159961817469252
Worst ∆h offender => oklch(49% 0.8 71.5)


---- GMA Max Sat ----
Lightness Range: 50 - 75
∆L = -0.0007738512478001613
∆h = 17.309076528961782
Worst ∆h offender => oklch(51% 0.8 73.5)


---- GMA Max Sat ----
Lightness Range: 75 - 100
∆L = 0.00007199020318604532
∆h = 1.8158473254680416
Worst ∆h offender => oklch(75% 0.8 97.5)

But I thought we could do better, so I thought, let's correct LH twice and scale one more time. It did even better, but the approach gives max saturation when decreasing chroma, so often the final scale did nothing. It doesn't increase chroma when we've overcorrected due to being on the RGB cube surface and correcting LH such that it puts you below the surface. This is where I got stuck for a bit. I thought, let's back off and maybe approach "softer", but you'd end up relatively in the same spot, close, but not quite right. Maybe yellows in some regions were a little more orange than you'd like.

So it occurred to me that I was close to the surface, but the chroma line back to the original color is no longer of use to me anymore because the colors don't change the same way in RGB as they do in OkLCh, but I was close to the surface, much closer to the real color I was after. That's when I realized if I just extended the line from the achromatic point through the new point, back outside the cube, ignoring the original color point, I could find the intersection one more time and I'd be closer to the actual color I wanted.

You can actually stop there, I think you get a max ∆h ~8 across all lightness at that point. Maybe that's good enough, and you'd be faster, but the perfectionist in me said, nah, correct LH one more time and then clip, now we have ∆h ~4.,
Essentially, the way in which we saturate the color in RGB had to change in order to get lower ∆L and ∆h. It's really not much different than Scale LH despite the name.

[facelessuser]

I take it back, Scale LH does improve considerably, but you must correct both L and H in both iterations. It still saturates high lightness more than other approaches, but it does get much better at preserving hue. I may have stepped away from Scale LH when I saw larger improvements with scale towards achromatic and doing the same thing, I also found the high saturation of high lightness difficult to work with when generating tones via interpolation.

---- GMA css ----
Lightness Range: 1 - 25
∆L = -0.012835737752495913
∆h = 63.344331062322226
Worst ∆h offender => oklch(2% 0.8 92.5)


---- GMA css ----
Lightness Range: 25 - 50
∆L = -0.01295842629077093
∆h = 11.86376412962801
Worst ∆h offender => oklch(25% 0.8 63)


---- GMA css ----
Lightness Range: 50 - 75
∆L = 0.018031608064365656
∆h = 6.327892650317779
Worst ∆h offender => oklch(50% 0.8 58)


---- GMA css ----
Lightness Range: 75 - 100
∆L = 0.018899920612734822
∆h = -25.93857150413521
Worst ∆h offender => oklch(99% 0.8 53)


---- GMA scale-lh ----
Lightness Range: 1 - 25
∆L = -0.09254326460637521
∆h = 53.93489660645503
Worst ∆h offender => oklch(5% 0.8 77.5)


---- GMA scale-lh ----
Lightness Range: 25 - 50
∆L = -0.03673338371658891
∆h = 17.105435595667586
Worst ∆h offender => oklch(38% 0.8 58)


---- GMA scale-lh ----
Lightness Range: 50 - 75
∆L = -0.027776454034458053
∆h = 9.272932279758493
Worst ∆h offender => oklch(50% 0.8 67.5)


---- GMA scale-lh ----
Lightness Range: 75 - 100
∆L = 0.06855749821462176
∆h = 3.432795105105356
Worst ∆h offender => oklch(75% 0.8 41.5)

[jamesnw]

What is the vertical axis?

It's the delta between oklch.h of the orginal and mapped colors, normalized in the same way as the Gamut Mapping app.

[svgeesus]

But I think Scale LH is more aimed at being a sane replacement for clip as opposed to a better overall gamut mapping approach.

Yes, absolutely; it is the "please at least do something less broken" method (and is surprisingly good, for that).

[facelessuser]

Yes, absolutely; it is the "please at least do something less broken" method (and is surprisingly good, for that).

I agree, it does a great job as a clip replacement.

[jamesnw]

I realize I took this on a bit of a sidetrack here, but returning to the proposed changes on the PR, what changes should be made to Scale-LH?

1. Return immediately if the input color is in gamut (to prevent cases like CSS keyword red [getting mapped] (https://colorjs.io/apps/gamut-mapping/?color=red)out of the sRGB gamut). Outstanding question- what color space should this be returned in? The input color space, meaning no conversion, but different output spaces depending on the input?
2. Should this algorithm also return white or black for 1/0?

[facelessuser]

In my opinion.

1. I think it should short-circuit out if a color is already in gamut. While some gamut mapping algorithms can do this, it sounds like, as a clip alternative, it should not touch in gamut colors.
2. Based on the clarification it is a clip alternative, and due to the way it works in extreme lightness, I don't think it makes sense to apply the same constraints on black and white. It does create a discontinuity, it just isn't designed to work like traditional gamut mapping.

[jamesnw]

Ok, I have implemented it with returning in the input color space, with no white/black.