Saturday, June 27, 2020

Visualizing Color Space in 2D

I present here an optimized visualization of a perceptually uniform 3D color gamut in 2D space, or "color map", and show a few ways it can be used. It is designed to provide persistent awareness of position within color space, so that it can be navigated, learned, and utilized more effectively by practitioners of digital color, such as painters, photographers, illustrators, etc.

This is a follow-up to my previous "Color Navigation Map" article, with the same goals but an alternate approach. In my view, this approach fully supersedes the previous one, although no single "best" visualization exists. Like a map projection, it represents a balance of competing tradeoffs.



5.0 JND, 541 colors

4.0 JND, 1,052 colors

3.0 JND, 2,509 colors




2.0 JND, 8,434 colors


1.0 JND, 67,455 colors

Technical Summary

The method used to generate these color maps is as follows:
  1. Generate a preliminary 5-bit color gamut in RGB space.
  2. Convert the gamut to Lab space under the standard D65 illuminant.
  3. Calculate the boundary of the gamut point cloud, obtaining an irregular concave polyhedron.
  4. Generate a final color gamut within the polyhedron, using a fixed ΔE resolution for perceptual uniformity.
  5. Section the final gamut at each L value.
  6. Convert each section to an RGB image, with exactly one pixel for each color.
  7. Arrange the sections laterally, with a axes horizontal, b axes vertical, and L axes aligned vertically.


By design, the final color map is extremely compact and efficient:




Why Map Color?

To begin, it bears mentioning why color mapping is worthwhile in the first place. In essence, it does for color what a geographical map does for global positioning. It tells the navigator where they are in space, what else is around, and in which direction to travel to reach their destination. Imagine how difficult navigation would be without maps. We are effectively "mapless" when it comes to color, which makes it harder than necessary to use and understand. Solving this problem is my goal.

Which Gamut To Map?

Now, some discussion on scope. For practical reasons, I attempt to visualize only the set of colors capable of being displayed by standard computer/phone screens. These screens generally show color in the RGB (red-green-blue) format with 8-bit resolution for each channel. Altogether, this paradigm provides 256^3 = 2^8^3 = 2^24 = 16.7 million unique colors. While RGB color is linear with respect to colored light, human perception of color is highly irregular and nonlinear. In other words, RGB is a good color model for screens, but not for people. It is therefore natural to introduce the Lab color space which by design is perceptually uniform.

Perceptual uniformity means that physical distance between color corresponds 1:1 with their perceived color distance (at least in theory - there are limitations). It's important to realize that color measurement can always be interpreted as geometry, e.g. a pixel's color can be plotted as a point in 3D space, and a gamut (set of colors) can be plotted as a point cloud in 3D space. Euclidean (straight-line) distance in Lab space is a very good measure of perceived color distance, and is conventionally denoted as ΔE, with the letter "E" tracing back to the original German empfindung for "sensation". It is also helpful to think of color distance in terms of JNDs, or just noticeable differences, commonly defined as 2.3 ΔE.

Filling the RGB gamut in Lab space with uniformly-distributed colors at ΔE = 2.3 = 1.0 JND generates about 67,000 unique colors. Compare this to the 16.7 million colors in the RGB gamut, and you'll see that perceptually speaking, we can do with many fewer colors. In fact, a gamut about 1/250 the size of the standard 8-bit RGB gamut is perceptually "complete", give or take. Of course, there are good practical reasons why a larger gamut is used. But for my purposes they are not relevant, so I use a smaller, perceptually uniform gamut instead.

As for the "best" ΔE value to use, it depends on the application. In some cases, a lower resolution is actually preferable, and a good example of this can be found in hardware stores paint swatches. It is actually helpful for there to be fewer choices, up to a point, because they are more differentiable, and more consideration can be given to each. In other cases, where very accurate matching is desired, a higher resolution will be preferable. For my purposes, primarily as an oil painter, I prefer a ΔE of 4.6, or 2.0 JND.


Hardware store paint swatches: a good example of the benefits of low resolution (Bill's True Value)

In short, the gamut I choose to map represents a perceptually uniform subset of all colors capable of being displayed by modern screens, with the resolution tailored to fit the application.

Visualization Goals

With the gamut established, the next task is deciding how to best visualize it. I stipulate that it be in 2D, i.e. that it be a simple image, so that a user can see all colors all at once without having to rotate, scroll, or otherwise interact. I also want it to be maximally continuous, consistent, and intuitive. By this I mean that colors should be near their closest neighbors, placed using the same set of "rules", and understood with minimal explanation.

Given the 2D constraint, it is not possible to be maximally continuous, consistent, and intuitive, all at once. In other words, it is not possible to meaningfully reduce the rank of color from 3D to 2D, in any general sense, without irreversible loss of information. This would be a true breakthrough, so it is not surprising that it has stubbornly resisted all attempts to "crack" it, from me and all previous color theorists.

After considering many visualization approaches, some of which are captured in the Appendix, I settled on a simple arrangement of cross-sections. This inherently introduces discontinuities (between cross-sections), but their quantity is minimized; otherwise this visualization is extremely consistent and intuitive.

Map Design

Given the cross-section approach, the next question is how to section, and how to arrange the sections. To minimize discontinuity, the natural choice is to take the sections along the axis with the smallest range. At 1.0 JND, the RGB gamut in Lab space has the following domains:

  • L: 2.3 to 98.9, 42 JND
  • a: -83.8 to 95.5, 78 JND
  • b: -105.6 to 92.2, 86 JND
Note that the size of the domains vary significantly by axis. Because of the perceptual uniformity of Lab space, position and direction are irrelevant to color distance - only the Euclidean distance matters. Therefore, it's completely valid to compare ΔE values regardless of which axis/axes they occur along.

The interpretation of this domain data is interesting: it is saying that L, or value/brightness, is the least perceptually consequential component of color. What we think of as hue (color name) and saturation (color intensity) are about twice as perceptually important. Hue and saturation are defined in Lab space as angle and distance about the L axis, which is also the "neutral" axis because all purely gray shades occur along it. The LCh color space uses these more intuitive coordinates, but it's fundamentally describing the same data. This is the same as how a point in 3D space can be described in Cartesian or spherical coordinates, and still refer to the same point.

RGB color gamut in Lab space, polyhedral boundary shown as gray mesh, uniformly filled

This indicates that the optimal section plane is a-b, i.e. perpendicular to / along the L axis, which has the smallest domain. Further, it suggests that the gamut is not exactly square, and that b is generally larger in magnitude than a, which turns out to be true. Therefore, to minimize the size of the color map image, the sections should be arranged with the b axis vertical, for better packing. This also lends itself nicely to having a horizontal and b vertical, which is more intuitive than vice versa due to the alphabetical Cartesian x-y convention. I arrange the sections such that their neutral axes are all aligned, they are as close together as possible without overlapping, with a and b positive corresponding to right and up, and with L increasing across sections from left to right.

Map Properties

To validate the method and build intuition about the visualization, it's worthwhile looking at the color map on a channel-by-channel basis.

 Lab hue channel

Lab hue corresponds to angle about the L/neutral axis, with an artificial appearance of a seam where 0° meets 360°. In reality, the two angles are equivalent, as in a circle.

 Lab saturation channel

Lab saturation corresponds to normal distance from the L/neutral axis, so each section has a circular appearance. Note that highly saturated colors occur more at higher values.

Lab value channel

Lab value is constant within sections, so there is a simple linear relationship across sections from left to right, progressing from darkest (black) to lightest (white).

Lab a channel

a is the horizontal distance within the section, corresponding to a theoretical green-magenta axis.

Lab b channel


b is the vertical distance within the section, corresponding to a theoretical blue-yellow axis. It is loosely accurate to think of it as a color temperature axis, from cool to warm.

Map Usage

The next question that should rightfully be asked is: so what? To answer that, I show three cases for how this color map can be used for interacting with color.

Case 1: which color should I pick?

I am not a fan of conventional color pickers such as those found in Photoshop, GIMP, etc., because they are fundamentally rooted in 2D and are disorientating to use.


Picking a color in GIMP in HSV space

Here, the color map has the advantage of keeping all colors visible and intuitively arranged at all times. The user can pick freely, find neighbors easily, and visualize how their selection fits within the broader gamut.


Demo of picking arbitrary colors from the color map

Case 2: where is this color within space?

Because of how relativistic and cognitive color perception is, it is often very challenging to actually identify "what" a color actually is. Its context can have a shockingly significant effect on how it is perceived, as many well-known optical illusions neatly show. It may be desirable to gauge or counteract these effects by showing where an arbitrary color, e.g. from an image, occurs within the color map.


Checker shadow illusion: A and B are the exact same color!

Because the color map has many fewer unique colors than images do, locating an arbitrary color on the color map is done by error minimization. It is approximate, not exact. In many cases, such as mine, exact matching is not important.


Demo: locating arbitrary image colors within the color map

Note that when the color is displayed as a standalone swatch and dissociated from its context, its appearance can change dramatically. For example, note that the cloud shadows are actually beige-brown, but appear gray in the context of the image.


Here, the dissociation shows that the grassy field, which "reads" as beige-brown, is actually better described as a strong orange.

Case 3: how is this image working?

A user may wish to better understand how a image is working, i.e. visualize how it is using the color gamut to achieve its effect(s). This is rarely trivial, so another case where visualization can provide insight.

The error minimization approach described above is fine for a single color, but for a whole image it is prohibitively slow. To work around this, I define a Lab precision in units of ΔE, round both image and map to the precision, then simply match exactly between the image and map. This is more approximate but operates dramatically faster. I set ΔE such that at least 95% of image colors are matched to map colors, which is typically around ΔE = 10. A small number of image colors are not matched and not shown on the map.

Here are some sample images, with the corresponding "usage" map beneath each, and some insights that can be gained from each. Any map color that is not used at least once is shown as "grayed out". Showing unused colors is important for a couple reasons: which colors are not used can be just as insightful as which colors are used, and the unused colors help visually position the used colors within the section.


  • Good value range - uses the gamut relatively fully without saturating limits
  • Saturated darks and highs, neutral mids
  • Color contrast primarily coming from b (blue-yellow or vertical) axis, minimal usage of a (green-magenta) axis for color differentiation



  • Nearly saturated on low end of value
  • Very warm palette - almost all values for b are positive
  • High saturation throughout value range


  • Value nearly saturated on both ends of range
  • Fairly constant green-yellow hue over value range
  • More saturation along b than a
  • Very warm temperature - all b values are positive, except in darks which are cooler


  • Very low saturation throughout image, clustering around neutral axis
  • Low end of value range is saturated
  • Image primarily based on value contrast
  • Higher saturation at higher value (sky region)


Results

Below are native-resolution (1 pixel per color) versions of my color map at various color resolutions for download. They are rendered in PNG format with a transparent background. These differ from the color maps shown in the introduction, which were upsampled for web display.


5.0 JND, 541 colors


4.0 JND, 1,052 colors


3.0 JND, 2,509 colors


2.0 JND, 8,434 colors


1.0 JND, 67,455 colors

Appendix: Alternate Sub-Optimal Visualizations



Along the way to developing this approach, I explored many alternate ideas, some of which are shown and briefly discussed here in chronological order.



Generated by sorting rows and columns according to various color properties. Depending on the sort parameters, some pleasant results are possible but not very useful. The common pitfall is noise/discontinuity and trying to fit the irregular gamut into a regular square.



A histogram approach, with the x axis representing hue. This looks good, but doesn't hold up under closer inspection, as within a single hue "bin", there is no good method of sorting value and saturation along a single dimension. Again, noise is a problem, though it is better hidden here.



A Mercator projection of the Lab gamut idealized on a spherical shell. The problem here is that saturation gets "left behind" by the geometric transforms, so it appears throughout the map as discontinuous sparse gray patches.



A projection of the gamut point cloud onto the a-b plane using a particle simulation to represent a projecting, divergent force. It works well initially, but falls apart as it works outward due to "leftover" colors with highly discontinuous values.




A compromise using a globe projection, wherein the color gamut is partitioned into "high" and "low" saturation hemispherical subsets which intersect at at the equator, exploiting the singularity at pure black for hemispherical continuity. This enables continuous encoding of hue and saturation, with 1-bit encoding of saturation. This approach could be extended to N "hemispheres" (here, N=2), but at the cost of sacrificing the intuitive globe framework.



A "pinwheel" model wherein N circular sectors have similar hue, and within a sector, radius corresponds to saturation and angle corresponds to value. This is not a bad visualization, but is fairly discontinuous and non-uniform.



Another particle simulation with attractive, repulsive, and cohesive forces simulated based on physical and color distance between particles. This produces interesting animations and results, but the result ultimately is not that good, and the axes essentially have no meaning.



A test of using a Sobel edge detection filter as a cost function to be minimized by random pixel swaps. Again, this method encodes no color information with the axes, merely arranges colors, and the cost function is prone to exploitation.



A few variations on the "rainbow smoke" algorithm which starts with a sparsely seeded image, then grows it outward pixel by pixel using color difference minimization at the border. I experimented with various seeding approaches and was somewhat successful at encoding hue with angle, but again this merely amounts to color arrangement, which may look nice but doesn't encode meaningful information. It is also very computationally expensive.

Sunday, May 17, 2020

Color Navigation Map

Overview

In this article, I introduce a new and useful tool for practitioners of digital color called the Color Navigation Map (CNM). Inspired by geographical maps, the CNM displays a perceptually-uniform and -complete subset of the RGB color gamut in a single image, with colors arranged continuously, consistently, and intuitively. It provides persistent awareness of position within color space, so that it can be navigated, learned, and utilized more effectively.


Color Navigation Map (CNM): ~67,000 perceptually-uniform and -complete colors, spanning the full RGB gamut

Technical Summary

The CNM samples the full RGB color gamut polyhedron in Lab space with a perceptually uniform discretization of 2.3 ΔE or 1.0 JND (just noticeable differences). It uses CIELCh hue, saturation, and value to map colors within the intuitive spherical system of meridians, hemispheres, and parallels. The singularity at pure black is exploited to bifurcate the gamut into two continuous and equal hemispherical subsets of "high" and "low" saturation, by scaling the gamut polyhedron along its "a" and "b" axes. This approach enables 1-bit encoding of saturation as hemisphere, and continuous encoding of hue and value as azimuth and elevation. A continuous spline histogram equalization algorithm is used to evenly distribute hues between the full range of azimuth angles. The equal-area Equal Earth map projection is used to flatten the spherical coordinate system into 2D, and a Voronoi diagram is used to convert the sparse points to a dense image. The CNM has constant hue along meridians, constant value along parallels, similar saturation in each hemisphere, and roughly equal representation for all hues.


CNM annotated with key features

Dimensionality and Lessons from Mapmaking

Fundamentally, digital color is 3D. Whether described in RGB, HSV, XYZ, Lab, or any number of other color spaces, all colors are defined by a unique triad of independent values. Meanwhile, screens are 2D. This inherent rank mismatch leads to significant awkwardness in working with digital color. Selecting a color is akin to leafing through a whole book to find a specific illustration, instead of simply picking it out from many illustrations hanging in gallery. Disorientation in color space is therefore very natural, and a key challenge to using color effectively.

Technically speaking, the task of continuously arranging all colors in 2D, with no gaps or exclusions, is not possible. However, mapmaking has shown that technical limitations need not preclude practical solutions: Earth's surface cannot be fully represented in 2D, yet map projections have successfully facilitated navigation for centuries. In terms of rank, color is more complex even than cartography  whereas an azimuth/elevation pair corresponds to a single value like altitude on a globe, in LCh space, any pair of properties corresponds to multiple values. This is like the difference between a solid spherical ball, and a hollow spherical shell.

Images Containing All Colors, and Their Limitations

This project originated from Bruce Lindbloom's "An RGB Image Containing All Possible Colors", which takes the RGB B value like a "page" index and creates 256 sub-images for each possible value.


Bruce Lindbloom's "An RGB Image Containing All Possible Colors"

I immediately appreciated the utility of the image, but wondered if the colors could arranged in a more useful way. After some experimentation I concluded that some strategic relaxation of constraints would be needed make a truly useful map of all colors. Shown below are some of these experiments, generated by sorting the pixels of the Lindbloom image by various properties.


Rows sorted by Lab hue, columns sorted by RGB B value

Rows sorted by HSV saturation, columns sorted by Lab hue

Although some macroscopic order exists, images created this way are plagued by high-frequency noise which at best creates the illusion of continuity, as if by dithering.

A more recent open-ended competition called "Images With All Colors" challenged coders to best arrange all RGB colors as single pixels as they saw fit, producing a number of interesting results:


"Rainbow Smoke" by József Fejes

"Tileable diamonds"

There is actually a website devoted specifically to this challenge, allRGB.com, showing a number of different approaches and results.


allRGB.com, a website devoted to images containing all colors

However, due to inherent limitations, no image produced under these constraints has continuous, consistent, and intuitive placement of color.

All mapmaking involves compromises and balancing competing priorities, and the "color mapping" problem is no different. After much of trial and error, I eventually developed a method that produces a result that is well-balanced in terms of my "continuous, consistent, and intuitive" guiding principles. To clarify these:


  • Continuous: adjacent colors have minimal color difference
  • Consistent: all colors are treated the same; strong alignment between colors with matching color properties
  • Intuitive: color positions can be predicted from color properties, and vice versa; map is simple and requires minimal explanation

Method Summary

Generate a uniform RGB color gamut. Only the boundary is of interest, so a resolution of about 25 values per channel, or 25^3 = ~15,000 total colors, is sufficient for convergence.

Calculate the polyhedral boundary of the RGB color gamut in Lab space. Note that the boundary is slightly concave, so a convex hull algorithm will provide slightly inaccurate results. MATLAB's built in boundary.m works well for this, provided that a non-zero shrink factor is used to account for the slight concavity.

Uniformly sample the volume inside the polyhedron using a fixed ΔE of 2.3 in L, a, and b. I use inpolyhedron.m for this purpose. In the figures below, I show ΔE = 10 for the sake of illustration. The ΔE value of 2.3 corresponds to one "just noticeable difference" or JND, a reasonably fine discretization of the continuous Lab space. Finer or coarser values of ΔE can be used as needed.


Perceptually uniform colors sampled at ΔE = 10 inside the RGB polyhedron in Lab space

View of ab plane for uniformly sampled colors




Perceptually uniform colors sampled at ΔE = 10 in RGB space


Purge neutral colors, defined as having an LCh saturation < 1.0. This purges "true" neutrals, among them pure white and pure black. Because true neutrals satisfy a = b = 0, and LCh hue is defined as atan2d(b,a), neutrals have an undefined hue and cannot be consistently placed in the map.

Linearly map LCh hue to azimuth angle, and LCh value to elevation.

Fit a monotonically increasing spline to the cumulative distribution function (CDF) of azimuth (hue) angle for continuous histogram equalization, resulting in a roughly even probability density function (PDF). This compensates for the significantly non-uniform distribution of hues within the gamut polyhedron.


Continuous spline histogram equalization of azimuth (LCh hue)


Top view of colors before (left) and after (right) azimuthal histogram equalization

Scale the gamut polyhedron by 69% about the a and b axes to partition the gamut into two hemispherical subsets of roughly equal volume and color quantity. Colors outside the scaled polyhedron are assigned to the upper hemisphere; inside to lower. Applying no scaling to L maintains the common black/white singularities, such that both sub-gamuts converge to the same color at extreme high and low values, a necessary condition for inter-hemispherical/equatorial continuity.


Scaled polyhedron boundary for bifurcation of color gamut into hemispheres

Invert elevation for the inner sub-gamut, i.e. allocate the less-saturated colors to the lower hemisphere.


Colors mapped to unit spherical surface by LCh hue, saturation, and value 

Use the Equal Earth map projection to flatten the spherical coordinate system into 2D. The choice of Equal Earth among other map projections was motivated by its equal-area constraint, symmetry, minimal distortion, illustrative shape, and ease of implementation.




Map projection of colors on spherical surface to flat plane

Create a Voronoi diagram for the map projection; fill cells using the closest color.

Overlay parallel and meridian lines as desired. Set area outside of map projection as transparent.

Properties of the CNM

By design, the CNM has several useful properties intended to make it as continuous, consistent, and intuitive as possible:

  • Meridians (vertical lines of constant azimuth/latitude) have constant LCh hue
  • Parallels (horizontal lines of constant elevation/longitude) have constant LCh value
  • Hemispheres have similar LCh saturation
  • LCh hues have roughly equal representation




Histogram equalization is deliberately not applied to the output CNM image, as the gamut polyhedron is itself not R/G/B balanced due to the perceptual non-uniformity of RGB space. As a consequence of the geometric mapping strategy, the CNM does not have equal area for each color contained, although the degree of non-uniformity is mitigated significantly by the use of azimuthal histogram equation and an equal-area map projection. Equality of area per color may be improved in future iterations, but probably cannot be solved completely without breaking more important CNM constraints.


Using the CNM

Because the CNM does not strictly contain all RGB colors, positioning oneself within it must be done by color difference minimization, rather than exact matching. Because an Lab discretization level of 2.3 ΔE or 1.0 JND was used, any arbitrary input color (e.g. from a photo) can be matched to within 1.0 JND of a point/color within the CNM.

In this demo, I select colors from a photo randomly, and they are matched and positioned on the CNM, providing persistent awareness of position within color space. Here I'm using CIE76 color difference for speed, but CIE94 or CIEDE2000 could be used for better accuracy on faster hardware and/or software.

Note that the GIF format used here for convenience is limited in terms of color depth, so some compression artifacts occur. Also, I've sped up the GIF for the sake of illustration.


Demo of matching photo colors against the CNM

The CNM can also serve as a perceptually complete and persistent palette of colors from which a user can freely select.


Demo of picking colors from the CNM


Compare the simplicity and persistence of this color selection process with the conventional approach found in GIMP and Photoshop:


Conventional color picking in GIMP

To get to any single color in GIMP, you must first adjust a hue slider, then pick an x, y point to define saturation and value. At any single moment in time, you can only see a single hue. Although the conventional approach offers much finer color resolution, it's much less intuitive, not persistent, and does not capture the highly relativistic aspect of color. Picking colors effectively requires awareness not just of the single color at hand, but also how it fits and interacts with all other colors in its scene. This aspect is simply missing from the conventional approach.

In a broader sense, as the name suggests, the CNM is designed for navigation; in other words, to be used as a compositional aid for painters, illustrators, designers, etc., who are interested in planning how their work will use the available gamut of colors and the associated relativistic effects to convey their ideas most effectively. This is analogous to how early explorers used geographical maps to plan their routes according to both the terrain and their goals.

I expect that other uses for the CNM may present themselves over time.

Other Variants

Up to this point, I've been describing what I consider to be the "default" parameters of the CNM, best tailored for my personal use, but many are adjustable. In this section, I'll show some variants to motivate other uses.


ΔE = 2.3, ~67,000 colors, white equator

Here the value mapping is inverted, so that the equator is white and the poles are black. I decided against this configuration as the RGB gamut is significantly fuller around black than white, resulting in fewer equatorial gaps. Still, this configuration might be preferable in some cases.

ΔE = 10.0, ~800 colors

I also see utility in a full-domain yet low-resolution CNM. Here I've drawn the Voronoi boundaries in black to further accentuate the low resolution and coarse cell boundaries. It's much easier to choose between hundreds of colors instead of millions; in many cases the full resolution is not helpful, or even a hindrance.


ΔE = 2.3, ~67,000 colors, meridians every 45° and parallels every 22.5°


Lines of constant elevation and/or latitude can also be overlaid to convey contours of constant property and suggest the shape and distortion of the map projection.

Source Code

Coming soon to my GitHub.