Most AR beauty tools were designed on a narrow slice of skin tones and it shows. A shopper with a Fitzpatrick Type V or VI complexion opens a try-on experience, picks a foundation shade, and sees something that looks nothing like how that product would actually look on her face. At that point the tool hasn't helped — it's actively damaged trust in the product and in the brand. We spent a significant portion of our early engineering time on exactly this problem.
Why Generic AR Tools Get Darker Skin Tones Wrong
The failure mode is specific. Most first-generation AR beauty tools were built by training their rendering models on datasets skewed toward lighter skin tones — Fitzpatrick Types I through III — because that's where the easiest labeled training data existed. When those models encounter a Type IV, V, or VI face, they make systematic errors in two places: pigment opacity estimation and undertone blending.
Pigment opacity estimation is the calculation of how opaque or translucent a cosmetic product appears against a given skin tone. A sheer blush on fair skin reads as a soft flush. The same product on a deep brown complexion needs different opacity rendering to look accurate — not because the product is different, but because the contrast mechanics are different. Generic models trained on lighter skin overestimate transparency on darker tones and produce results that look washed out or muddy.
Undertone blending is even more technically demanding. Skin undertones — warm, cool, neutral — interact with cosmetic pigments in complex ways. A warm golden-brown lipstick lands differently on a neutral deep complexion versus a warm deep complexion. Getting this right requires a model that actually measures and classifies undertone, not one that guesses based on overall lightness.
We've seen tools from established vendors that simply apply the same pigment blend model across all skin tones, adjusting only brightness. The result is that lighter skin tones get a passable simulation while darker skin tones get something that no informed shopper would find useful. That's not an edge case problem. That's a product that works for some customers and not others.
How Fitzpatrick Classification Changes the Problem
The Fitzpatrick scale was developed in 1975 as a dermatological tool for measuring skin's response to UV radiation. It classifies skin into six types based on melanin concentration and phototype characteristics. In beauty AR, it's useful not because we need the medical precision but because it provides a structured vocabulary for the ranges we need to render accurately.
Lumeglint's on-device classifier measures the shopper's Fitzpatrick scale range using the device's front camera and ambient light conditions, then uses that classification to select the appropriate pigment opacity curve and undertone mixing model for that session. This runs entirely on the device — no images leave the browser, no biometric data is stored.
The practical result: a deep mauve lipstick renders with the correct richness on a Type VI complexion, not the faded version that a generic model produces. A sheer blush reads correctly on a Type II complexion without looking over-applied. Each shade in the brand's catalog gets calibrated against the actual skin being rendered, not a hypothetical average.
In our testing across a range of cosmetic product types, we measured classification accuracy at 91% on Fitzpatrick Type mapping for device-camera inputs under standard indoor lighting. Performance drops to approximately 84% under very low light conditions, which we flag to users in-session. That's honest engineering: the tool tells you when conditions aren't ideal rather than silently producing a worse output.
The Equity Argument Is Also a Business Argument
There's a values case for getting skin-tone accuracy right — beauty brands that fail diverse shoppers aren't just causing individual disappointment, they're signaling something about who the product was designed for. We hold that value genuinely at Lumeglint. But the equity argument and the business argument point in the same direction here.
Darker skin tones represent a large and growing segment of the US beauty market. Brands that have historically underserved this segment — in shade range, in marketing imagery, in AR tool accuracy — are leaving real revenue on the table. A try-on tool that works accurately across the full Fitzpatrick range isn't a niche feature. It's table stakes for any brand that wants to credibly reach the full US DTC beauty market.
"When we were testing early prototypes with friends and family, the feedback we got from deeper skin tone testers was immediate and unambiguous. 'This doesn't look like me.' That told us we hadn't solved the problem yet. The classifier was the fix, not just better lighting."
— Yemi Adebayo, CTO & Co-Founder, Lumeglint
A shopper who sees her actual complexion rendered accurately in a try-on is more likely to buy. That's not a surprise. But it's also a compounding effect: she's more likely to come back, more likely to explore the shade range further, and more likely to share the experience. Accuracy drives session depth. Session depth drives conversion. It's a direct line.
The Finish-Type Dimension
Skin-tone accuracy isn't only about complexion classification — it also intersects with product finish type. Matte, satin, glossy, and metallic finishes each interact with skin texture and undertone differently, and those interactions vary across the Fitzpatrick range.
| Finish Type | Key Rendering Challenge | Fitzpatrick Range Impact |
|---|---|---|
| Matte | Absorbs light; must avoid ashy rendering on deeper tones | Higher variance across Types IV–VI |
| Satin | Low specular highlight; most forgiving across range | Lower variance; works well broadly |
| Glossy | Specular highlights must track natural lip shape and skin sheen | Undertone interaction is critical |
| Metallic | Foil-effect layer must not flatten on darker tones | Requires separate metallic overlay model |
Lumeglint handles each finish type with a separate rendering layer. The matte model uses a Lambertian reflectance approximation adjusted for melanin concentration. The metallic model applies a separate specular layer that preserves the foil effect across all skin tones rather than flattening it on deeper complexions, which is a common failure mode in single-model renderers.
What This Means for Shade Catalog Design
Accurate AR rendering surfaces something important for brand teams: it makes shade catalog gaps visible. If a brand has 40 lipstick shades but only 8 of them render with obvious appeal across the darker Fitzpatrick range, the try-on analytics will show that. Brands using our dashboard can see which shades drive engagement across which skin-tone classifications and identify where their shade range has dead zones.
We've seen this accelerate shade development conversations at a couple of our early pilot brands. The data made visible something that brand managers had suspected but couldn't prove — certain shade families were performing well in try-on for some complexions and poorly for others. That's actionable product intelligence that was invisible before accurate cross-tone rendering existed.
For brand teams building the case for shade expansion, that kind of try-on analytics data is a cleaner input than return data alone, because it captures pre-purchase behavior: where shoppers engaged and then dropped off, rather than what they bought and returned.
Where the Technology Is Heading
On-device Fitzpatrick classification is table stakes, but it's not the end of the accuracy problem. We're working on two areas that will matter increasingly as shade ranges grow. First: undertone sub-classification within Fitzpatrick types, which would let the rendering engine distinguish between a warm Type III and a cool Type III at the rendering level. Current models treat within-type variation as noise. It isn't. Second: lighting environment detection, so that the rendering adapts to whether the shopper is in natural light, indoor overhead light, or a bathroom with mixed sources. Lighting is one of the biggest variables in how cosmetics actually look in use, and a try-on that only renders for one lighting condition is solving a partial problem.
Neither of these is in production yet. We mention them to be honest about where the limits of current accuracy sit, not to overpromise. What we have today is meaningfully better than what most brands have access to. What we're building is better than that.
