Build Log: Adding De-lighting to Our 3D Pipeline (Chrome Hair Included)

This is a build log, not a launch post. We spent four build cycles wiring texture de-lightinginto MeshAura's pipeline, produced a man with chrome hair along the way, and ended by switching the feature off — on purpose, with the research to show why. All three models are embedded below, unretouched, because the failure explains the feature better than the success does.

The problem: photos carry their lighting with them

Every photo has its lighting baked into every pixel — the shadow under a chin, the sheen on a forehead, the dark side of a jacket. When an image-to-3D model generates a texture from that photo, those shadows and highlights get painted permanentlyonto the 3D model. Drop it into a game engine and it's double-lit: the engine adds real light on top of frozen light, and the model never quite looks like it belongs in the scene.

What an engine actually wants is albedo— pure surface color, photographed under perfectly boring, even light. Removing the baked lighting to recover it is called de-lighting, and it's a feature the premium AI 3D platforms sell. It's also available as open research: Tencent's Hunyuan3D ships an open de-lighting model (a stock Stable Diffusion InstructPix2Pix fine-tune) with released weights. So we wired it into our pipeline: after background removal, before generation, the input image takes a trip through the delight model and comes back flat-lit.

Exhibit A: the control

Our test subject — a character bust generated without de-lighting. It looks good in a viewer, but numerically the texture is starved: mean brightness 26/255, because the studio lighting's shadows are baked into the paint. Under a game engine's lights, those frozen shadows fight the real ones.

Exhibit B: the chrome-hair incident

Build cycle two. The delight model loaded, ran, and the pipeline completed — technically a success. Spin it around:

The skin is actually the proof the idea works — warm, even, shadow-free, exactly what relightable albedo should be. But the hair, beard, and jacket have turned to brushed silver. What happened is a chain of two mistakes, and the second one is the interesting part:

1. Our color-correction step computed its statistics over the whole image — including the white backgroundthe object sits on. The white dominated the math, the correction "fixed" the background, and the object's color washed out: the teal jacket went grey, the dark hair went neutral silver. (Numerically: texture saturation halved, 68 → 31.)
2. Then the 3D model's material estimator looked at that desaturated, highlight-free grey hair and drew the only reasonable conclusion: this is metal.Neutral color with no diffuse variation is precisely what brushed metal looks like to a PBR estimator. It wasn't hallucinating — it was correctly interpreting an image we had corrupted.

That's the general lesson of this whole build log: in a multi-model pipeline, an upstream bug doesn't just degrade the output — it gets reinterpretedby every downstream model into something confidently, coherently wrong. Chrome hair isn't noise; it's a plausible reading of bad input.

The fix: stop approximating the reference

Our color correction was a "lightweight version" of the one in Tencent's reference implementation. The real one differs in three ways, and each turned out to matter: it computes statistics only over the object's pixels(masked by alpha, so the background can't dominate); it uses a slightly different affine correction; and it keeps a safety check we'd dropped — if the "correction" moves the image farther from the original than the uncorrected output, it throws the correction away.

So we replaced our approximation with a line-for-line port of their function, and unit-tested it on synthetic data before spending another GPU cycle: a washed-out grey-teal restores to within ~2% of the original teal. The rule we've now written into our engineering docs: when a reference implementation exists, port it verbatim — approximate nothing.

Cycles three and four: winning battles, losing the war

The verbatim port worked — and revealed the next layer. Tencent's correction defines "fixed" as looks like the original photo again. But the original photo's look includes the shadows we're removing — so the working correction faithfully dragged the brightness back down (54 → 37). One tool mopping, another pouring the bucket back out.

Cycle four split the job on principle: brightness from the delit output, color from the original photo — each channel from the source that's good at it. The mixing math had a flaw (additive chroma at higher brightness reads as pastel), but by then the numbers were telling a bigger story. Here's the full scorecard — texture brightness should rise and color saturation should hold:

• Control (no delight): brightness 26, saturation 68 — colorful but shadow-crushed
• v1: brightness 55, saturation 32 — bright but grey (chrome hair)
• v2: brightness 54, saturation 28 — correction silently discarded
• v3: brightness 37, saturation 23 — correction undoing the delight
• v4: brightness 57, saturation 23 — best brightness, color still lost

Four different color strategies, one constant: every delighted input produced muted textures and metallic misreads.When four different fixes fail the same way, the problem isn't the fixes.

The sentence that explained everything

The answer was in Tencent's own paper, one sentence we'd read past: their texture model's training renders were lit "under even white light to accommodate the delighting model." Their de-lighter works in their pipeline because their downstream generator was trained to expect de-lit input— the two halves are a matched pair. The generator we run (Microsoft's TRELLIS.2) was trained on naturally-lit images. Feed it an artificially flattened photo and it's a chef trained on raw ingredients being handed pre-cooked ones: it reinterprets the strange input — as muted color, as metal — no matter how carefully we condition it. That's not fixable from our side of the model.

The verdict: parked, with a map to the right answer

So we switched the stage off (it lives behind a flag, one env var from resurrection) and filed the real design for later: de-light the output, not the input — let the generator work on the natural photo it expects, then strip the baked shading from the finished texture, where nothing downstream can be confused. Better still, this problem has a proper name — intrinsic image decomposition — and a purpose-built state-of-the-art tool: Marigold-IID (ETH Zürich), which predicts albedo, roughness, and metallicity directly. That's the version we'll build when it's earned its place in the queue.

What did the failed experiment cost? About $3 of GPU time, one evening, and a man with chrome hair. What it bought: the pipeline plumbing (built, tested, dormant), the exact knowledge of the right architecture, and this post. If you're building on AI pipelines yourself, take the meta-lessons: make experimental stages fail open; keep a control artifact for every A/B; port reference implementations verbatim, never approximate; and when output looks wrong, measure it— "it looks grey" is a complaint, "saturation halved from 68 to 31" is a diagnosis. And the big one: in a multi-model pipeline, models reinterpret bad input into coherent, confident wrongness— chrome hair isn't noise, it's a plausible reading of an image we corrupted.

Meanwhile, MeshAura today generates the same quality as the control model above — which, spin it again, is pretty good — at a flat $0.35 per model, no subscription. The de-lit version will land when it's ready to be better, not just different.