Microsoft TRELLIS Explained: The Open-Source Model Behind Modern Image-to-3D

If you've used any image-to-3D tool recently, there's a good chance TRELLIS was somewhere under the hood. Released by Microsoft Research in late 2024 with open weights and code, TRELLIS reset expectations for what a single image could become: not a blobby approximation, but a detailed, textured 3D asset. This article explains what TRELLIS is, the core idea that makes it work, and what that means practically if you just want good meshes.

What TRELLIS is

TRELLIS is a family of 3D generative models from Microsoft Research, introduced in the paper "Structured 3D Latents for Scalable and Versatile 3D Generation." The headline capabilities:

• Image-to-3D and text-to-3D from a single unified architecture.
• Multiple output representations from one generation — the same latent can be decoded into a radiance field, 3D Gaussians, or a textured mesh, so you pick the format that suits your use case.
• Scale — the largest released models have around two billion parameters, trained on a corpus of roughly half a million 3D objects.
• Open source — code and model weights are publicly available under a permissive license, which is why an ecosystem of tools and services (ours included) exists around it.

The core idea: structured latents (SLAT)

Earlier 3D generators mostly bet on one representation. Mesh-based methods struggled with topology; NeRF-style methods produced pretty renders but poor geometry; point clouds and voxels traded detail for tractability. TRELLIS's insight is to generate in a representation-agnostic intermediate space and defer the choice of output format to a decoder.

That intermediate space is the structured latent, or SLAT: a sparse 3D grid of voxels that each carry a learned feature vector. The sparsity matters — only voxels near the object's surface are active, so capacity is spent where the detail is, not on empty space. The features are distilled from strong 2D vision models (DINOv2) applied to many rendered views of each training object, which is a big part of why textures and fine appearance come out so well: the representation inherits what large image models already know.

Generation is a two-stage pipeline of rectified-flow transformers: one model generates the sparse structure (which voxels exist — the coarse shape), and a second fills in the feature vectors (appearance and fine geometry). Separate decoders then turn the same SLAT into whichever output you asked for — a textured mesh via FlexiCubes, 3D Gaussians, or a radiance field.

Why this produces better assets

• Geometry and texture are learned together. Because the latent encodes both shape and appearance, textures line up with geometric features instead of looking projected on.
• The invented parts are plausible. A single photo shows one side of an object; the model must infer the rest. Training across hundreds of thousands of objects gives it strong priors, so the back of a generated chair looks like a chair.
• Meshes are extracted, not stitched. Decoding to a mesh via FlexiCubes yields watertight-ish, coherent geometry that imports cleanly into Blender, Unity, Unreal, or a slicer.

Running TRELLIS yourself vs. using a service

Because TRELLIS is open source, you can absolutely self-host it. Realistically, that path looks like: a CUDA-capable GPU with enough VRAM (16 GB is a comfortable floor for the large image model), a Linux environment, a specific stack of PyTorch, sparse-convolution and rendering dependencies that are sensitive to version drift, and some patience the first time. For an ML engineer it's a weekend; for a game developer who just needs props, it's a distraction.

That trade is the entire reason hosted services exist. Lattice3D runs TRELLIS-class generation on cloud GPUs behind a browser UI: upload an image or type a prompt, preview the result in 3D, download a GLB. Our pricingmirrors the simplicity — a prepaid balance with a flat price per generation. No subscription to remember to cancel, no lock-in, and your balance never expires. You're paying for the convenience of not managing GPUs, and only when you actually generate something — failed runs go straight back to your balance.

TRELLIS in the wider landscape

TRELLIS isn't alone — commercial systems and other open models compete on quality, and the field moves quickly. Its distinctive position is the combination of open weights, multi-format output, and strong single-image fidelity, which made it the reference architecture that much of the current image-to-3D ecosystem builds on. If you're evaluating tools, the practical questions are the same regardless of the model underneath: How good is the geometry on your inputs? Does the export drop into your pipeline? And does the pricing let you experiment freely?

FAQ

Is TRELLIS free?

The code and weights are free to download and use. Running it requires a capable GPU — that's the real cost, whether you rent one, buy one, or use a hosted service like ours and pay per generation.

What input does TRELLIS-based generation like best?

A single clear image of one object on a simple background. See our game dev workflow guide for input preparation tips.

What formats can I get out?

The architecture supports radiance fields, 3D Gaussians, and textured meshes. For game engines and 3D printing, the textured mesh (exported as GLB) is what you want, and it's what we serve by default.