Diffusion-Generated Minecraft Skins

Learn how to fine‑tune a diffusion model with IP adapters to generate valid 64×64 Minecraft skin files from photos, covering dataset construction and live demo.

Overview

I built a model that generates Minecraft skins you can use in the game, not just pictures that look like skins. Most image models fail at this because Minecraft skins need a specific 64×64 pixel UV map format. I scraped 1,000 skins, created composite training images showing both the raw pixel layout and 3D character renders, then fine-tuned a diffusion model with IP adapters for image conditioning. You upload a photo and get back a working skin file. I’ll show the live demo and walk through why this seemingly simple problem is actually pretty tricky to solve right. [Disclosure: a lot of the innovation here was from following a blog post training SD2 on MC skins. I only built on top of that with IP adapters]

Links

https://dhawley.com/crafty-skins/
Fine-tuned Stable Diffusion generates playable 64x64 Minecraft UV maps via IP Adapters.
https://github.com/dylanhawley/crafty-skins
Crafty Skins generates Minecraft skins using IP-Adapter with Stable Diffusion XL.

Tech stack