From a diffusion image to a layered painting
diffusion-to-painting: turning an AI-generated flat illustration into a layered Krita painting where every pixel has a provenance. This is its handbook.
A diffusion image looks like a painting, but it was not painted: no layers, no brush marks, blurry when enlarged, and pixel steps along its edges. This handbook shows how to use dtp (a Python library plus headless Krita plus a local ComfyUI) to turn one into a .kra an artist can take over, following an artist’s own stages: every layer named, whole, and editable on its own; the lines individual strokes; and not one pixel taken from scaling the source up.
One step per stage, one chapter per step (part one):
one diffusion image
segment cut it into named parts (boundary barriers + the SAM majority vote; lines can be walls)
lineart the lines on the source: two sources into a candidate pool, judged by the source,
fitted into single strokes
complete each layer's hidden part is handed to diffusion to invent a whole shape, while the
visible part is pinned
lines lineart extracted from each layer alone and arbitrated against that layer's image; the
invented lines are accounted for
draft the layer tree + each layer's shape + flat colour + its own lines -> a person looks
paint each part's own method: one shadow band for skin, four shapes for an eye, the baseline
for the rest; plus the background colour field
gauge numbers normalized to one scale and anchored to the source + review images + the journal
trace the other route: paint straight after segmenting, two commands to a finished picture
Two routes (part two). One-shot: segment -> trace, which resembles the source but whose layers are not what a person wants. Layered: segment -> lineart -> complete -> lines -> draft (which a person looks at) -> paint, whose layers are made for a person.
How to run it
There is no command line. One stage is one directory with one main.py in it, copied from that stage’s template with the upstream paths and knobs at the top edited. Run that:
mkdir -p work/pic/input && mv pic.png work/pic/input/origin.png && cp dtp/templates/project/{parts.py,README.md} work/pic/
mkdir -p work/pic/phase1-layers/step01-segment && cp dtp/templates/segment/main.py work/pic/phase1-layers/step01-segment/
uv run python work/pic/phase1-layers/step01-segment/main.py # the first run writes only the region map; fill in ASSIGN in parts.py and run it again, then trace, or lineart -> complete -> lines -> draft -> paint -> gauge
How to read this
- Just want it running -> Installing and self-checking for the environment, then Ten minutes for a run-through.
- Want to know what each step does, what to look at, and what to change when it is wrong -> part one, one chapter per step, all with the same skeleton.
- Want to change how a part is painted -> ch. 6, “Paint”, one page per part, and appendix G,
parts.pyfield by field. - Want a different lineart extractor -> the subsections of ch. 2, one section per method.
- Want to take over the code -> appendix B, the code index, and appendix F, the file formats.
- Want to do something else in Krita -> part three, a reference for what headless Krita can do, all of it measured.
- Want to save yourself some detours -> ch. 17, the three iron rules of measuring the wrong thing and the case collection – the most expensive chapter in the book.
- Want to see what a real picture comes out like, and what the numbers are -> ch. 19, the worked example (origin). The main text never quotes its numbers.
Status marks
The first line of every chapter carries a mark, so that nothing unmeasured ends up in a table of measurements:
| mark | meaning |
|---|---|
| 👤 checked by a human | a person looked at the output and the review images and was very satisfied. Only a person can award this |
| ✅ verified | there is a probe or measured data, the numbers reproduce, and they come from a step produced by the committed code – the author considers it passed, but no one else has accepted it |
| 🧪 in progress | it runs, but has not settled into something satisfying |
| 📐 designed | only the questions, the interface and the probes to be written; nothing has been run |
| ⬜ placeholder | the skeleton is up but nothing has been studied: only “how an artist paints it” and “what has to be decided”, with an empty module in the code |
Repository layout
book.toml the mdBook config
po/ the Chinese edition (gettext; the English src/ is the single source of truth)
src/
README.md the cover (this file)
SUMMARY.md the table of contents
00-start/ 01-stages/ 02-routes/ 03-krita/ 04-measure/ 05-worked-example/ 99-appendix/
assets/ review images and comparisons
tools/ audit.py the content audit - links.py the link check - figures.py the diagrams -
publish.sh builds both editions and deploys to Cloudflare Pages
../dtp/ the library (a uv project, its own git): src/dtp/ the five ranks, io.py the
readers and writers, templates/ one main.py template per stage plus the parts
table template, tools/ environment scripts, jobs/ the painting job, probes/
../work/<name>/ subprojects: input/ the picture + parts.py + one stepNN-<stage>/ per stage, each
with one main.py (the conventions live in AGENTS.md one level up)
Environment
Fedora 44 · Krita 6.0.2.1 (RPM) · PyQt6 · Xvfb · Python 3.14 (numpy / scipy / scikit-learn / Pillow / matplotlib) · ComfyUI (lineart extraction, GroundingDINO+SAM) · See-through (the layering model used for completion; one card with 16 GB of VRAM). Krita 5.x has no Node.paintLine, and this book’s code is not compatible with it (Installing and self-checking).