Criteria: the numbers, the review images, honesty
✅ verified · the last step of both routes · measuring one finished picture on its own, without a step: appendix A · code: appendix B
What this step does
Three kinds of acceptance, and all three have to pass. The numbers: measure everything measurable, with the thresholds anchored to the source. The review images: what numbers cannot measure (whether a layer is whole, why a line is there) goes in front of a person. Honesty: how many numbers it took to say what this painting is, with the journal to prove it.
Inputs and outputs
From the paint or trace step: the finished composite, the painting journal, the manifest, the Krita log and the layer tree (for the description count). From paint as well: each group’s mask, the painting plan and the lines. From segment: the sub-part ids, so that “inside the hair” can be measured separately. It writes every reading to one file, the table into the README, and the matching-position comparisons into their own directory. The file names are in appendix F.
The numbers
Every whole-picture measure is taken after normalizing to 2508 px wide (the same output reads differently at different widths, and comparing across resolutions gives conclusions pointing the wrong way). The thresholds are anchored to the source, not to a reference painting: a real painting’s edges are far sharper than a diffusion image’s, and demanding that a study be sharper than what it studies is a gate nothing can ever pass.
| Criteria | threshold | what it measures |
|---|---|---|
| description numbers over source numbers | at most 5% (the target is 2%; above 20% it is judged to be re-encoding the source in dots) | how many numbers it took to say what this painting is |
| bare pixel writes | = 0 | writing pixels is allowed, but every write has to say why |
| gradient p99 | at least 85% of the source’s | how sharp the outlines are |
| 8-level quantized colour count | at least 55% of the source’s | the value structure |
| the 8 px stair fingerprint | below the source’s | whether there is an upscaling trace (the periodic signal a 1x pixel grid leaves when enlarged) |
| low-frequency likeness | at least 30 dB, with both sides reduced to 256 wide first | it should resemble the source at the low-frequency scale only |
| interior sharpness | anchored to the source, crossing it in neither direction | the gradient p90 more than 30 px from the outline – p99 is propped up by the outlines and cannot see a blurry interior; the hair’s interior is listed separately |
| the darkest pixel | (0,0,0) | mistakes of the “the darkest pixel came from the background” kind |
| the restack difference | about 0 inside the subject | stacking the group layers back in occlusion order and comparing pixel by pixel with the source (ch. 3) |
| lines | flips per 100 px at most 3; one stroke per edge; a fitting deviation of at most 2 px@1x; and lines in the invented region listed separately | whether a line is one stroke; reported per line layer, with each layer’s lines recording which layer they came from |
| per-group interior sharpness and at the hard edges | anchored to the source | the gradient p90 at least min(30, req/3) px inside the group’s edge; at the hard edges is the output’s gradient p50 over the pixels in the source’s top 5% of gradient (on a large flat group, p90 measures the flat area and the shadow edge drowns in it) |
| per-group 1x PSNR | 0.6 dB between two paintings is noise | the masked PSNR inside the group after reducing the output to 1x (likeness cannot be measured per group: at 256 wide a small group is only tens of pixels) |
| per-group 8-level colour count | anchored to the source | how many colours this group has after 8-level quantization. A method’s faults only show when the whole picture’s colour count is split by group: quantize something carrying a hue shift into a few flat colours and the whole picture’s count barely moves (the other groups hold it up) while this group collapses – measured, the source’s hair had 123, one flat colour per lock 11, and one cylinder per lock 65, while the same outputs’ gradient, stair, likeness and sharpness all read normally |
| the band edges | anchored to the source | the gradient p90 over a 2 px band either side of a band’s Shape ring – this measures “is the shadow edge hard” directly |
| Layers | at most 10 nodes per group, and at most 5 + 10 x the number of groups overall | whether the tree is the one a person wants (ch. 5): every node is a Shape, a Flat, a Shade, a named band, a Blend, a Strand or a Line, counted from the layer tree without opening Krita. The one-shot route’s tree fails this ruler by design – it answers a different question (ch. 8) |
The per-group rulers use the masks the paint step wrote: they are mutually exclusive, and a group cut out of a mask (the eyes) is subtracted from the group it sits in (the skin) – without that, the skin’s “at the hard edges” measures nothing but eye sockets and pupils.
Noise: painting the same plan twice differs by 0.6 dB, 3 points of colour count, 0.2 of p99 and 0.2 of p90 – Krita’s brush engine is stochastic, and a difference smaller than that is noise and carries no conclusion. Never compare across multipliers: the same code reads differently at 8x and at 4x, and two columns of a table have to be at the same multiplier.
Knobs
| what it points at | meaning |
|---|---|
| a paint or trace step | which step to measure |
| another paint step in the same subproject | for comparison: the band-edge table gains a column |
| another subproject’s finished picture | the matching-position comparisons are produced: the source, it, and this one |
The constant names are at the top of this stage’s main.py and are listed in appendix A.
review images
The review images are part of what is delivered – another model’s output has to be reviewed too.
| image | what to look at |
|---|---|
| the segmentation review | what segmentation changed (three columns: the hand-written table, the vote, and what moved) |
| the completion review | per group: the occluded area / the completed image / the layer alone / its outline; plus the restack difference |
| the lineart review, on the source and then one per layer | why each line is there and why each dropped one is not; and each layer’s own lines, the pale ones being in the invented region |
| the 1x composite, and the 1:1 close-ups | the whole picture, and the close-ups centred where the parts table says – at least the face, the neck, a hand and a foot |
| one image per layer, one per group | each layer alone on grey, and each group rendered alone with nothing over it – “is the kettle whole” |
| the comparison crops | the matching-position comparison against the other route |
The file names are all in appendix F.
Where to overrule: for lines, overrides.json (shared by ch. 2 and ch. 4); for segmentation, the tables and words in parts.py, or redoing it once with the lineart as walls; for completion, which of the model’s layers that group uses; for a method, the choice a person wrote for that group (appendix G). After changing one, rerun only that step under a new number, and the ones downstream of it.
The images are to be looked at, not merely produced. Every mistake that pointed the wrong way – judging by the source of a line, ribbons mixed into hair, completion removing a top, cel banding, the neck given to the hair – was caught on a review image, and some of them sat there through several versions with nobody looking.
Honesty
The criterion asks: how many numbers did it take to say what this painting is?
The old threshold was “at most 2 pixel writes”. It once caught real cheating – an output with every criterion green whose journal was nothing but set_pixels and not one stroke, which had scaled the 1x pixels up in blocks and pushed them into the layers. What deserved to survive of that gate was its spirit, not its letter. It confused two things into one: “do not copy the source” and “use nothing but brushes”. Gradients, colour fields, erasing and filters all go through set_pixels and hit the gate on first use; so a background had to be laid with twelve thousand strokes over something blurred out.
The new way of counting: recursively count how many numbers this layer tree holds, divided by the source’s pixel count times three.
| saying one thing | costs how many numbers |
|---|---|
| filling the whole canvas with one colour | 3 |
| a linear gradient covering 8K | about 19, even though it setPixelData-ed 25 million pixels |
| one vector fill | control points x 2 + 3 |
| one stroke | points x 2 + 5 |
| a 32x48 colour field | 4,608 |
| pasting the 1x source in | 1024 x 1536 x 3 = 4,718,592 – straight through the gate |
A raster node counts how many numbers the stored image has (size x 3), not how many pixels it covers once stretched. That one rule does two things at once: pasting the source blows the gate immediately, while “describing a blurred-out background with a small colour field” is legitimate – because it really did write down only that many numbers.
Writing pixels is not the sin; being unable to say why is. So every set_pixels carries a why, and one without is a bare write, of which the threshold is zero.
With the criterion changed, several things that could not be done before follow naturally: a shadow really can be “a multiply layer at 65%” rather than the composited RGB filled flat; “draw the whole highlight band and then cut it” can be expressed with a transparency mask; a background can be a colour field; and a band’s edge can be softened with one blur mask instead of tens of thousands of soft strokes.
The numbers have to come from output produced by this code. Numbers from a run before the commit do not count; a failed job has to fail at its entry point; and the criteria check that the file being measured is the one this run wrote (ch. 17, case 35).
Every number has to answer “over what area, measured how”: gauge prints the normalized width and the close-up windows; a probe’s docstring states its criterion, how to read it and what it is anchored to; and anything unmeasured is marked 📐.
Where it breaks
The whole of ch. 17. Three iron rules: settle whether you are measuring one layer or the composite before you start; comparing across resolutions is not a comparison at all; and the thresholds are anchored to the source.
On origin
The full criteria table for both routes: ch. 19, “The two routes side by side”.