Complete: every part’s whole shape

🧪 in progress · step 3 of the layered route · code: appendix B · the parts-table fields: appendix G

This chapter is still to be rewritten for the new method. The default is now the See-through merge: a model trained on Live2D layer data produces each layer’s whole shape, and we take only the shape and pixels of what is hidden, with everything visible pinned to the source (the last section of appendix E). The ring-by-ring approach described below, which handed the work to diffusion, was deleted from the library in 1.0.0 and is kept here only for comparison; its pitfalls are in appendix E.

What this step does

“The kettle is occluded, but the kettle itself can be painted whole as a layer and then covered by another layer.” Every layer – one group of the parts table – needs its whole shape, including the part hidden behind the layers above, so that hiding the dress shows a whole leg and hiding the body shows the whole of the back hair. A person decides the layers before this step, writing the groups from ch. 1’s region map and ch. 2’s lineart.

The rule: what is hidden is handed entirely to diffusion to invent the most primitive shape, rather than “spreading outward a bit more”; and the acceptance test is that stacking the completed layers back in occlusion order differs in nothing from the original, while each layer on its own is a whole, correct object.

Inputs and outputs

The file names are all in appendix F; the parts-table fields are in appendix G.

The method

For each group G, bottom to top in occlusion order:

1. The occluded area is the pixels of every group above it (visible background is not an occluder, and G does not reach there).
2. Have the inpainting model repaint the hole; everything visible is pinned to the source – VAEEncodeForInpaint’s mask covers the hole only, and when compositing, the visible area always uses the source’s pixels.
3. Recover G from the completed image: GroundingDINO plus SAM cuts once with the words a person chose for G, subtracts the words to be taken away, intersects with the hole, and keeps only the pieces connected to what was already known; where SAM catches nothing it falls back to a colour rule (resembles G rather than the occluder).
4. The whole shape is the visible area united with what was recovered; the layer’s pixels are the source where visible plus the completed image where invented; and solving alpha, tracing the rings and fitting them gives ch. 5 its Shape.
5. The restack test: stack the groups back in occlusion order and compare pixel by pixel with the source inside the subject. The visible area is pinned, so the difference is zero by construction – what this measures is holes and misalignment in the masks, and any reading other than zero means one of those went wrong.

The shape of the hole decides what the model paints. This is the most expensive sentence in the chapter: to an inpainting model, the shape of the hole is the prompt. Erase the whole occluded area at once and a dress-shaped hole produces a dress, a person-shaped hole produces a person, and neither the prompt nor the context changes that; “show it only itself” (erasing every other part to background) behaves the same way. It only holds when the hidden part is small relative to the hole.

So it has to grow outward one ring at a time: the hole is a ring d_step px outside what is already known, and the model sees only “this object plus background” and an annular hole – an annulus has no silhouette to copy. Whatever the model paints of the object into that ring is taken in, and another ring is grown; when it stops painting the object into the ring (less than 5% of the ring is recovered) it stops, up to max_it rings. Each ring is one inpainting pass and one recovery.

What is under clothing is invented too, but only its shape is used. The pixels under the skirt always sit below the dress layer and are never visible in the finished painting; the prompt asks the model for plain close-fitting clothing rather than a nude (nsfw / nude / underwear are permanently in the negative prompt). Ch. 4’s per-layer lineart extracts lines from those invented pixels, and ch. 6’s painting copies them in the hidden area – the only places they ever surface, and both are accounted for.

A group with nothing above it is not completed, and neither is a group cut out of a mask (the eyes): it sits on the face, and its mask is already the whole shape.

Knobs

What a person decides here

Under the current method there is one decision: which of the layering model’s layers each group may be completed from – a group nobody listed only gets its seam filled. Its field name and format are in appendix G.

The two below belong to the ring-by-ring method described in this chapter, which is no longer in the library (see the note at the top); they are kept for comparison.

Neither of those two fields is in the parts table any more; appendix G lists the fields that are.

Changing these means rerunning complete under a new number, and rerunning lines / draft / paint after it.

Reviewing and overruling

The completion review: one row of four panels per group – the occluded area / the completed image / this layer alone / its outline. Look for: whether what was invented is this object at all (plain clothing under the skin rather than a nude or a black top; hair that is hair rather than a person-shaped silhouette); whether the shape grew too far; and whether SAM degenerated in the last few rings, which the tally records as the rule that ended up being used. The restack image: source | restacked | the difference in red and what nobody claimed in yellow – which must be 0.

What to change: it invented the wrong thing -> what that group is asked for, and what it is asked not to produce; it recovered too much or too little -> the words the group is recovered from (appendix G); the shape changes wildly with the seed -> try other seeds to see how stable it is, and instability means the model has no evidence.

Where it breaks

• Erasing everything at once: the hole’s shape becomes the prompt, and it invents the occluder’s shape.
• Showing it only itself: the same.
• The model dresses or undresses the body: only the shape is taken, the negative prompt is there, and once the visible area is pinned nothing visible can be changed.
• Subtracting into nothing: during recovery a word in minus degenerates into the whole person on the completed image, the subtraction empties the mask, and the occluder ends up in this layer. Every step of the recovery rule needs the “discard anything covering more than 90% of the subject” check, not just the first.
• SAM degenerating in the last rings: when it catches nothing it falls back to the colour rule, and stopping at max_it is stopping, not converging.
• The large area behind has no evidence: for back hair hidden by the whole body there is nothing in the source at all, and it rests entirely on the model’s prior that long hair hangs behind; a different seed gives a different shape. A real amodal-completion model, of the kind that produces shape without pixels, has not been tried.

Honesty

The invented pixels are invented. The account holds, per group, the hole and the whole shape, and beside it this layer’s pixels – the source where visible, the completed image elsewhere – so which pixels came from the model is never in doubt. A line ch. 4 extracts from those pixels carries its own invented fraction, in the same account.

On origin

How much each group gained, how many rings, which recovery rule, the restack difference, and the comparison images for erasing everything at once and for showing it only itself: ch. 19, “The layered route”.