I’d wanted to cover this topic in the last article, but underestimated how much space was required just to talk about inpainting. This is a similar process, but on adding more detail outside the bounds of the original. As a reminder, the workspace left off set up for inpainting like so:

This looks familiar.

Just off that screen I have a second load image node that doesn’t have an inpainting mask. I could clear the mask from the existing node, but this other one is also already attached to some other nodes that I will need for outpainting, so we’ll just go with those. Here’s all the nodes we’ll eventually be dragging into this mess.

More illegible workflow diagrams

So, note that we’re not using the mask line from the loader. The mask will be generated by the next node in line – “Pad Image for Outpainting”. This node has four options – one for each edge, and “Feathering” The first four is just a value in pixels for how far we’re extending the image. Since we’re starting with 512 and going to 1024, we need another 512. I don’t need more sky, so I’m going a full 512 off the bottom and 256 off each side. Pad Image outputs an image and a mask. On the image, the extended regions are a flat gray, the mask is the padding region and a fade into the main image, ad determined by the value in the Feathering option. The images left in that workflow are from an earlier test where I only padded the sides. The preview image is of the mask generated, the two white bars being the masked region. The “Convert Mask to Image” node was added only so I could see what was happening under the covers, and does not impact he actual outpainting operation.

For reference, we’re starting from this image created back at the start of the previous article:

The results are getting better as I understand the process more.

An Image and Mask sounds a lot like what we had from the Inpainting article. In fact, we are going to take over that VAE Encode node with the outputs from the Pad Image. This is still feeding the sampler. I want to change the prompt, because I care about trying to get the legs drawn in. So the positive prompt has been updated to “space marine in desertpunk power armor holding banner, space marine, 1man, solo, red hair, banner:1.5, desertpunk power armor, 1man, solo, loincloth, full body”.

Just letting it do its thing gives a result that’s… interesting.

It turned the landscape into cloth.

As far as AI limbs go, those legs are not bad, except for the third one behind him. The background is also messed up. Changing the denoise value is worthless, because there’s nothing behind the mask to work with. There’s nothing obvious in the prompt, so lets grab a random seed and try again. After more than a few tweaks of seed and prompt, I started to realize the padding was simply too big. The original image was only a quarter of the total area, so the software wasn’t giving it enough weight as the source of the rest of the image. I decided to trim down the outpainting. 64 pixels a side, 128 on the bottom. With less freedom to operate, the output is better attached to the original. There’s some funkiness, but we can clean that up later on. I have some tricks up my sleeve.

Baby steps

Continuing with the incrementalism, we’re going to feed this output back into the load image node and add another layer of 64 and 128 pixels to the sides and bottom. Now we’ve got an image that’s 768×768, lets see if one more pass gets me something reasonable at 1024×1024…

It does not.

Of course the legs are Wonky – it’s an AI image.

Time to go back to incrementalism. I see wonky legs, but the trick I referenced earlier might help. What is that trick? Resampling. But the last time we resampled we had a funky latent created by the upscaling node. That provided the noise to resample. Where are we going to get some noise? We already have a source in the inpainting mask, we just have to turn down the denoising. But the inpainting mask is currently set up to cover empty space. What I’m going to do is add a new load image node to pull in the output from that last iteration. I’ll then pull the image from the new load node and feed the vae encode, replacing the image out from the pad image node. I’m also going to switch the existing load node to go back to the original image and adjust the padding to the full 256/256/512. For this to work, we have to tune the denoise rating so that it has some respect for the original.

The adjustments to the workflow look like this:

The obligatory workflow update

Running it through the resampler with a 0.7 denoise cleaned up a lot of wonkiness.

That ray of light looks out of place.

But I do notice that the original image is getting pretty fuzzy at this point. We have to clean this up. We’re going to resample again. For this we don’t need the mask from the padding nodes, so we’ll disconnect that and replace the mask with the output from the top load node, switching to our most recent unwonky but fuzzy image. Now we’re going to pull in a node that was hiding in an earlier workflow. “Latent Interpolate” takes two latent inputs and outputs a merged latent at a ratio as defined in the node’s only option. I have this set to 0.5, and feed it the latent from the VAE encode. But it takes two latents. Well, we’re going to feed it a blank latent from the empty latent image node that’s been hanging around unused.

We could have masked the entire image on the load node, or created a solid white image that we converted to a mask, both of these are viable approaches. All of them do the same thing – it feeds a noisy latent into the sampler. As we want some of the original to be left, we have to tune down the denoise. After a few tests, I found 0.5 worked to maintain the banner and cleaned up a lot of the problems.

It’s not as shiny as it used to be

It’s not exactly the same as the original, since we introduced issues and then resampled. The resample changed details. It’s not a perfect outpainting, but it’s still very close. There may be a better way to do this, but I was still more successful than with the inpainting goal. You know what would help with that inpainting if only we could sketch something to guide the drawing… oh wait, we can.

Lets see if I can figure that one out. But we’re out of space for one article. And besides, that a revisit of inpainting.