Tileable texture from Stable Diffusion

[hlky]

Someone requested Tileable textures be implemented, this code is for diffusers codebase, needs implementing for stable-diffusion codebase

[KawaNae]

I simply inserted this partial code from here in line 34 of optimized_txt2img.py and got Tileable texture scripts for stable-diffusion codebase. (I'm new to python so I could be wrong on this.)

def patch_conv(klass):
	init = klass.__init__
	def __init__(self, *args, **kwargs):
		return init(self, *args, **kwargs, padding_mode='circular')
	klass.__init__ = __init__

for klass in [torch.nn.Conv2d, torch.nn.ConvTranspose2d]:
	patch_conv(klass)

[FuouM]

https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html

from the documentation, the "padding mode" have several options:

padding_mode (string, optional) – 'zeros', 'reflect', 'replicate' or 'circular'. Default: 'zeros'

the default mode is zeros so I think it would be wise to have a toggle to switch between them

[Alephrin]

Unfortunately, the code KawaNae has only works if it's done before model load as the padding method is used during initialization of those networks and changing it afterwards seems to require iterating through the already loaded model, saving the weights of and recreating the conv2d, then restoring the weights or completely reloading the model after running the patch code.

[OleJ1964]

I would also love to see Tileable textures implemented, as it is now i go back and forth between implementations since i need that feature.
As of now i use AUTOMATIC1111](https://github.com/AUTOMATIC1111 for textures

[denizbinay]

i really want to have that too!

[OleJ1964]

it works in this one, As of now i use AUTOMATIC1111](https://github.com/AUTOMATIC1111 for textures, but it would be nice not to switch between. But i do realize its early days - a lot is happening - cant imagine where all this will be in just 3 month

[denizbinay]

thats what i'm try right now. Its Amazing!

[OleJ1964]

Its a god send for us using textures in eg. Blender. No more hunting down semi usable images - most with water marks, or using procedural materials, and then having to bake them all for game engines. Now we just need SD to create Normal maps as well. :-) (I know that a addon for blender (and im sure lots of others) is in the works - so probably Normalmaps can be generated in that

[jamestiotio]

I managed to implement tileable textures for my implementation of a Blender add-on using Stable Diffusion here. It seems pretty straightforward.

There are two options:

1. You can directly use the code in the GitLab snippet that @KawaNae linked to:

   import torch
   
   def patch_conv(cls):
   	init = cls.__init__
   	def __init__(self, *args, **kwargs):
   		return init(self, *args, **kwargs, padding_mode='circular')
   	cls.__init__ = __init__
   
   patch_conv(torch.nn.Conv2d)
   
   from diffusers import StableDiffusionPipeline
   
   pipe = StableDiffusionPipeline.from_pretrained(...).to("cuda")

   Note that it only works before initializing the model (and there is no need to patch ConvTranspose2d). It will also globally affect all Conv2d modules used by other models after patch_conv is executed, which might or might not be desirable depending on the use case.

2. You could initialize the model first, and then "hijack" the model afterwards and set padding_mode to circular:

   import torch
   from diffusers import StableDiffusionPipeline
   
   pipe = StableDiffusionPipeline.from_pretrained(...).to("cuda")
   
   targets = [pipe.vae, pipe.unet]
   for target in targets:
   	for module in target.modules():
   		if isinstance(module, torch.nn.Conv2d):
   			module.padding_mode = "circular"

   This will only affect all Conv2d layers of that particular StableDiffusionPipeline, allowing for better granularity of control.

[dancingdays]

Hi. I was just wondering if anyone has successfully implemented tileable outputs with SDXL 1.0? Trying to figure it out so we can implement it for an app via API. Thanks in advance for any insights.