Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question about data normalization #9

Closed
TArdelean opened this issue Nov 11, 2024 · 2 comments
Closed

Question about data normalization #9

TArdelean opened this issue Nov 11, 2024 · 2 comments

Comments

@TArdelean
Copy link

Hello,
Firstly, thanks for the great research and sharing your implementation!
I have a question regarding data normalization when using the sd-vae-ft-mse latent encoder. In your source code you encode_pixels before applying the VAE, which scales the images in [0, 1]. However, as far as I can tell, the SD VAE was trained using a [-1, 1] normalization (huggingface/diffusers#3726 (comment)).
Could you please clarify this discrepancy?

Thanks,
Timotei

@SonicCodes
Copy link

def encode_pixels(self, x): # raw pixels => raw latents

here's where the encoding happens to be exact , the OpenAI consistency paper apparently discovered this:
image

@tkarras @TArdelean

Thanks,

@TArdelean
Copy link
Author

I see, that clears it up.
Thanks @SonicCodes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants