Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large Databases Support In PATE-CTGAN #612

Open
RYCBABAC opened this issue Dec 15, 2024 · 0 comments
Open

Large Databases Support In PATE-CTGAN #612

RYCBABAC opened this issue Dec 15, 2024 · 0 comments

Comments

@RYCBABAC
Copy link

When I ran a synthesizer with a PATE-CTGAN model over a large database (not too large tbh, a little bit less than 100MB) the fit will fail and the kernel will crash.
I tried to debug the problem and figured out the model doesn't even start the fitting process but gets stuck at the _get_train_data method. I assume that there is some inefficient memory load while transforming the train data, but I didn't look too deep into this code segment :).
I ran this over a process with 400GB, so memory shouldn't be an issue, unless something really weird is happening.
The code I ran is the same code as the one in PATE-CTGAN in the smart noise README.md file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant