Replies: 19 comments 49 replies
-
|
thanks you very much you really saved me a lot of hassle |
Beta Was this translation helpful? Give feedback.
-
|
Exact same problem here. Any fixes? |
Beta Was this translation helpful? Give feedback.
-
|
The ubuntu_setup.sh has an error but doesn't block the rest of the script: Is it something expected? |
Beta Was this translation helpful? Give feedback.
-
|
I suppose that's the error of unable to install TK the author is talking, I think it happens to everyone, just after the script finished I installed TK with the two scripts the author of this post provided. But I'm unable to use the UI yet for the error y wrote below in other comment |
Beta Was this translation helpful? Give feedback.
-
|
I've installed TK manually as the author noted, previously, this error might not be related to it. |
Beta Was this translation helpful? Give feedback.
-
|
Well, line 3 of that file says "sudo apt install python3-tk" I'm pretty sure that the installation of TK, I just installed after the ubuntu.sh script, but as I said, even after launching the UI I'm unable to use Kohya for other error, better wait to solve those error that make this unusable. I'm thinking to use a Linux Desktop Template that makes everything easier |
Beta Was this translation helpful? Give feedback.
-
|
I think they just updated the requirements ubuntu.sh like 12 hours ago and now tk installs automatically without more problem, at least now that im trying this again |
Beta Was this translation helpful? Give feedback.
-
|
Hi! I get this error I tried installed tkinter using |
Beta Was this translation helpful? Give feedback.
-
|
Everything was installed correctly, although I did remove sudo and placed apt update before installing pythontk. Folder 100_nnrmml: 0 images found [Dataset 0] The images are in the folder correctly. Locally it works. |
Beta Was this translation helpful? Give feedback.
-
|
Doing this solves the missing library issues, but still no images found. Make sure to run gui.sh in the same terminal you set the envs. cd /workspace/kohya_ss/venv/lib/python3.10/site-packages/tensorrt cd /workspace/kohya_ss/venv/lib/python3.10/site-packages/nvidia/cuda_runtime/lib export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/workspace/kohya_ss/venv/lib/python3.10/site-packages/tensorrt/ |
Beta Was this translation helpful? Give feedback.
-
|
The problem was that I had .JPG (with capital letters) and in lora_gui.py there were if statements checking for lowercase jpg ending. I've added if for JPG but most probably something after this also did check because it found at the beginning but not after. After renaming the images to *.jpg it works. I'm able to train the lora. use AdamW optimizer | {} |
Beta Was this translation helpful? Give feedback.
-
|
Strangely my virtual environment doesn't have tensorrt. Should I just manually install it with pip? |
Beta Was this translation helpful? Give feedback.
-
|
I think so. You can follow this: |
Beta Was this translation helpful? Give feedback.
-
|
I've written down the process of how I make it work on runpod. Hope it helps. Run sd 1.5 runpod template with at least 15gb+ container storage and 30gb+ persistent storageExectute the following commands in jupiter terminalhttps://www.runpod.io/console/gpu-secure-cloud?template=runpod-stable1. Optional, kill the SD server if you don't need itfuser -k 3000/tcp 2. Setup kohya_ssgit clone https://github.com/bmaltais/kohya_ss.git Choose "This machine", "No distributed training", 3x "NO", then type "all", and finally select fp163. Install tensorrtsource /workspace/kohya_ss/venv/bin/activate 4. Fix missing libnvinfer.so.7 and libnvinfer_plugin.so.7 librariescd /workspace/kohya_ss/venv/lib/python3.10/site-packages/tensorrt 5. Not sure if it's needed but I always rerun the ubuntu_setup.sh after post-installing tensorrt, it downloads some additional things. Choose the same options as before.cd /workspace/kohya_ss 6. Optional install gdown and pull your datapip install gdown && apt install -y unzip 7. Start the servercd /workspace/kohya_ss && ./gui.sh |
Beta Was this translation helpful? Give feedback.
-
I usually have the models on google drive and just use gdown to pull it. |
Beta Was this translation helpful? Give feedback.
-
|
Oh gdown creates a copy? I always used !mv in Collab and then I had to upload all things again to drive after I finished XD. Thanks for the tip |
Beta Was this translation helpful? Give feedback.
-
|
I can done step 2. setup kohya_ss and then to install tensorrt, type below.. but, root@14c1f5703258:/workspace/kohya_ss# source /workspace/kohya_ss/venv/bin/activate Why it does not work?? why I can not find file /workspace/kohya_ss/venv/bin/activate please help me... |
Beta Was this translation helpful? Give feedback.
-
which pod are you using? I encountered the same issue when using the fast stable diffusion pod, But it works for me using the stable difussion 1.5 pod |
Beta Was this translation helpful? Give feedback.
-
|
I have a few updates for @ppetrucz 's excellent walkthrough. I was gonna post a separate script version, but you can copy/paste it yourself and comment out the appropriate lines. Changes:
(Not sure if gcc toolkit is actually needed) Note that to successfully run LORA training, I did have to change |
Beta Was this translation helpful? Give feedback.
-
|
thank you
I changed to SD 1.5 pod. This trouble is cleared.
but there is another issue..
NameError: st2optimizer8bit_blockwise
When I changed to AdamW instead AdamW8bit, it is cleared but... very slow
trainning speed .. ... T.T
2023년 3월 24일 (금) 오후 5:44, HuanchengHu ***@***.***>님이 작성:
… I can done step 2. setup kohya_ss select fp16
and then to install tensorrt, type below.. but,
***@***.***:/workspace/kohya_ss# source
/workspace/kohya_ss/venv/bin/activate bash:
/workspace/kohya_ss/venv/bin/activate: No such file or directory
Why it does not work?? why I can not find file
/workspace/kohya_ss/venv/bin/activate
please help me...
which pod are you using? I encountered the same issue when using the fast
stable diffusion pod, But it works for me using the stable difussion 1.5 pod
—
Reply to this email directly, view it on GitHub
<#379 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AK7SS7Q5ULPAW4WEPVEZRZ3W5VNIFANCNFSM6AAAAAAV2TIMXQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Here's a sh script to make everything at once and start the gui : run.sh put that in another script to restart gui if needed : restart.sh |
Beta Was this translation helpful? Give feedback.
-
|
after cloning the repo when we run ubuntu_setup.sh there's no file or directory. The only two files are setup.sh and gui.sh |
Beta Was this translation helpful? Give feedback.
-
same |
Beta Was this translation helpful? Give feedback.
-
|
if you don't want Auto1111 to run, edit pod, expand the Environment Variables section, add a key killing the Auto1111 server with |
Beta Was this translation helpful? Give feedback.
-
|
can someone please fix NameError: st2optimizer8bit_blockwise |
Beta Was this translation helpful? Give feedback.
-
|
Not sure about fixing it but till then, as a workaround you can use adamW. |
Beta Was this translation helpful? Give feedback.
-
|
how to do that? |
Beta Was this translation helpful? Give feedback.
-
|
@dpyy change the setting in the gui, might be down lower in Advanced Settings. |
Beta Was this translation helpful? Give feedback.
-
Here is the problem I got,
|
Beta Was this translation helpful? Give feedback.
-
|
I've seen that before, it might be because you have spaces somewhere in
your filenames, probably in the images names...
Le jeu. 30 mars 2023 à 20:11, SGKino ***@***.***> a écrit :
… Here is the problem I got, and don't know how to fix... Traceback (most
recent call last):
File
"/workspace/kohya_ss/venv/lib/python3.10/site-packages/gradio/routes.py",
line 384, in run_predict
output = await app.get_blocks().process_api(
File
"/workspace/kohya_ss/venv/lib/python3.10/site-packages/gradio/blocks.py",
line 1024, in process_api
result = await self.call_function(
File
"/workspace/kohya_ss/venv/lib/python3.10/site-packages/gradio/blocks.py",
line 836, in call_function
prediction = await anyio.to_thread.run_sync(
File
"/workspace/kohya_ss/venv/lib/python3.10/site-packages/anyio/to_thread.py",
line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File
"/workspace/kohya_ss/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py",
line 937, in run_sync_in_worker_thread
return await future
File
"/workspace/kohya_ss/venv/lib/python3.10/site-packages/anyio/_backends/
*asyncio.py", line 867, in run result = context.run(func, *args) File
"/workspace/kohya_ss/lora_gui.py", line 422, in train_model repeats =
int(folder.split('*')[0])
ValueError: invalid literal for int() with base 10: '.ipynb'
Folder 100_CandyL: 15 images found
Folder 100_CandyL: 1500 steps
Traceback (most recent call last):
File
"/workspace/kohya_ss/venv/lib/python3.10/site-packages/gradio/routes.py",
line 384, in run_predict
output = await app.get_blocks().process_api(
File
"/workspace/kohya_ss/venv/lib/python3.10/site-packages/gradio/blocks.py",
line 1024, in process_api
result = await self.call_function(
File
"/workspace/kohya_ss/venv/lib/python3.10/site-packages/gradio/blocks.py",
line 836, in call_function
prediction = await anyio.to_thread.run_sync(
File
"/workspace/kohya_ss/venv/lib/python3.10/site-packages/anyio/to_thread.py",
line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File
"/workspace/kohya_ss/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py",
line 937, in run_sync_in_worker_thread
return await future
File
"/workspace/kohya_ss/venv/lib/python3.10/site-packages/anyio/_backends/
*asyncio.py", line 867, in run result = context.run(func, *args) File
"/workspace/kohya_ss/lora_gui.py", line 422, in train_model repeats =
int(folder.split('*')[0])
ValueError: invalid literal for int() with base 10: '.ipynb'
—
Reply to this email directly, view it on GitHub
<#379 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABEKG55H2JPF6Q2V5RFGUWLW6XEGJANCNFSM6AAAAAAV2TIMXQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Thank you for your reply.
|
Beta Was this translation helpful? Give feedback.
-
|
I finally got the location right... |
Beta Was this translation helpful? Give feedback.
-
|
Here I got a new problem now. Can anyone tell me what should I do? |
Beta Was this translation helpful? Give feedback.
-
|
this effing tkinter is preventing me from using kohya_ss on runpod for weeks now, Validating that requirements are satisfied. |
Beta Was this translation helpful? Give feedback.
-
|
"fuser -k 3000/tcp" should do the trick |
Beta Was this translation helpful? Give feedback.
-
|
worksss. |
Beta Was this translation helpful? Give feedback.
-
|
sucessfully ran in this method, but having issues trying to train Lora as follow. Any thoughts? Thanks. Traceback (most recent call last): |
Beta Was this translation helpful? Give feedback.
-
There must be a .ipynb folder created in your image folder. Its hidden so do "ls -a" , And delete it . |
Beta Was this translation helpful? Give feedback.
-
|
dam!! you are awesome!! a millions thx!! |
Beta Was this translation helpful? Give feedback.
-
|
Here is my setup script: |
Beta Was this translation helpful? Give feedback.
-
|
The script seems to complete just fine, but the gui doesn't work for opening folders or selecting files for some reason. |
Beta Was this translation helpful? Give feedback.
-
|
oO you do have jupyter's file finder in the bg, it's hindered yes but you could also in suggestion add emjoy finder to the thigns you could adhoc to your setup. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you. It;s work |
Beta Was this translation helpful? Give feedback.
-
|
Anyone ever notice you CAN use ocalmfuse for gdrive? runpodctl never works for me it confuses me and drives me nuts - but evidently in theory you can install ocalmfuse on anyhting that's not colab XD and google's the one that developed it. |
Beta Was this translation helpful? Give feedback.
-
|
Edit: ignore this, unless you're interested in the cause, use the script 15ky3 mentioned, and see my comment below it if your python 3 still complains about tkinter after that. I tried running on runpod, After install the tkinter bug strikes. Nothing mentioned here really resolves it. There is no ubuntu_upgrade.sh anymore, only upgrade.sh. If you install the python3-tk via apt-get the links still point to python 3.10, you need to make python link to python 3.8, because only that one has tkinter. I just removed the links and recreated the new ones. cd venv/bin rm python3 ln -s /usr/bin/python3.8 python3 however, then you're missing pip, which you need to install using however, before installing it, you need to
|
Beta Was this translation helpful? Give feedback.
-
|
Maybe you can use my Script for vast.ai provided here. Maybe you should, if it downt work with the script, execute: |
Beta Was this translation helpful? Give feedback.
-
|
thanks, might have helped, but what I also needed was: apt-get install python3.10-tk
apt install python3.10-distutils
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python3.10 get-pip.pyIf you do not specify 3.10 explicitly you get python3.8 which is ignored when running gui.sh |
Beta Was this translation helpful? Give feedback.
-
|
If someone its having problems with this error very recently: i just with the virtual environment activated updated pytorch to use cuda 11.8: pip install torch==2.0.0+cu118 torchvision==0.15.1+cu118 torchaudio==2.0.1 --index-url https://download.pytorch.org/whl/cu118 |
Beta Was this translation helpful? Give feedback.
-
|
didn't work for me, same error |
Beta Was this translation helpful? Give feedback.
-
|
I have reworked the runpod setup. It should now be easier then ever to run on runpod. Run the Torch 2.0.1 template. I hope I have not broken everything else in the process... If I did I appoligise and let's work on making runpor support top notch |
Beta Was this translation helpful? Give feedback.
-
|
Install seems to work, however no venv is created. I manually installed python3.10-venv and python3-venv and it still won't make and use it. I am able to manually make a venv with python3.10 -m venv but it won't use the venv that is already created inside the kohya_ss folder. EDIT: |
Beta Was this translation helpful? Give feedback.
-
|
Probably a noob question, but if I have a multi-gpu runpod, would I need to consider the distributed training when installing Kohya? |
Beta Was this translation helpful? Give feedback.
and others
-
I had some trouble with the other linux ports (& the kohya_ss-linux that runpod has as a template)
instead you can use the latest bmaltais/kohya_ss fork:
deploy their existing RunPod Stable Diffusion v1.5 template
git clone https://github.com/bmaltais/kohya_ss.gitcd kohya_ssbash ubuntu_setup.shwait a while
the
ubuntu_setup.shinstalls tk, but it didn't seem to work for me, I had to do this manually afterward with:apt-get update -yapt-get install python3-tk -yubuntu_setup also creates the venv, for reference the command to create it is
python3 -m venv venvif you ever need to activate the venv yourself use
source venv/bin/activatebefore running gui.sh, in Jupiter Lab explorer tab:
open kohya_gui.py
back to terminal tab:
bash gui.shclick the public grado link
in kohya gui:
Folders tab, you have to manually set the images and output folder, ex:
/workspace/imagesand/workspace/uploading training data can be done via Jupiter
can also use
runpodctl(better to zip data before sending too) https://github.com/runpod/runpodctlget latest binary for your local PC
usage in windows
cmd runpodctl send filename.extcopy the command it prints
in a Jupiter terminal, paste the command, the file sends
install unzip if needed
apt install unzipex usage
unzip file.zip -d toFolderkohya gui notes
Beta Was this translation helpful? Give feedback.
All reactions