分批预处理，不一次性读取
#320
Replies: 3 comments
-
|
意义不大，可以自己写一个，我这边从来没遇到过。另外你内存多少() |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
内存16G，数据集的wavs文件夹里算上.wav emo和bert一共不到1G，那个模型也才2G，不知道为什么会卡住(windows的表都不走了) |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
bert模型的话加载是比较吃内存的，可以尝试开1 worker，另外16G跑训练确实捉襟见肘了 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
bert_gen.py里一个with open干爆了我的内存和虚拟内存。能不能当数据集很大时分批预处理
Beta Was this translation helpful? Give feedback.
All reactions