You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I built a SDXL engine via TensorRT Exporter tab doing the following:
I selected "768x768 - 1024x1024|Batch Size 1-4" from the dropdown
I opened Advanced settings and changed the settings to the following
Min/Optimal/Max Batch Size all set to 4
Min/Optimal/Max Width & Height all set to 1024
Min/Optimal prompt token count 75
Max prompt token count 300
After generating the engine with above-mentioned parameters, the resulting engine actually had all batch sizes set to 2 instead of 4. When inspecting the engine through the TensorTR tab, the reported batch sizes are all set to 2 instead of 4. (It's worth noting that the console process of building the engine reported correct parameters during the build.)
To verify whether it is a UI bug, I attempted to generate images using the just generated engine. When trying to generate images with batch size 4, the generation failed reporting back to UI error "RuntimeError: The size of tensor a (4) must match the size of tensor b (8) at non-singleton dimension 0".
Console reports exception:
File "\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 70, in forward
self.engine.allocate_buffers(feed_dict)
File "\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 304, in allocate_buffers
tuple(shape), dtype=numpy_to_torch_dtype_dict[dtype]
ValueError: __len__() should return >= 0
However, as one would expect, when I tried to generate images with batch size 2, it proceeded to do so without issue, therefore confirming that the generated engine indeed has wrong parameters.
I went to regenerate the engine again, trying to get it to properly generate with batch size 4, but it reported that the engine I am trying to generate already exists. Forcing regeneration regenerated it again with batch size 2 instead of batch size 4 that I wanted.
I proceeded in the endeavor of trying to produce an engine with batch size 4, this time changing min batch size to 2.
After generating the engine, the result had min/optimal/max batch size set to 1/4/4. Still not correct, but close enough this time to be usable while I wait for a fix.
The text was updated successfully, but these errors were encountered:
I built a SDXL engine via TensorRT Exporter tab doing the following:
Min/Optimal/Max Batch Size all set to 4
Min/Optimal/Max Width & Height all set to 1024
Min/Optimal prompt token count 75
Max prompt token count 300
After generating the engine with above-mentioned parameters, the resulting engine actually had all batch sizes set to 2 instead of 4. When inspecting the engine through the TensorTR tab, the reported batch sizes are all set to 2 instead of 4. (It's worth noting that the console process of building the engine reported correct parameters during the build.)
To verify whether it is a UI bug, I attempted to generate images using the just generated engine. When trying to generate images with batch size 4, the generation failed reporting back to UI error "RuntimeError: The size of tensor a (4) must match the size of tensor b (8) at non-singleton dimension 0".
Console reports exception:
However, as one would expect, when I tried to generate images with batch size 2, it proceeded to do so without issue, therefore confirming that the generated engine indeed has wrong parameters.
I went to regenerate the engine again, trying to get it to properly generate with batch size 4, but it reported that the engine I am trying to generate already exists. Forcing regeneration regenerated it again with batch size 2 instead of batch size 4 that I wanted.
I proceeded in the endeavor of trying to produce an engine with batch size 4, this time changing min batch size to 2.
After generating the engine, the result had min/optimal/max batch size set to 1/4/4. Still not correct, but close enough this time to be usable while I wait for a fix.
The text was updated successfully, but these errors were encountered: