Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sluggish performance on some object-detection models in WebGPU mode #983

Open
2 of 5 tasks
ButzYung opened this issue Oct 19, 2024 · 0 comments
Open
2 of 5 tasks
Labels
bug Something isn't working

Comments

@ButzYung
Copy link

ButzYung commented Oct 19, 2024

System Info

transfomers.js v3.0.0, electron v32.0.1 (Chromium 128), Windows 10 22H2

Environment/Platform

  • Website/web-app
  • Browser extension
  • Server-side (e.g., Node.js, Deno, Bun)
  • Desktop app (e.g., Electron)
  • Other (e.g., VSCode extension)

Description

I was trying to run some object-detection models, namely Xenova/yolos-tiny, in WebGPU mode, but the performance was abnormally sluggish. It's slower than expected, often blocking the UI, and when checking Task Manager the GPU usage sometimes jump to over 90% with the System process using some significant amount of GPU.

I use the standard proceduce to load the pipeline.

const detector = await pipeline('object-detection', 'Xenova/yolos-tiny', { device:'webgpu' });

I am not sure if this is the corrent way as the Xenova/yolos-tiny page on Hugging Face doesn't have an example, but since there are spaces on that page that runs the model properly under WebGPU, I suppose there should be a correct way to make it work.

BTW Xenova/yolos-tiny works properly under CPU mode. I tried a different model like Xenova/detr-resnet-50 in WebGPU mode and it worked properly.

Reproduction

// running under a worker thread
const module = await import('./@huggingface/transformers/dist/transformers.js');// 'https://cdn.jsdelivr.net/npm/@huggingface/[email protected]'); //

module.env.allowLocalModels = false;

const canvas = new OffscreenCanvas(1280,720);
const ctx = canvas.getContext('2d');

const detector = await module.pipeline('object-detection', 'Xenova/yolos-tiny', { device:'webgpu' });

const output = await detector(new module.RawImage(ctx.getImageData(0,0,1280,720).data, 1280,720, 4), { threshold:0.2 });

@ButzYung ButzYung added the bug Something isn't working label Oct 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant