Sluggish performance on some object-detection models in WebGPU mode

[ButzYung]

System Info

transfomers.js v3.0.0, electron v32.0.1 (Chromium 128), Windows 10 22H2

Environment/Platform

• Website/web-app
• Browser extension
• Server-side (e.g., Node.js, Deno, Bun)
• Desktop app (e.g., Electron)
• Other (e.g., VSCode extension)

Description

I was trying to run some object-detection models, namely Xenova/yolos-tiny, in WebGPU mode, but the performance was abnormally sluggish. It's slower than expected, often blocking the UI, and when checking Task Manager the GPU usage sometimes jump to over 90% with the System process using some significant amount of GPU.

I use the standard proceduce to load the pipeline.

const detector = await pipeline('object-detection', 'Xenova/yolos-tiny', { device:'webgpu' });

I am not sure if this is the corrent way as the Xenova/yolos-tiny page on Hugging Face doesn't have an example, but since there are spaces on that page that runs the model properly under WebGPU, I suppose there should be a correct way to make it work.

BTW Xenova/yolos-tiny works properly under CPU mode. I tried a different model like Xenova/detr-resnet-50 in WebGPU mode and it worked properly.

Reproduction

// running under a worker thread
const module = await import('./@huggingface/transformers/dist/transformers.js');// 'https://cdn.jsdelivr.net/npm/@huggingface/[email protected]'); //

module.env.allowLocalModels = false;

const canvas = new OffscreenCanvas(1280,720);
const ctx = canvas.getContext('2d');

const detector = await module.pipeline('object-detection', 'Xenova/yolos-tiny', { device:'webgpu' });

const output = await detector(new module.RawImage(ctx.getImageData(0,0,1280,720).data, 1280,720, 4), { threshold:0.2 });