scheduler job not running

[surajsharma]

Tesseract.js version (version number for npm/GitHub release, or specific commit for repo)
5.1.1

Describe the bug

then clause does not get triggerred after first image

ocrJobs[i] = ocrScheduler.addJob('recognize', ocrStack[i]).then((x) => console.log('\n',x.data.text));

To Reproduce

    const ocrWorkers = Array(ocrStack.length);
    const ocrJobs = Array(ocrStack.length);

    for (let i = 0; i < ocrWorkers.length; i++) {
      ocrWorkers[i] = ocrWorkerGen();
    }

    await Promise.all(ocrWorkers);

    for (let i = 0; i < ocrJobs.length; i++) {
      try {
        ocrJobs[i] = ocrScheduler.addJob('recognize', ocrStack[i]).then((x) => console.log('\n',x.data.text));
      } catch (error) {
        console.log("🚀 ~ createSearchIndexDirs ~ error:", error)
      }
    }

    await Promise.all(ocrJobs);

    await ocrScheduler.terminate();

Expected behavior

All images in array ocrScheduler should be processed

Device Version:

• OS + Version: Ubuntu WSL2
• Node 20.10

[Balearica]

Please update the example code so it can be easily run by other users to replicate the issue. This snippet references multiple objects/functions that are not defined in the snippet.

At a glance, one possible issue is that this code appears to create an undefined number of workers, as a worker is created for every element in ocrStack. The maximum number of workers should be capped at a defined value that your server (Node.js) or users devices (browser) can always handle. When used in a browser application a sensible limit is in the 4-8 range.

One of the primary reasons why the scheduler interface exists is to allow for creating a pool of a fixed number of workers. Workers use a significant amount of processing and memory resources, so if you try and run 20 at the same time (for example) your program will crash.

[surajsharma]

if you try and run 20 at the same time (for example) your program will crash.

regardless of this, even if i try to batch one worker at a time (without scheduler), recognize returns without an output after the first image.

[Balearica]

As noted above, if you want any assistance you will need to provide a reproducible example that can be easily run without access to your project (i.e. using only Tesseract.js functions).

[surajsharma]

const { createWorker, createScheduler } = require('tesseract.js');

const ocrScheduler = createScheduler();

const ocrWorkerGen = async () => {
  const worker = await createWorker("eng", 1, { cachePath: "." });
  ocrScheduler.addWorker(worker);
}

• in the code below ocrStack can be an array of image paths, its length can vary from 0 to 100s if not 1000s

• please note that this code is in a loop (there are multiple ocrStacks to process)

const wait1Second = () => new Promise((resolve) => {
  setTimeout(() => {
    console.log('1 second has passed');
    resolve(true);
  }, 1000);
});


const ocrWorkers = Array(ocrStack.length);
const ocrJobs = Array(ocrStack.length);

for (let i = 0; i < ocrWorkers.length; i++) {
  ocrWorkers[i] = ocrWorkerGen();
}

await Promise.all(ocrWorkers);

for (let i = 0; i < ocrJobs.length; i++) {
  try {
    ocrJobs[i] = await ocrScheduler.addJob('recognize', ocrStack[i]);
    await wait1Second();
    console.log("🚀 ~ createSearchIndexDirs ~  ocrJobs[i]:", ocrJobs[i].data.text)
  } catch (error) {
    console.log("🚀 ~ createSearchIndexDirs ~ error:", error)
  }
}

await Promise.all(ocrJobs);

await ocrScheduler.terminate();

i was forced to use the scheduler since the strategies mentioned in the documentation for processing one file at a time did not work either.

your suggestion to batch the images in smaller sized batches does not make sense to me for the snippet above, as well as for code that uses this example i get nothing logged except the first image.

please also note that the strategy to use wait1Second was picked up by me from one of the issues mentioning a similar problem, but it did not work.

[Balearica]

This code appears to again be insufficient for replicating the issue you are describing. I pasted it into a file in the examples/node directory and made basic modifications so it would recognize a few example images in this repo. The entire file is below. It ran as expected with no errors.

To receive assistance, you need to provide a complete, self-contained file that can be copy/pasted into this repo and run (without any edits) to reproduce the error. Please test this yourself before posting. There are many sample images within this repo that can be used for a reproducible example, as occurs in my snippet below. If your error can only be produced using the example images then the inputs that cause it would need to be provided.

#!/usr/bin/env node
const path = require('path');
const { createWorker, createScheduler } = require('../..');

const [,, imagePath] = process.argv;
const image1 = path.resolve(__dirname, (imagePath || '../../tests/assets/images/cosmic.png'));
const image2 = path.resolve(__dirname, (imagePath || '../../tests/assets/images/bill.png'));
const image3 = path.resolve(__dirname, (imagePath || '../../tests/assets/images/testocr.png'));

const ocrScheduler = createScheduler();

const ocrWorkerGen = async () => {
  const worker = await createWorker('eng', 1, { cachePath: '.' });
  ocrScheduler.addWorker(worker);
};

const wait1Second = () => new Promise((resolve) => {
  setTimeout(() => {
    console.log('1 second has passed');
    resolve(true);
  }, 1000);
});

(async () => {
  const ocrStack = [image1, image2, image3];

  const ocrWorkers = Array(3);
  const ocrJobs = Array(ocrStack.length);

  for (let i = 0; i < ocrWorkers.length; i++) {
    ocrWorkers[i] = ocrWorkerGen();
  }

  await Promise.all(ocrWorkers);

  for (let i = 0; i < ocrJobs.length; i++) {
    try {
      ocrJobs[i] = await ocrScheduler.addJob('recognize', ocrStack[i]);
      await wait1Second();
      console.log('🚀 ~ createSearchIndexDirs ~  ocrJobs[i]:', ocrJobs[i].data.text);
    } catch (error) {
      console.log('🚀 ~ createSearchIndexDirs ~ error:', error);
    }
  }

  await Promise.all(ocrJobs);

  await ocrScheduler.terminate();
})();

[surajsharma]

thanks! kind of defeats the purpose of the lib if i have to provide the images imo..