Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ImageProcessor doesn't run through tasks during process since 0.3.1 #50

Closed
espbee opened this issue Dec 31, 2021 · 1 comment
Closed

Comments

@espbee
Copy link

espbee commented Dec 31, 2021

I'm running a custom Yolov4 model (also a Yolo3 for what it's worth) through here. Everything was lovely while I ran tflite_flutter 0.8.0 with 0.2.0 the helper. Since upgrading to 0.9.0 and 0.3.1, I don't get a suitable image input to run through the predictor, and it winds up breaking.

More specifically, I pre-process the camera frame after it's been wrapped in a TensorImage, and send it through the ImageProcessor--

and afterwards, in runForMultipleInputs, I getInputTensors and then try to run inputTensors.elementAt(i).setTo()inputs[i]). But there's an error from within setTo()-- one of the checks fails and throws a Bad state: failed precondition.

Back during image prep, I run the ImageProcessor from this helper lib like this:

imageProcessor ??= ImageProcessorBuilder()
.add(ResizeWithCropOrPadOp(padSize!, padSize!))
.add(ResizeOp(INPUT_SIZE.toInt(), INPUT_SIZE.toInt(), ResizeMethod.NEAREST_NEIGHBOUR))
.add(NormalizeOp(1, 255))
.build();

inputImage = imageProcessor!.process(inputImage);

I think I've isolated the problem.

  • In the functional app running the old libraries, I can print out what's happening as it runs through these operations in image_processor.dart. I see the rects and the points forming as it goes.

  • When I run the updated libs, it does make an instance of the ImageProcessor, and that instance does have an operator list and an operator index that look like you'd expect. But I don't get any output when I try to print the actually running operations in image_processor.dart . No rects or points. Nothing there. i can wrap things in try / catches and it doesn't hit them. It doesn't seem to run through them at all when i run imageProcesor!.process(inputImage) (with a TensorImage wrapping an im.Image that converted correctly from a camera frame image as inputImage).

  • NOTE though: if I print out what's happening in the actual process() function in sequential_processor.dart it is iteratring through the list there and returning an instance of TensorImage at the bottom.If I wrap the little loop inside process() in a try catch, it doesn't throw anything and proceeds on to the bottom. But again, nothing in image_processor.dart, no points or rects.

After running the TensorImage wrapped im.Image through the processor in the functional app using the older libs, I can no longer see the image as an im.image-- it's a float-value image instead and throws an error if I try to access image properties. However, in the non-functionla app with the newer libs, I can still see the image afterward, thogh it's byte length is slightly more than half of what it had been.

Any insight would be appreciated... Is the idea just to run those ops manually, do you think? Or has anyone run into anything like this?

@espbee espbee changed the title ImageProcessorBuilder() doesn't run through tasks during process since 0.3.1 ImageProcessor doesn't run through tasks during process since 0.3.1 Dec 31, 2021
@espbee
Copy link
Author

espbee commented Jan 1, 2022

Solved. If anyone happens upon this in a similar boat--

In the older lib, to make the inputImage mentioned above, I could do
TensorImage inputImage = TensorImage.fromImage(image), image being an image lib image.

Now I had to specify the type, like:
TensorImage inputImage = TensorImage(TfliteType.float32);
inputImage.loadImage(image);

Thanks to @sDobrzanski
in am15h/tflite_flutter_plugin#167

@espbee espbee closed this as completed Jan 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant