Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ERR_MMEORY_OVERLAP #62

Open
glebss opened this issue Aug 8, 2023 · 0 comments
Open

ERR_MMEORY_OVERLAP #62

glebss opened this issue Aug 8, 2023 · 0 comments

Comments

@glebss
Copy link

glebss commented Aug 8, 2023

Hello.

I am trying to compile and run my custom model. I use custom-model-onnx.ipynb notebook as a basis.
It compiles fine to onnx format. After several modifications, all layers are now supported by TIDL. Nevertheless, when I do

img = np.zeros((1, 3, 608, 960), dtype='float32')
output = list(sess.run(['det_out', 'laneaf_out'],
                           input_feed={'image': img}))

in the error log I have

Error : Error Code = <ERR_MMEORY_OVERLAP>
Segmentation fault (core dumped)

By commenting in and out some parts of my code I figured out that something illegal happens in this code snippet:

xs_out = torch.narrow(y, 2, 12, 4) + torch.cat([torch.narrow(grid, 2, 0, 1)] * 4, dim=2)
res_out = torch.cat([bboxes_out, xs_out], 2)

where y is a torch.Tensor of size (1, 3, 24, 9120), grid is a torch.Tensor of shape (1, 3, 2, 9120). Therefore, here two tensors of the exact same shape (1, 3, 4, 9120) are added together.

However, when I skip addition and leave only

xs_out = torch.narrow(y, 2, 12, 4)
res_out = torch.cat([bboxes_out, xs_out], 2)

it compiles and runs without errors.

My compilation parameters are the following

compile_options = {
        'tidl_tools_path' : os.environ['TIDL_TOOLS_PATH'],
        'artifacts_folder' : output_dir,
        'tensor_bits' :  8,
        'accuracy_level' : 0,
        'advanced_options:calibration_frames' : len(calib_images), 
        'advanced_options:calibration_iterations' : 0, # used if accuracy_level = 1
        'debug_level' :  2
    }

Have no idea how to fix it myself. Is there any way to debug it?

Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant