Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Empty SparseTensor convolution segfault #591

Open
francois-drielsma opened this issue May 17, 2024 · 1 comment
Open

Empty SparseTensor convolution segfault #591

francois-drielsma opened this issue May 17, 2024 · 1 comment

Comments

@francois-drielsma
Copy link

Describe the bug
When passing an empty SparseTensor object through a MinkowskiConvolution, MinkowskiEngine throws a segfault.


To Reproduce
Docker image: deeplearnphysics/larcv2:ub20.04-cuda12.1-pytorch2.2.1-larndsim

Code snippet

import torch
import MinkowskiEngine as ME

input = ME.SparseTensor(coordinates=torch.empty(0,4).to(dtype=torch.int), features=torch.empty(0,1))
conv = ME.MinkowskiConvolution(in_channels=1, out_channels=3, dimension=3, kernel_size=3)
conv(input)

Expected behavior
Would hope for an empty output with the appropriate number of features, e.g.:

SparseTensor(
  coordinates=tensor([], size=(0, 4), dtype=torch.int32)
  features=tensor([], size=(0, 3), grad_fn=<MinkowskiConvolutionFunctionBackward>)
  coordinate_map_key=coordinate map key:[1, 1, 1]
  coordinate_manager=CoordinateMapManagerCPU(
	[1, 1, 1, ��]:	CoordinateMapCPU:0x4
	algorithm=MinkowskiAlgorithm.DEFAULT
  )
  spatial dimension=3)

Desktop (please complete the following information):

  • OS: Ubuntu 20.04
  • Python version: 3.8.10
  • Pytorch version: 2.2.1
  • CUDA version: 12.1
  • NVIDIA Driver version: 535.161.07
  • Minkowski Engine version: 0.5.4
  • Output of the following command. (If you installed the latest MinkowskiEngine, paste the output of python -c "import MinkowskiEngine as ME; ME.print_diagnostics()". Otherwise, paste the output of the following command.)
/usr/local/lib/python3.8/dist-packages/MinkowskiEngine/__init__.py:36: UserWarning: The environment variable `OMP_NUM_THREADS` not set. MinkowskiEngine will automatically set `OMP_NUM_THREADS=16`. If you want to set `OMP_NUM_THREADS` manually, please export it on the command line before running a python script. e.g. `export OMP_NUM_THREADS=12; python your_program.py`. It is recommended to set it below 24.
 warnings.warn(
==========System==========
Linux-4.18.0-372.32.1.el8_6.x86_64-x86_64-with-glibc2.29
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.6 LTS"
3.8.10 (default, Nov 22 2023, 10:22:35) 
[GCC 9.4.0]
==========Pytorch==========
2.2.1+cu121
torch.cuda.is_available(): True
==========NVIDIA-SMI==========
/usr/bin/nvidia-smi
Driver Version 535.161.07
CUDA Version 12.2
VBIOS Version 92.00.19.00.13
Image Version G506.0202.00.02
GSP Firmware Version 535.161.07
==========NVCC==========
/usr/local/cuda/bin/nvcc
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Mon_Apr__3_17:16:06_PDT_2023
Cuda compilation tools, release 12.1, V12.1.105
Build cuda_12.1.r12.1/compiler.32688072_0
==========CC==========
/usr/bin/c++
c++ (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

==========MinkowskiEngine==========
0.5.4
MinkowskiEngine compiled with CUDA Support: True
NVCC version MinkowskiEngine is compiled: 12010
CUDART version MinkowskiEngine is compiled: 12010

Additional context
I did not do extensive version testing, but this seems to work for torch 1.13 in an older container: deeplearnphysics/larcv2:ub20.04-cuda11.6-pytorch1.13-larndsim

@francois-drielsma
Copy link
Author

Having a similar issue with MinkowskiConvolutionTranspose, even with the older version of PyTorch. This:

import torch
import MinkowskiEngine as ME

input = ME.SparseTensor(coordinates=torch.empty(0,4).to(dtype=torch.int), features=torch.empty(0,1))
conv = ME.MinkowskiConvolution(in_channels=1, out_channels=4, kernel_size=[2, 2, 2], stride=[2, 2, 2], dilation=[1, 1, 1], dimension=3)
conv_t = ME.MinkowskiConvolutionTranspose(in_channels=4, out_channels=4, kernel_size=[2, 2, 2], stride=[2, 2, 2], dilation=[1, 1, 1], dimension=3)
x = conv(input)
x = conv_t(x)

throws a segfault on the last line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant