Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

integration tests with numpy 1.24 #5776

Closed
wyli opened this issue Dec 20, 2022 · 2 comments · Fixed by #5778
Closed

integration tests with numpy 1.24 #5776

wyli opened this issue Dec 20, 2022 · 2 comments · Fixed by #5778
Assignees
@wyli

Comments

@wyli
Copy link
Contributor

wyli commented Dec 20, 2022

there is some integration issue with the newly released numpy 1.24.0 (https://github.com/numpy/numpy/releases)

[2022-12-20T01:19:54.990Z] EMONAI version: 1.1.0+1.g69463a69
[2022-12-20T01:19:54.990Z] Numpy version: 1.24.0
[2022-12-20T01:19:54.990Z] Pytorch version: 1.13.1+cu117
[2022-12-20T01:19:54.990Z] MONAI flags: HAS_EXT = True, USE_COMPILED = True, USE_META_DICT = False
[2022-12-20T01:19:54.990Z] MONAI rev id: 69463a69637d4383056455b1b12bdff58294363e
[2022-12-20T01:19:54.990Z] MONAI __file__: /home/jenkins/agent/workspace/Monai-latest-image/monai/__init__.py
[2022-12-20T01:19:54.990Z] 
[2022-12-20T01:19:54.990Z] Optional dependencies:
[2022-12-20T01:19:54.990Z] Pytorch Ignite version: 0.4.10
[2022-12-20T01:19:54.990Z] Nibabel version: 4.0.2
[2022-12-20T01:19:54.990Z] scikit-image version: 0.19.3
[2022-12-20T01:19:54.990Z] Pillow version: 9.3.0
[2022-12-20T01:19:54.990Z] Tensorboard version: 1.15.0+nv
[2022-12-20T01:19:54.990Z] gdown version: 4.6.0
[2022-12-20T01:19:54.990Z] TorchVision version: 0.14.1+cu117
[2022-12-20T01:19:54.990Z] tqdm version: 4.53.0
[2022-12-20T01:19:54.990Z] lmdb version: 1.1.1
[2022-12-20T01:19:54.990Z] psutil version: 5.8.0
[2022-12-20T01:19:54.990Z] pandas version: 1.1.4
[2022-12-20T01:19:54.990Z] einops version: 0.6.0
[2022-12-20T01:19:54.990Z] transformers version: 4.21.3
[2022-12-20T01:19:54.990Z] mlflow version: 2.0.1
[2022-12-20T01:19:54.990Z] pynrrd version: 1.0.0
[2022-12-20T01:19:54.990Z] 
[2022-12-20T01:19:54.990Z] For details about installing the optional dependencies, please visit:
[2022-12-20T01:19:54.990Z]     https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies
[2022-12-20T01:19:54.990Z] 
[2022-12-20T01:19:54.990Z] E
[2022-12-20T01:19:54.990Z] ======================================================================
[2022-12-20T01:19:54.990Z] ERROR: test_timing (__main__.IntegrationSegmentation3D)
[2022-12-20T01:19:54.990Z] ----------------------------------------------------------------------
[2022-12-20T01:19:54.990Z] monai.utils.module.OptionalImportError: from torch.utils.tensorboard import SummaryWriter (module 'numpy' has no attribute 'object').
[2022-12-20T01:19:54.990Z] 
[2022-12-20T01:19:54.990Z] For details about installing the optional dependencies, please visit:
[2022-12-20T01:19:54.990Z]     https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies
[2022-12-20T01:19:54.990Z] 
[2022-12-20T01:19:54.990Z] The above exception was the direct cause of the following exception:
[2022-12-20T01:19:54.990Z] 
[2022-12-20T01:19:54.990Z] Traceback (most recent call last):
[2022-12-20T01:19:54.990Z]   File "/home/jenkins/agent/workspace/Monai-latest-image/tests/utils.py", line 616, in _wrapper
[2022-12-20T01:19:54.990Z]     raise RuntimeError(res.traceback) from res
[2022-12-20T01:19:54.990Z] RuntimeError: Traceback (most recent call last):
[2022-12-20T01:19:54.990Z]   File "/home/jenkins/agent/workspace/Monai-latest-image/tests/utils.py", line 567, in run_process
[2022-12-20T01:19:54.990Z]     output = func(*args, **kwargs)
[2022-12-20T01:19:54.990Z]   File "/home/jenkins/agent/workspace/Monai-latest-image/tests/utils.py", line 648, in _call_original_func
[2022-12-20T01:19:54.990Z]     return f(*args, **kwargs)
[2022-12-20T01:19:54.990Z]   File "/home/jenkins/agent/workspace/Monai-latest-image/tests/test_integration_segmentation_3d.py", line 298, in test_timing
[2022-12-20T01:19:54.990Z]     self.train_and_infer(idx=3)
[2022-12-20T01:19:54.990Z]   File "/home/jenkins/agent/workspace/Monai-latest-image/tests/test_integration_segmentation_3d.py", line 261, in train_and_infer
[2022-12-20T01:19:54.990Z]     losses, best_metric = run_training_test(self.data_dir, device=self.device, cachedataset=idx, readers=_readers)
[2022-12-20T01:19:54.990Z]   File "/home/jenkins/agent/workspace/Monai-latest-image/tests/test_integration_segmentation_3d.py", line 118, in run_training_test
[2022-12-20T01:19:54.990Z]     writer = SummaryWriter(log_dir=os.path.join(root_dir, "runs"))
[2022-12-20T01:19:54.990Z]   File "/home/jenkins/agent/workspace/Monai-latest-image/monai/utils/module.py", line 415, in __call__
[2022-12-20T01:19:54.990Z]     raise self._exception
[2022-12-20T01:19:54.990Z]   File "/home/jenkins/agent/workspace/Monai-latest-image/monai/utils/module.py", line 367, in optional_import
[2022-12-20T01:19:54.990Z]     pkg = __import__(module)  # top level module
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/torch/utils/tensorboard/__init__.py", line 12, in <module>
[2022-12-20T01:19:54.990Z]     from .writer import FileWriter, SummaryWriter  # noqa: F401
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/torch/utils/tensorboard/writer.py", line 13, in <module>
[2022-12-20T01:19:54.990Z]     from tensorboard.summary.writer.event_file_writer import EventFileWriter
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/tensorboard/summary/__init__.py", line 25, in <module>
[2022-12-20T01:19:54.990Z]     from tensorboard.summary import v1
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/tensorboard/summary/v1.py", line 26, in <module>
[2022-12-20T01:19:54.990Z]     from tensorboard.plugins.histogram import summary as _histogram_summary
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/tensorboard/plugins/histogram/summary.py", line 38, in <module>
[2022-12-20T01:19:54.990Z]     from tensorboard.plugins.histogram import summary_v2
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/tensorboard/plugins/histogram/summary_v2.py", line 37, in <module>
[2022-12-20T01:19:54.990Z]     from tensorboard.util import tensor_util
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/tensorboard/util/tensor_util.py", line 24, in <module>
[2022-12-20T01:19:54.990Z]     from tensorboard.compat.tensorflow_stub import dtypes, compat, tensor_shape
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/tensorboard/compat/tensorflow_stub/__init__.py", line 25, in <module>
[2022-12-20T01:19:54.990Z]     from .dtypes import as_dtype  # noqa
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py", line 568, in <module>
[2022-12-20T01:19:54.990Z]     (np.object, string),
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/numpy/__init__.py", line 284, in __getattr__
[2022-12-20T01:19:54.990Z]     raise AttributeError("module {!r} has no attribute "
[2022-12-20T01:19:54.990Z] monai.utils.module.OptionalImportError: from torch.utils.tensorboard import SummaryWriter (module 'numpy' has no attribute 'object').
[2022-12-20T01:19:54.990Z] 
[2022-12-20T01:19:54.990Z] For details about installing the optional dependencies, please visit:
[2022-12-20T01:19:54.990Z]     https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies
[2022-12-20T01:19:54.990Z] 
[2022-12-20T01:19:54.990Z] 
[2022-12-20T01:19:54.990Z] ======================================================================
[2022-12-20T01:19:54.990Z] ERROR: test_training (__main__.IntegrationSegmentation3D)
[2022-12-20T01:19:54.990Z] ----------------------------------------------------------------------
[2022-12-20T01:19:54.990Z] Traceback (most recent call last):
[2022-12-20T01:19:54.990Z]   File "tests/test_integration_segmentation_3d.py", line 290, in test_training
[2022-12-20T01:19:54.990Z]     results = self.train_and_infer(i)
[2022-12-20T01:19:54.990Z]   File "tests/test_integration_segmentation_3d.py", line 261, in train_and_infer
[2022-12-20T01:19:54.990Z]     losses, best_metric = run_training_test(self.data_dir, device=self.device, cachedataset=idx, readers=_readers)
[2022-12-20T01:19:54.990Z]   File "tests/test_integration_segmentation_3d.py", line 118, in run_training_test
[2022-12-20T01:19:54.990Z]     writer = SummaryWriter(log_dir=os.path.join(root_dir, "runs"))
[2022-12-20T01:19:54.990Z]   File "/home/jenkins/agent/workspace/Monai-latest-image/monai/utils/module.py", line 415, in __call__
[2022-12-20T01:19:54.990Z]     raise self._exception
[2022-12-20T01:19:54.990Z]   File "/home/jenkins/agent/workspace/Monai-latest-image/monai/utils/module.py", line 367, in optional_import
[2022-12-20T01:19:54.990Z]     pkg = __import__(module)  # top level module
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/torch/utils/tensorboard/__init__.py", line 12, in <module>
[2022-12-20T01:19:54.990Z]     from .writer import FileWriter, SummaryWriter  # noqa: F401
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/torch/utils/tensorboard/writer.py", line 13, in <module>
[2022-12-20T01:19:54.990Z]     from tensorboard.summary.writer.event_file_writer import EventFileWriter
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/tensorboard/summary/__init__.py", line 25, in <module>
[2022-12-20T01:19:54.990Z]     from tensorboard.summary import v1
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/tensorboard/summary/v1.py", line 26, in <module>
[2022-12-20T01:19:54.990Z]     from tensorboard.plugins.histogram import summary as _histogram_summary
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/tensorboard/plugins/histogram/summary.py", line 38, in <module>
[2022-12-20T01:19:54.990Z]     from tensorboard.plugins.histogram import summary_v2
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/tensorboard/plugins/histogram/summary_v2.py", line 37, in <module>
[2022-12-20T01:19:54.990Z]     from tensorboard.util import tensor_util
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/tensorboard/util/tensor_util.py", line 24, in <module>
[2022-12-20T01:19:54.990Z]     from tensorboard.compat.tensorflow_stub import dtypes, compat, tensor_shape
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/tensorboard/compat/tensorflow_stub/__init__.py", line 25, in <module>
[2022-12-20T01:19:54.990Z]     from .dtypes import as_dtype  # noqa
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py", line 568, in <module>
[2022-12-20T01:19:54.990Z]     (np.object, string),
[2022-12-20T01:19:54.990Z]   File "/opt/conda/lib/python3.8/site-packages/numpy/__init__.py", line 284, in __getattr__
[2022-12-20T01:19:54.990Z]     raise AttributeError("module {!r} has no attribute "
[2022-12-20T01:19:54.990Z] monai.utils.module.OptionalImportError: from torch.utils.tensorboard import SummaryWriter (module 'numpy' has no attribute 'object').
[2022-12-20T01:19:54.990Z] 
[2022-12-20T01:19:54.990Z] For details about installing the optional dependencies, please visit:
[2022-12-20T01:19:54.990Z]     https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies
[2022-12-20T01:19:54.990Z] 
[2022-12-20T01:19:54.990Z] ----------------------------------------------------------------------
[2022-12-20T01:19:54.990Z] Ran 2 tests in 32.653s
[2022-12-20T01:19:54.990Z] 
[2022-12-20T01:19:54.990Z] FAILED (errors=2)
script returned exit code 1
@wyli wyli self-assigned this Dec 20, 2022
@wyli wyli mentioned this issue Dec 20, 2022
Merged
7 tasks
@wyli
Copy link
Contributor Author

wyli commented Dec 20, 2022

for issues in the user environment, a workaround would be pip install -U tensorboard.

@wyli wyli closed this as completed Dec 20, 2022
wyli added a commit that referenced this issue Dec 20, 2022
@wyli
5776 Fixes integration (#5778)
97918e4 
Fixes #5776 


### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Non-breaking change (fix or new feature that would not break
existing functionality).
- [ ] Breaking change (fix or new feature that would cause existing
functionality to change).
- [ ] New tests added to cover the changes.
- [ ] Integration tests passed locally by running `./runtests.sh -f -u
--net --coverage`.
- [ ] Quick tests passed locally by running `./runtests.sh --quick
--unittests --disttests`.
- [ ] In-line docstrings updated.
- [ ] Documentation updated, tested `make html` command in the `docs/`
folder.

Signed-off-by: Wenqi Li <[email protected]>
@wyli
Copy link
Contributor Author

wyli commented Dec 20, 2022

this error only occurs when the tensorboard version is old (we detected it in pytorch 21.02 from a year ago). our docker image releases are all good @Nic-Ma

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wyli wyli

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

5776 Fixes integration wyli/MONAI
1 participant
@wyli