Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix demos on other platforms #95

Merged
merged 4 commits into from
Jun 25, 2024
Merged

Fix demos on other platforms #95

merged 4 commits into from
Jun 25, 2024

Conversation

ziw-liu
Copy link
Collaborator

@ziw-liu ziw-liu commented Jun 25, 2024

Fixed the issue of normalization statistics (float from JSON) being auto-casted to torch.float64 so that MPS works on macos_arm64.

Also disabled multiprocessing. Or we can put the whole thing in a if __name__ == '__main__': guard.

@edyoshikun edyoshikun self-requested a review June 25, 2024 01:27
@edyoshikun
Copy link
Contributor

I need to debug further but I got this error

---------------------------------------------------------------------------
MisconfigurationException                 Traceback (most recent call last)
File /Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/examples/demos/demo_vsneuromast.py:3
      [1](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/examples/demos/demo_vsneuromast.py:1) # %%
      [2](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/examples/demos/demo_vsneuromast.py:2) # Setup the Trainer
----> [3](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/examples/demos/demo_vsneuromast.py:3) trainer = VSTrainer(
      [4](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/examples/demos/demo_vsneuromast.py:4)     accelerator="gpu",
      [5](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/examples/demos/demo_vsneuromast.py:5)     callbacks=[HCSPredictionWriter(output_path)],
      [6](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/examples/demos/demo_vsneuromast.py:6) )
      [8](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/examples/demos/demo_vsneuromast.py:8) # Start the predictions
      [9](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/examples/demos/demo_vsneuromast.py:9) trainer.predict(
     [10](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/examples/demos/demo_vsneuromast.py:10)     model=model_VSNeuromast,
     [11](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/examples/demos/demo_vsneuromast.py:11)     datamodule=data_module,
     [12](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/examples/demos/demo_vsneuromast.py:12)     return_predictions=False,
     [13](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/examples/demos/demo_vsneuromast.py:13) )

File ~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/utilities/argparse.py:70, in _defaults_from_env_vars.<locals>.insert_env_defaults(self, *args, **kwargs)
     [67](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/utilities/argparse.py:67) kwargs = dict(list(env_variables.items()) + list(kwargs.items()))
     [69](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/utilities/argparse.py:69) # all args were already moved to kwargs
---> [70](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/utilities/argparse.py:70) return fn(self, **kwargs)

File ~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:400, in Trainer.__init__(self, accelerator, strategy, devices, num_nodes, precision, logger, callbacks, fast_dev_run, max_epochs, min_epochs, max_steps, min_steps, max_time, limit_train_batches, limit_val_batches, limit_test_batches, limit_predict_batches, overfit_batches, val_check_interval, check_val_every_n_epoch, num_sanity_val_steps, log_every_n_steps, enable_checkpointing, enable_progress_bar, enable_model_summary, accumulate_grad_batches, gradient_clip_val, gradient_clip_algorithm, deterministic, benchmark, inference_mode, use_distributed_sampler, profiler, detect_anomaly, barebones, plugins, sync_batchnorm, reload_dataloaders_every_n_epochs, default_root_dir)
    [397](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:397) # init connectors
    [398](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:398) self._data_connector = _DataConnector(self)
--> [400](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:400) self._accelerator_connector = _AcceleratorConnector(
    [401](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:401)     devices=devices,
    [402](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:402)     accelerator=accelerator,
    [403](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:403)     strategy=strategy,
    [404](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:404)     num_nodes=num_nodes,
    [405](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:405)     sync_batchnorm=sync_batchnorm,
    [406](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:406)     benchmark=benchmark,
    [407](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:407)     use_distributed_sampler=use_distributed_sampler,
    [408](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:408)     deterministic=deterministic,
    [409](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:409)     precision=precision,
    [410](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:410)     plugins=plugins,
    [411](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:411) )
    [412](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:412) self._logger_connector = _LoggerConnector(self)
    [413](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:413) self._callback_connector = _CallbackConnector(self)

File ~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/connectors/accelerator_connector.py:143, in _AcceleratorConnector.__init__(self, devices, num_nodes, accelerator, strategy, plugins, precision, sync_batchnorm, benchmark, use_distributed_sampler, deterministic)
    [141](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/connectors/accelerator_connector.py:141)     self._accelerator_flag = self._choose_auto_accelerator()
    [142](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/connectors/accelerator_connector.py:142) elif self._accelerator_flag == "gpu":
--> [143](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/connectors/accelerator_connector.py:143)     self._accelerator_flag = self._choose_gpu_accelerator_backend()
    [145](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/connectors/accelerator_connector.py:145) self._check_device_config_and_set_final_flags(devices=devices, num_nodes=num_nodes)
    [146](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/connectors/accelerator_connector.py:146) self._set_parallel_devices_and_init_accelerator()

File ~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/connectors/accelerator_connector.py:353, in _AcceleratorConnector._choose_gpu_accelerator_backend()
    [351](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/connectors/accelerator_connector.py:351) if CUDAAccelerator.is_available():
    [352](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/connectors/accelerator_connector.py:352)     return "cuda"
--> [353](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/~/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/lightning/pytorch/trainer/connectors/accelerator_connector.py:353) raise MisconfigurationException("No supported gpu backend found!")

MisconfigurationException: No supported gpu backend found!

@edyoshikun
Copy link
Contributor

It looks like a torch issue for me. Putting it here in case you have seen this before.

Package                   Version                          Editable project location
------------------------- -------------------------------- ----------------------------------------------------
absl-py                   2.1.0
aiohttp                   3.9.5
aiosignal                 1.3.1
anyio                     4.4.0
appnope                   0.1.4
argon2-cffi               23.1.0
argon2-cffi-bindings      21.2.0
arrow                     1.3.0
asciitree                 0.3.3
asttokens                 2.4.1
async-lru                 2.0.4
attrs                     23.2.0
Babel                     2.15.0
beautifulsoup4            4.12.3
bleach                    6.1.0
blosc2                    2.7.0
certifi                   2024.6.2
cffi                      1.16.0
charset-normalizer        3.3.2
click                     8.1.7
cloudpickle               3.0.0
comm                      0.2.2
contourpy                 1.2.1
cycler                    0.12.1
dask                      2024.6.2
debugpy                   1.8.1
decorator                 5.1.1
defusedxml                0.7.1
docstring_parser          0.16
exceptiongroup            1.2.0
executing                 2.0.1
fasteners                 0.19
fastjsonschema            2.20.0
filelock                  3.15.4
fonttools                 4.53.0
fqdn                      1.5.1
frozenlist                1.4.1
fsspec                    2024.6.0
grpcio                    1.64.1
h11                       0.14.0
httpcore                  1.0.5
httpx                     0.27.0
huggingface-hub           0.23.4
idna                      3.7
imageio                   2.34.2
importlib_metadata        7.2.1
importlib_resources       6.4.0
iohub                     0.1.0
ipykernel                 6.29.4
ipython                   8.25.0
ipywidgets                8.1.3
isoduration               20.11.0
jedi                      0.19.1
Jinja2                    3.1.4
json5                     0.9.25
jsonargparse              4.30.0
jsonpointer               3.0.0
jsonschema                4.22.0
jsonschema-specifications 2023.12.1
jupyter                   1.0.0
jupyter_client            8.6.2
jupyter-console           6.6.3
jupyter_core              5.7.2
jupyter-events            0.10.0
jupyter-lsp               2.2.5
jupyter_server            2.14.1
jupyter_server_terminals  0.5.3
jupyterlab                4.2.2
jupyterlab_pygments       0.3.0
jupyterlab_server         2.27.2
jupyterlab_widgets        3.0.11
kiwisolver                1.4.5
lazy_loader               0.4
lightning                 2.3.0
lightning-utilities       0.11.2
locket                    1.0.0
Markdown                  3.6
MarkupSafe                2.1.5
matplotlib                3.9.0
matplotlib-inline         0.1.7
mistune                   3.0.2
monai                     1.3.1
mpmath                    1.3.0
msgpack                   1.0.8
multidict                 6.0.5
natsort                   8.4.0
nbclient                  0.10.0
nbconvert                 7.16.4
nbformat                  5.10.4
ndindex                   1.8
ndtiff                    3.1.0
nest_asyncio              1.6.0
networkx                  3.3
notebook                  7.2.1
notebook_shim             0.2.4
numcodecs                 0.12.1
numexpr                   2.10.1
numpy                     2.0.0
overrides                 7.7.0
packaging                 24.1
pandas                    2.2.2
pandocfilters             1.5.1
parso                     0.8.4
partd                     1.4.2
pexpect                   4.9.0
pickleshare               0.7.5
pillow                    10.3.0
pip                       24.0
platformdirs              4.2.2
prometheus_client         0.20.0
prompt_toolkit            3.0.47
protobuf                  4.25.3
psutil                    5.9.8
ptyprocess                0.7.0
pure-eval                 0.2.2
py-cpuinfo                9.0.0
pybind11                  2.12.0
pycparser                 2.22
pydantic                  1.10.17
Pygments                  2.18.0
pyparsing                 3.1.2
python-dateutil           2.9.0
python-json-logger        2.0.7
pytorch-lightning         2.3.0
pytz                      2024.1
PyYAML                    6.0.1
pyzmq                     26.0.3
qtconsole                 5.5.2
QtPy                      2.4.1
referencing               0.35.1
requests                  2.32.3
rfc3339-validator         0.1.4
rfc3986-validator         0.1.1
rpds-py                   0.18.1
safetensors               0.4.3
scikit-image              0.24.0
scipy                     1.14.0
Send2Trash                1.8.3
setuptools                69.5.1
six                       1.16.0
sniffio                   1.3.1
sortedcontainers          2.4.0
soupsieve                 2.5
stack-data                0.6.2
sympy                     1.12.1
tensorboard               2.17.0
tensorboard-data-server   0.7.2
terminado                 0.18.1
tifffile                  2023.2.28
timm                      1.0.7
tinycss2                  1.3.0
toolz                     0.12.1
torch                     2.2.2
torchmetrics              1.4.0.post0
torchvision               0.17.2
tornado                   6.4.1
tqdm                      4.66.4
traitlets                 5.14.3
types-python-dateutil     2.9.0.20240316
typeshed_client           2.5.1
typing_extensions         4.12.2
tzdata                    2024.1
uri-template              1.3.0
urllib3                   2.2.2
viscy                     0.1.0rc1.dev4+gb93881b.d20240625 /Users/eduardo.hirata-miyasaki/Documents/repos/VisCy
wcwidth                   0.2.13
webcolors                 24.6.0
webencodings              0.5.1
websocket-client          1.8.0
Werkzeug                  3.0.3
wheel                     0.43.0
widgetsnbextension        4.0.11
yarl                      1.9.4
zarr                      2.15.0
zipp                      3.19.2

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/ipykernel_launcher.py", line 18, in <module>
    app.launch_new_instance()
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/traitlets/config/application.py", line 1075, in launch_instance
    app.start()
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/ipykernel/kernelapp.py", line 739, in start
    self.io_loop.start()
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/tornado/platform/asyncio.py", line 205, in start
    self.asyncio_loop.run_forever()
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/asyncio/base_events.py", line 608, in run_forever
    self._run_once()
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/asyncio/base_events.py", line 1936, in _run_once
    handle._run()
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/asyncio/events.py", line 84, in _run
    self._context.run(self._callback, *self._args)
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/ipykernel/kernelbase.py", line 545, in dispatch_queue
    await self.process_one()
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/ipykernel/kernelbase.py", line 534, in process_one
    await dispatch(*args)
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/ipykernel/kernelbase.py", line 437, in dispatch_shell
    await result
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/ipykernel/ipkernel.py", line 362, in execute_request
    await super().execute_request(stream, ident, parent)
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/ipykernel/kernelbase.py", line 778, in execute_request
    reply_content = await reply_content
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/ipykernel/ipkernel.py", line 449, in do_execute
    res = shell.run_cell(
  File "/var/folders/2k/183m7pjs6cnd3ng7t92jn62c0000gq/T/ipykernel_27115/2675205427.py", line 29, in wrapper
    result = old_func(*args, **kwargs)
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/ipykernel/zmqshell.py", line 549, in run_cell
    return super().run_cell(*args, **kwargs)
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3075, in run_cell
    result = self._run_cell(
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3130, in _run_cell
    result = runner(coro)
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner
    coro.send(None)
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3334, in run_cell_async
    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3517, in run_ast_nodes
    if await self.run_code(code, result, async_=asy):
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3577, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-1-68b8a4a1d0db>", line 7, in <module>
    from viscy.data.hcs import HCSDataModule
  File "/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/viscy/data/hcs.py", line 11, in <module>
    import torch
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/torch/__init__.py", line 1477, in <module>
    from .functional import *  # noqa: F403
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/torch/functional.py", line 9, in <module>
    import torch.nn.functional as F
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/torch/nn/__init__.py", line 1, in <module>
    from .modules import *  # noqa: F403
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/torch/nn/modules/__init__.py", line 35, in <module>
    from .transformer import TransformerEncoder, TransformerDecoder, \
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/torch/nn/modules/transformer.py", line 20, in <module>
    device: torch.device = torch.device(torch._C._get_default_device()),  # torch.device('cpu'),
[/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/torch/nn/modules/transformer.py:20](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/torch/nn/modules/transformer.py:20): UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at [/Users/runner/work/pytorch/pytorch/pytorch/torch/csrc/utils/tensor_numpy.cpp:84](https://file+.vscode-resource.vscode-cdn.net/Users/runner/work/pytorch/pytorch/pytorch/torch/csrc/utils/tensor_numpy.cpp:84).)
  device: torch.device = torch.device(torch._C._get_default_device()),  # torch.device('cpu'),A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/ipykernel_launcher.py", line 18, in <module>
    app.launch_new_instance()
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/traitlets/config/application.py", line 1075, in launch_instance
    app.start()
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/ipykernel/kernelapp.py", line 739, in start
    self.io_loop.start()
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/tornado/platform/asyncio.py", line 205, in start
    self.asyncio_loop.run_forever()
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/asyncio/base_events.py", line 608, in run_forever
    self._run_once()
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/asyncio/base_events.py", line 1936, in _run_once
    handle._run()
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/asyncio/events.py", line 84, in _run
    self._context.run(self._callback, *self._args)
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/ipykernel/kernelbase.py", line 545, in dispatch_queue
    await self.process_one()
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/ipykernel/kernelbase.py", line 534, in process_one
    await dispatch(*args)
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/ipykernel/kernelbase.py", line 437, in dispatch_shell
    await result
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/ipykernel/ipkernel.py", line 362, in execute_request
    await super().execute_request(stream, ident, parent)
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/ipykernel/kernelbase.py", line 778, in execute_request
    reply_content = await reply_content
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/ipykernel/ipkernel.py", line 449, in do_execute
    res = shell.run_cell(
  File "/var/folders/2k/183m7pjs6cnd3ng7t92jn62c0000gq/T/ipykernel_27115/2675205427.py", line 29, in wrapper
    result = old_func(*args, **kwargs)
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/ipykernel/zmqshell.py", line 549, in run_cell
    return super().run_cell(*args, **kwargs)
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3075, in run_cell
    result = self._run_cell(
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3130, in _run_cell
    result = runner(coro)
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner
    coro.send(None)
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3334, in run_cell_async
    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3517, in run_ast_nodes
    if await self.run_code(code, result, async_=asy):
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3577, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-1-68b8a4a1d0db>", line 7, in <module>
    from viscy.data.hcs import HCSDataModule
  File "/Users/eduardo.hirata-miyasaki/Documents/repos/VisCy/viscy/data/hcs.py", line 11, in <module>
    import torch
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/torch/__init__.py", line 1477, in <module>
    from .functional import *  # noqa: F403
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/torch/functional.py", line 9, in <module>
    import torch.nn.functional as F
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/torch/nn/__init__.py", line 1, in <module>
    from .modules import *  # noqa: F403
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/torch/nn/modules/__init__.py", line 35, in <module>
    from .transformer import TransformerEncoder, TransformerDecoder, \
  File "/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/torch/nn/modules/transformer.py", line 20, in <module>
    device: torch.device = torch.device(torch._C._get_default_device()),  # torch.device('cpu'),
[/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/torch/nn/modules/transformer.py:20](https://file+.vscode-resource.vscode-cdn.net/Users/eduardo.hirata-miyasaki/opt/anaconda3/envs/viscy/lib/python3.11/site-packages/torch/nn/modules/transformer.py:20): UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at [/Users/runner/work/pytorch/pytorch/pytorch/torch/csrc/utils/tensor_numpy.cpp:84](https://file+.vscode-resource.vscode-cdn.net/Users/runner/work/pytorch/pytorch/pytorch/torch/csrc/utils/tensor_numpy.cpp:84).)
  device: torch.device = torch.device(torch._C._get_default_device()),  # torch.device('cpu'),

@ziw-liu
Copy link
Collaborator Author

ziw-liu commented Jun 25, 2024

@edyoshikun looks like you're using a torch build that is not compatible with numpy v2. Probably downgrade numpy. Also check if your python is native arm64.

@ziw-liu
Copy link
Collaborator Author

ziw-liu commented Jun 25, 2024

Actually maybe upgrade torch to 2.3: pytorch/pytorch#107302

edyoshikun
edyoshikun approved these changes Jun 25, 2024
View reviewed changes
Copy link
Contributor

@edyoshikun edyoshikun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took a bit of fuzzing around on my side (my conda version was old...) so updating conda did the trick.

This worked well for the VSNeuromast and VSCyto3D on M1.
Thank you @ziw-liu

@ziw-liu ziw-liu added bug Something isn't working documentation Improvements or additions to documentation labels Jun 25, 2024
@ziw-liu ziw-liu merged commit 2747e83 into main Jun 25, 2024
4 checks passed
@ziw-liu ziw-liu deleted the macos-demos-fix branch June 25, 2024 05:09
edyoshikun pushed a commit that referenced this pull request Jun 29, 2024
@ziw-liu @edyoshikun
Fix demos on other platforms (#95)
09bbf21 
* load statistics explicitly to avoid autocasting to float64
this allows the GPU transfer on MPS
see Lightning-AI/pytorch-lightning#16213

* disable multiprocssing by default to avoid fork/spawn difference

* remove unused patch size argument

* black
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@edyoshikun edyoshikun edyoshikun approved these changes

Assignees
No one assigned
Labels
bug Something isn't working documentation Improvements or additions to documentation
Projects
None yet
Milestone
v0.2.0 release - DL@MBL exercise
Development

Successfully merging this pull request may close these issues.
2 participants
@ziw-liu @edyoshikun