[Feature Request] Expand BetseFileException exception messages with explanatory advice

[dcliner77]

I may be doing something simple wrong, but when I run betse seed single_cell.yaml, I get this error message:

[betse] Welcome to <<BETSE 1.4.1 | CPython 3.12.7 | Ubuntu 22.04.5>>.
[betse] Loading third-party dependencies...
qt.qpa.plugin: Could not find the Qt platform plugin "wayland" in ""
[betse] Exiting prematurely due to fatal error:

BetseFileException: File "/home/dcliner7/extra_configs/expression_data.yaml" not found or unreadable.

Traceback (most recent call last):
File "/home/dcliner7/anaconda3/envs/betse141/lib/python3.12/site-packages/betse/util/cli/cliabc.py", line 180, in run
profile_callable(
File "<@beartype(betse.util.py.pyprofile.profile_callable) at 0x7f258e159260>", line 131, in profile_callable
File "/home/dcliner7/anaconda3/envs/betse141/lib/python3.12/site-packages/betse/util/py/pyprofile.py", line 195, in profile_callable
return profiler(
^^^^^^^^^
File "/home/dcliner7/anaconda3/envs/betse141/lib/python3.12/site-packages/betse/util/py/pyprofile.py", line 216, in _profile_callable_none
return call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/dcliner7/anaconda3/envs/betse141/lib/python3.12/site-packages/betse/util/cli/clicmdabc.py", line 95, in _do
self._run_subcommand(
File "<@beartype(betse.util.cli.clicmdabc.CLISubcommandableABC._run_subcommand) at 0x7f258e15bec0>", line 69, in _run_subcommand
File "/home/dcliner7/anaconda3/envs/betse141/lib/python3.12/site-packages/betse/util/cli/clicmdabc.py", line 187, in _run_subcommand
return subcommand_method()
^^^^^^^^^^^^^^^^^^^
File "/home/dcliner7/anaconda3/envs/betse141/lib/python3.12/site-packages/betse/cli/climain.py", line 374, in _do_seed
return self._sim_runner.seed()
^^^^^^^^^^^^^^^^
File "<@beartype(betse.cli.climain.BetseCLI._sim_runner) at 0x7f258dfc4900>", line 12, in _sim_runner
File "", line 7, in property_method_cached
File "/home/dcliner7/anaconda3/envs/betse141/lib/python3.12/site-packages/betse/cli/climain.py", line 303, in _sim_runner
p = Parameters.make(conf_filename=self._args.conf_filename)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<@beartype(betse.lib.yaml.abc.yamlfileabc.YamlFileABC.make) at 0x7f258695cfe0>", line 33, in make
File "/home/dcliner7/anaconda3/envs/betse141/lib/python3.12/site-packages/betse/lib/yaml/abc/yamlfileabc.py", line 82, in make
yaml_file_conf.load(conf_filename)
File "<@beartype(betse.science.parameters.Parameters.load) at 0x7f2586845620>", line 12, in load
File "/home/dcliner7/anaconda3/envs/betse141/lib/python3.12/site-packages/betse/science/parameters.py", line 792, in load
self.expression_data = yamls.load(
^^^^^^^^^^^
File "<@beartype(betse.lib.yaml.yamls.load) at 0x7f2586cddee0>", line 54, in load
File "/home/dcliner7/anaconda3/envs/betse141/lib/python3.12/site-packages/betse/lib/yaml/yamls.py", line 66, in load
with iofiles.reading_chars(filename) as yaml_file:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<@beartype(betse.util.io.iofiles.reading_chars) at 0x7f258de43060>", line 50, in reading_chars
File "/home/dcliner7/anaconda3/envs/betse141/lib/python3.12/site-packages/betse/util/io/iofiles.py", line 196, in reading_chars
files.die_unless_file(filename)
File "<@beartype(betse.util.path.files.die_unless_file) at 0x7f258e1a4a40>", line 23, in die_unless_file
File "/home/dcliner7/anaconda3/envs/betse141/lib/python3.12/site-packages/betse/util/path/files.py", line 74, in die_unless_file
raise BetseFileException(

The above exception wrapped the following originating exception:

AttributeError: 'BetseCLI' object has no attribute '_betse_cached__property__sim_runner'. Did you mean: '_betse_cached__property__help_epilog'?

Traceback (most recent call last):
File "", line 5, in property_method_cached

Thank you!

[leycec]

Oh, ho, ho. I'll admit. That's definitely not the most readable error output. The core issue is this:

BetseFileException: File "/home/dcliner7/extra_configs/expression_data.yaml" not found or unreadable.

Let's break this down. The default sample_sim/ directory created by the betse try command contains the default sample_sim/sample_sim.yaml configuration file. This file configures the current simulation being run. So far, so good. This file also configures the gene regulatory network (GRN) simulated by this simulation with a YAML block resembling:

general network:

  implement network: true  # Turn the entire network defined below on (True) or off (False)

  expression data file: extra_configs/expression_data.yaml

The lower-level extra_configs/expression_data.yaml configuration file configures the expression data governing this GRN. So far, so good. Maybe? Hopefully?

Directory Structure: It's a Pain, But It Matters

Given this, the issue here becomes (more or less) clear: your simulation configuration appears to be kinda messed up. Ideally, the directory structure for a BETSE simulation should look something like this:

/
  home/
    dcliner7/
      BETSE/
        sample_sim/
          sample_sim.yaml
          extra_configs/
            expression_data.yaml
            ...

In other words, please try the following:

1. Create a new top-level BETSE/ subdirectory in your home /home/dcliner7/ directory (if you haven't already): e.g.,

   mkdir ~/BETSE  # <-- make a new "BETSE/" subdirectory in your home directory
   cd BETSE       # <-- change the current directory to this new "BETSE/" subdirectory

2. Re-run the betse try command here to create a new default ~/BETSE/sample_sim configuration: e.g.,

   betse try

3. List the contents of the new ~/BETSE/sample_sim/ subdirectory created by the betse try command. Make sure that everything looks "sane": e.g.,

   ls sample_sim

The output should resemble:

.  ..  extra_configs  geo  INITS  RESULTS  SIMS  sample_sim.yaml

Do you see something like that? If so, great!

What BETSE Wants: It's a Pain, But It Matters

You can now modify this simulation however you like and run any BETSE commands you'd like against this simulation. The key takeaway here is:

• Always change the current directory to ~/BETSE first before running any betse commands: e.g.,

  cd ~/BETSE

• Always pass the simulation to be run as the string sample_sim/sample_sim.yaml rather than the string sample_sim.yaml: e.g.,

  betse seed sample_sim/sample_sim.yaml

Get Everything You Want and More with These Five Easy Steps

At this point, you probably want to create a number of different BETSE simulations. Right? In this case, you probably want to create a new single_cell simulation. Thankfully, this is trivial – but there's a catch. You can't simply copy a single file. You have to copy the entire directory structure. BETSE needs all of the files in this directory structure to do its perfidious work.

Let's demonstrate how you might create a new ~/BETSE/single_cell/ simulation from your current default ~/BETSE/sample_sim/ simulation:

• Just recursively copy sample_sim/ to single_cell/: e.g.,

  cd ~/BETSE                    # <-- change the current directory to your "BETSE/" subdirectory.
  cp -R sample_sim single_cell  # <-- recursively create "single_cell/" from "sample_sim/"

• Pass the simulation to be run as the string single_cell/sample_sim.yaml: e.g.,

  betse seed single_cell/sample_sim.yaml

Should work? Maybe? I'm squinting and waving my hands around here. Let us know if that patches things up for your scientific workflow! Excited about what you find out next. Bioelectricity intensifies. 😋

[dcliner77]

Thank you so much! The seed and init works now, but the sim didn't. I think it was because I forgot to change some things in the config file for a single cell sim to work. If I've already run a seed and init, can I rerun the seed and init with the updated config file or do I first need to delete the results of the first seed and init run?

[leycec]

Excellent question! The answer is... nope. You should never need to explicitly delete the results of any seed or init run (i.e., the sample_sim/INITS/, sample_sim/RESULTS/, and sample_sim/SIMS/ subdirectories). Re-running a seed or init with an updated config file should always "do the right thing" and thus be safe.

Of course, it's conceivable that there might be a bug or two here. There shouldn't be. Neither my wife (@pietakio) or I ever hit any issues with this. If you do, please just submit an issue and we'll get you sorted out.

Thanks again for playing along with BETSE. Issues like this are invaluable. In fact, there is actually something BETSE could do here to improve everyone's lives: we should expand the unhelpful exception message "BetseFileException: File "/home/dcliner7/extra_configs/expression_data.yaml" not found or unreadable." to actually explain what's (probably) going on and point the user in the right direction.

Let's edit this issue title. Go, Team BETSE. Go! 🚗 💨