Enabled global get_sile specifiers based on env-vars

Now one can do SISL_IO_DEFAULT=contains=siesta
to globally override the specifiers.

This clarifies the usage when dealing with output files
of various codes as one can pre-specify which code
one is dealing with.

For instance only working in siesta one could do:

export SISL_IO_DEFAULT=siesta

and run your code without modifications.

This commit also introduces a context manager for
the environment variables.

with sisl_environ(SISL_IO_DEFAULT=...):
   ...

will temporarily change the global env-var.
The env-vars can not be changed through os.environ.

This fixes #576.

Signed-off-by: Nick Papior <[email protected]>

CHANGELOG.md

@@ -8,6 +8,20 @@ we hit release version 1.0.0.
 ## [X.Y.Z] - YYYY-MM-DD
 
 ### Added
+- allowed sile specifiers to be more explicit:
+     - "hello.xyz{contains=<name>}" equivalent to "hello.xyz{<name>}"
+     - "hello.xyz{startswith=<name>}" class name should start with `<name>`
+     - "hello.xyz{endswith=<name>}" class name should end with `<name>`
+        This is useful for defining a currently working code:
+
+            SISL_IO_DEFAULT=siesta
+
+- added environment variable ``SISL_IO_DEFAULT`` which appends a sile specifier
+  if not explicitly added. I.e. ``get_sile("hello.xyz")`` is equivalent to
+  ``get_sile("hello.xyz{os.environ["SISL_IO_DEFAULT"]}"``.
+  Fixes #576
+- added a context manager for manipulating the global env-vars in temporary
+  locations. ``with sisl_environ(SISL_IO_DEFAULT=...)``
 - enabled `Geometry.append|prepend` in `sgeom` command (reads other files)
 - added `fdfSileSiesta.write_brillouinzone` to easily write BandLines to the fdf output,
   see #141
@@ -25,40 +39,41 @@ we hit release version 1.0.0.
 - `BrillouinZone.merge` allows simple merging of several objects, #537
 
 ### Changed
+- bumped Python requirement to >=3.8
 - Added printout of the removed couplings in the `RecursiveSI`
-- SuperCell class is officially deprecated in favor of Lattice, see #95 for details
+- `SuperCell` class is officially deprecated in favor of `Lattice`, see #95 for details
   The old class will still be accessible and usable for some time (at least a year)
 - Enabled EigenState.wavefunction(grid) to accept grid as the initialization of
-	the grid argument, so one does not need to produce the Grid on before-hand
-- Geometry.rotate(only=) to (what=), this is to unify the interfaces across, #541
+	the grid argument, so one does not need to produce the `Grid` on before-hand
+- ``Geometry.rotate(only=)`` to ``(what=)``, this is to unify the interfaces across, #541
   Also changed the default value to be "xyz" if atoms is Not none
-- tbtncSileTBtrans(only=) arguments are changed to (what=) #541
+- ``tbtncSileTBtrans(only=)`` arguments are changed to (what=) #541
 - `SelfEnergy.scattering_matrix` is changed to `SelfEnergy.broadening_matrix`
   ince the scattering matrix is an S-matrix usage.
   Also changed `se2scat` to `se2broadening` #529
 - allow `BrillouinZone` initialization with scalar weights for all k-points #537
-- Geometry.swapaxes and SuperCell.swapaxes, these are now more versatile by
+- `Geometry.swapaxes` and `SuperCell.swapaxes`, these are now more versatile by
 	allowing multiple swaps in a single run, #539
 - deprecated `set_sc`
 - internal build-system is changed to `scikit-build-core`, the `distutils` will be
   deprecated in Python>=3.12 so it was a needed change.
-  This resulted in a directory restructuring
+  This resulted in a directory restructuring.
 
 
 ### Fixed
 - rare cases for non-Gamma calculations with actual Gamma matrices resulted
   in crashes #572
-- MonkhorstPack.replace now checks for symmetry k-points if the BZ is using
+- `MonkhorstPack.replace` now checks for symmetry k-points if the BZ is using
   trs. Additionally the displacements are moved to the primitive point before
   comparing, this partly fixed #568
 - spin-orbit Hamiltonians in `RealSpaceSE` and `RealSpaceSI`, fixes #567
 - ufunc reductions on `SparseGeometry` where `axis` arguments reduces
   dimensionality
 - interaction with pymatgen
-- fdfSileSiesta.includes would fail when empty lines were present, #555
+- `fdfSileSiesta.includes` would fail when empty lines were present, #555
   fixed and added test
 - Documentation now uses global references
-- Geometry.swapaxes would not swap latticevector cartesian coordinates, #539
+- `Geometry.swapaxes` would not swap latticevector cartesian coordinates, #539
 
 
 ### toolbox.btd

docs/api/io/index.rst

@@ -14,6 +14,31 @@ of DFT software and other post-processing tools.
 In `sisl` all files are conventionally named *siles*
 to distinguish them from files from other packages.
 
+All files are generally accessed through the `get_sile` method
+which exposes all siles through their extension recognition.
+
+The current procedure for determining the file type is based on these
+steps:
+
+1. Extract the extension of the filename passed to `get_sile`, for instance
+   ``hello.grid.nc`` will both examine extensions of ``grid.nc`` and ``nc``.
+   This is necessary for leveled extensions.
+2. Determine whether there is some specification in the file name ``hello.grid.nc{<specification>}``
+   where ``<specification>`` can be:
+
+   - ``contains=<name>`` (or simply ``<name>``) the class name must contain ``<name>``
+   - ``endswith=<name>`` the class name must end with ``<name>``
+   - ``startswith=<name>`` the class name must start with ``<name>``
+
+   When a specification is used, only siles that obey that specification will be searched
+   for the extension.
+   This may be particularly useful when there are multiple codes using the same extension.
+   For instance output files exists in several of the code bases.
+3. Search all indexed (through `add_sile`) siles which obey the specification (all if no specifier)
+   and collect all that matches the longest extension found.
+4. If there is only 1 match, then return that class. If there are multiple `sisl` will try all ``read_*``
+   methods and if all fail, then the sile will be removed from the eligible list.
+
 
 .. toctree::
    :maxdepth: 2

src/sisl/_environ.py

@@ -2,16 +2,38 @@
 # License, v. 2.0. If a copy of the MPL was not distributed with this
 # file, You can obtain one at https://mozilla.org/MPL/2.0/.
 import os
+from contextlib import contextmanager
 from pathlib import Path
 from textwrap import dedent
 
-__all__ = ["register_environ_variable", "get_environ_variable"]
+
+__all__ = ["register_environ_variable", "get_environ_variable", "sisl_environ"]
+
 
 # Local variable for retaining the variables, may be used for
 # extroversion
 SISL_ENVIRON = {}
 
 
+@contextmanager
+def sisl_environ(**environ):
+    r""" Create a new context for temporary overwriting the sisl environment variables
+
+    Parameters
+    ----------
+    environ : dict, optional
+        the temporary environment variables that should be used in this context
+    """
+    global SISL_ENVIRON
+    old = {}
+    for key, value in environ.items():
+        old[key] = SISL_ENVIRON[key]["value"]
+        SISL_ENVIRON[key]["value"] = value
+    yield # nothing to yield
+    for key in environ:
+        SISL_ENVIRON[key]["value"] = old[key]
+
+
 def register_environ_variable(name, default,
                               description=None,
                               process=None):
@@ -50,6 +72,7 @@ def process(arg):
         "default": default,
         "description": description,
         "process": process,
+        "value": os.environ.get(name, default),
     }
 
 
@@ -62,9 +85,7 @@ def get_environ_variable(name):
         the name of the environment variable.
     """
     variable = SISL_ENVIRON[name]
-    value = variable["process"](os.environ.get(name, variable["default"]))
-    variable["value"] = value
-    return value
+    return variable["process"](variable["value"])
 
 
 # We register a few variables that may be used several places
@@ -103,3 +124,7 @@ def get_environ_variable(name):
 register_environ_variable("SISL_SHOW_PROGRESS", "false",
                           "Whether routines which can enable progress bars should show them by default or not.",
                           process=lambda val: val and val.lower().strip() in ["1", "t", "true"])
+
+register_environ_variable("SISL_IO_DEFAULT", "",
+                          "The default DFT code for processing files, Siles will be compared with endswidth(<>).",
+                          process=lambda val: val.lower())

src/sisl/conftest.py

@@ -215,6 +215,7 @@ def pytest_configure(config):
                  "collection",
                  "category", "geom_category", "plot",
                  'slow', 'selector', 'overlap', 'mixing',
+                 "typing", "only",
                  'viz', 'plotly', 'blender']:
         config.addinivalue_line(
             "markers", f"{mark}: mark test to run only on named environment"

src/sisl/io/__init__.py

@@ -14,6 +14,18 @@
 to distinguish them from files from other packages.
 
 
+General file retrieval is done through the file extensions and through
+specifications at read time.
+For instance
+
+    >>> get_sile("hello.xyz")
+
+will automatically recognize the `xyzSile`. If one wishes to explicitly
+specify the filetype one can denote using a specifier:
+
+    >>> get_sile("this_file_is_in_xyz_format.dat{xyz}")
+
+
 Basic IO methods/classes
 ========================
 
@@ -39,6 +51,10 @@
 
    asciiSileBigDFT - the input for BigDFT
 
+FHIaims
+=======
+
+   inSileFHIaims - input file for FHIaims
 
 GULP
 ====
@@ -56,7 +72,7 @@
 ORCA
 ====
 
-   outputSileORCA - standard output file
+   outSileORCA - standard output file
    txtSileORCA - property.txt file
 
 
@@ -71,35 +87,32 @@
 Siesta
 ======
 
-   fdfSileSiesta - input file
-   outSileSiesta - output file
-   xvSileSiesta - xyz and vxyz file
+   aniSileSiesta - xyz file in a trajectory format
    bandsSileSiesta - band structure information
-   eigSileSiesta - EIG file
-   pdosSileSiesta - PDOS file
-   gridSileSiesta - Grid charge information (binary)
-   gridncSileSiesta - NetCDF grid output files (netcdf)
+   ionxmlSileSiesta - Basis-information from the ion.xml files
+   ionncSileSiesta - Basis-information from the ion.nc files
    onlysSileSiesta - Overlap matrix information
    dmSileSiesta - density matrix information
    hsxSileSiesta - Hamiltonian and overlap matrix information
-   wfsxSileSiesta - wavefunctions
+   gridSileSiesta - Grid charge information (binary)
+   gridncSileSiesta - NetCDF grid output files (netcdf)
    ncSileSiesta - NetCDF output file
-   ionxmlSileSiesta - Basis-information from the ion.xml files
-   ionncSileSiesta - Basis-information from the ion.nc files
-   orbindxSileSiesta - Basis set information (no geometry information)
-   faSileSiesta - Forces on atoms
-   fcSileSiesta - Force constant matrix
-   kpSileSiesta - k-points from simulation
-   rkpSileSiesta - k-points to simulation
-
-
-TranSiesta
-==========
-
    tshsSileSiesta - TranSiesta Hamiltonian
    tsdeSileSiesta - TranSiesta (energy) density matrix
    tsgfSileSiesta - TranSiesta surface Green function files
    tsvncSileSiesta - TranSiesta specific Hartree potential file
+   wfsxSileSiesta - wavefunctions
+   eigSileSiesta - EIG file
+   faSileSiesta - Forces on atoms
+   fcSileSiesta - Force constant matrix
+   fdfSileSiesta - input file
+   kpSileSiesta - k-points from simulation
+   rkpSileSiesta - k-points to simulation
+   orbindxSileSiesta - Basis set information (no geometry information)
+   structSileSiesta - structure information
+   outSileSiesta - output file
+   pdosSileSiesta - PDOS file
+   xvSileSiesta - xyz and vxyz file
 
 
 TBtrans

src/sisl/io/gulp/got.py

@@ -8,7 +8,7 @@
 
 from sisl._internal import set_module
 from sisl.messages import info, warn
-from sisl.messages import deprecate_argument, deprecate_method
+from sisl.messages import deprecate_argument, deprecation
 from .sile import SileGULP
 from .fc import fcSileGULP
 from ..sile import add_sile, sile_fh_open
@@ -17,7 +17,7 @@
 from sisl.physics import DynamicalMatrix
 
 
-__all__ = ['gotSileGULP']
+__all__ = ["gotSileGULP"]
 
 
 @set_module("sisl.io.gulp")
@@ -51,7 +51,7 @@ def set_lattice_key(self, key):
         """ Overwrites internal key lookup value for the cell vectors """
         self.set_key('lattice', key)
 
-    set_supercell_key = deprecate_method("set_supercell_key is deprecated in favor of set_lattice_key", "0.15")(set_lattice_key)
+    set_supercell_key = deprecation("set_supercell_key is deprecated in favor of set_lattice_key", "0.15")(set_lattice_key)
 
     @sile_fh_open()
     def read_lattice_nsc(self, key=None):
@@ -298,5 +298,6 @@ def _r_dynamical_matrix_fc(self, geometry, **kwargs):
 
 
 # Old-style GULP output
-add_sile('gout', gotSileGULP, gzip=True)
-add_sile('got', gotSileGULP, gzip=True)
+add_sile("gout", gotSileGULP, gzip=True)
+add_sile("got", gotSileGULP, gzip=True)
+add_sile("out", gotSileGULP, gzip=True)

src/sisl/io/orca/__init__.py

@@ -9,5 +9,5 @@
    txtSileORCA - ORCA property.txt file
 """
 from .sile import *
+from .out import *
 from .txt import *
-from .output import *

src/sisl/io/orca/output.py → src/sisl/io/orca/out.py

@@ -5,15 +5,16 @@
 from .sile import SileORCA
 from ..sile import add_sile, sile_fh_open
 
+from sisl.messages import deprecation
 from sisl.utils import PropertyDict
 from sisl._internal import set_module
 from sisl.unit import units
 
-__all__ = ['outputSileORCA']
+__all__ = ["outputSileORCA", "outSileORCA"]
 
 
 @set_module("sisl.io.orca")
-class outputSileORCA(SileORCA):
+class outSileORCA(SileORCA):
     """ Output file from ORCA """
 
     def _setup(self, *args, **kwargs):
@@ -389,4 +390,8 @@ def readE(itt):
         return None
 
 
-add_sile('output', outputSileORCA, gzip=True)
+outputSileORCA = deprecation("outputSileORCA has been deprecated in favor of outSileOrca.", "0.15")(outSileORCA)
+
+add_sile("output", outSileORCA, gzip=True, case=False)
+add_sile("orca.out", outSileORCA, gzip=True, case=False)
+add_sile("out", outSileORCA, gzip=True, case=False)