Add optional dependency management

[mauicv]

TODO:

• feature branch for tensorflow and pytorch optional dependencies
• feature branch for numba optional dependencies
• feature branch for prophet optional dependencies
• Image branch (see notion)
• docs branch
• Update Licenses

---

What is this PR?

This PR addresses optional dependency import functionality for Alibi Detect. The core behaviour this adds is to throw errors informing the user if they haven't installed the necessary optional dependency for the functionality they wish to use. These errors are thrown at use time rather than at import.

There are three behaviours in general:

1. Optional dependency in a public object. In this case, we might have an object that can optionally use an optional dependency.
   • In this case, if the object is used in a way that doesn't use this dependency no error should be thrown.
   • If The object is used such that the optional dependency is used then the error should be thrown.
2. Conditionally Optional dependency in an object. As an example, the value backend should be one of 'tensorflow' or 'torch' but not neither.
   • An error should be thrown if the user requests a backend that isn't installed.
3. Objects that are completely dependent on an optional dependency
   • If the user imports and uses these then an error should be thrown

This PR addresses behaviours 1 and 3 above.

---

Note:

In order to manually test alibi-detect in each of the different environments developers can use:

make repl tox-env=<env>

and this will give them a REPL with the <env> optional dependencies installed.

For example: make repl tox-env=torch will give them the torch optional dependency environment REPL.

---

The main changes are:

1. The MissingDependency and optional_import functionality,
2. The refactoring of the codebase
3. The tests.

1. MissingDependency

The MissingDependency metaclass is used to allow imports that have missing dependencies. If these imports are used later they will throw an error.

Implementation

The basic pattern is to replace constructs that require uninstalled dependencies with something like:

class MissingDependency:
    def __init__(self, missing_dependency: str, object_name: str):
        self.object_name = object_name
        self.missing_dependency = missing_dependency

    def __getattr__(self, key):
        raise ImportError()

    def __call__(self, *args, **kwargs):
        raise ImportError()

We also use an import function which should be used throughout by developers when importing constructs dependent on optional dependencies in order to ensure consistent behaviour. This looks roughly like:

def import_optional(module_name: str):
    try:
        return import_module(module_name)
    except (ImportError, ModuleNotFoundError) as err:
        return MissingDependency(
            missing_dependency=err.name, 
            object_name=module_name)

The above is called with:

UsefulClass = import_optional('alibi_detect.utils.useful_class')

The above raises an error if the user attempts to access an attribute or call the object.

1. The error message informs the user that the UsefulClass object is missing optional dependencies.
2. It tells the user how to resolve using pip install alibi-detect[optional-dependency].
3. And finally it links to the original error thrown.

2. Refactoring:

The idea here is wherever the is an object that is dependent on an optional install the relevant functionality should be fenced off into a single file or collection of files. Access to that object should be via the __init__.py file. Within the __init__.py we will implement the MissingDependency pattern mentioned above.

Note on Type checking issue:

• The import_optional function returns an object instance rather than an object. This will cause typechecking to fail
  if not all optional dependencies are installed. Because of this we also need to 1. Conditionally import the true object
  dependent on TYPE_CHECKING and 2. Use forward referencing within typing constructs such as Union. We use forward
  referencing because in a user environment the optional dependency may not be installed in which case it'll be replaced
  with an instance of the MissingDependency class. This will throw an error when passed to Union. See CONTRIBUTING.md note for more details and example.

3. Tests:

The tests import all the named objects from the public API of alibi and test that they throw the correct errors if the relevant optional dependencies are not installed. If these tests fail, it is likely that:

1. The optional dependency relation hasn't been added to the test script. In this case, this test assumes that your
   functionality should work for the default alibi-detect install. If this is not the case you should add the exported object name to the dependency_map in the relevant test.
2. The relevant export in the public API hasn't been protected with the MissingDependency class. In this case, see the docs string for the utils.missing_dependency m1odule.

Notes:

1. The tests will be skipped in the normal test suite. To run correctly use tox. This should be as simple as running tox from the terminal. The environments will take a long time to build the first time but afterwards, this should be a quick process.
2. If you need to configure a new optional dependency you will need to update the setup.cfg file and add a testenv environment as well as to the extra-requires.txt file
3. Backend functionality may be unique to specific explainers/functions and so there may be multiple such modules that need to be tested separately.
4. We assume all public functionality is exposed in modules via the __init__.py files.
5. We assume all imports are top-level, if an import is nested within an object or function call this functionality will avoid being caught by the tests.

---

See also Further Notes

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[ascillitoe]

The other test functions are all named test_torch_... rather than test_pytorch_.... Change for consistency?

[ascillitoe]

Change alibi to alibi_detect? Maybe worth a quick grep to check if any other places where comments have been copied from the alibi implementation?

[ascillitoe]

Commenting here as can't comment on empty file. For alibi_detect/utils/tests/__init__.py, is it worth adding a comment to the file explaining why we need an empty __init__.py? Assuming its for duplicate conftest.py's...

[ascillitoe]

Few final comments. LGTM once resolved!

Before merging we should remove [WIP] from the PR name, and decide on whether to squash before merging. @jklaise @mauicv

[bluepark-sk]

Hi. Do you have any plan to add saving/loading functionality for PyTorch model/backend?
If you have, when will it be developed?

[ascillitoe]

Hi. Do you have any plan to add saving/loading functionality for PyTorch model/backend? If you have, when will it be developed?

Hi @bluepark-sk, we sure do. Time permitting, I will be beginning work on PyTorch save/load this week (for drift detectors only)!

[bluepark-sk]

@ascillitoe Thank you! I'll be waiting~~