Remove ensemble specific code
#305
Conversation
- Removing `abstractmethod` from `transform` to reflect optional implementation
| @@ -107,6 +67,7 @@ def transform( | |||
| """ | |||
| return transformable | |||
|
|
|||
| @abstractmethod | |||
There was a problem hiding this comment.
This export method of the TritonOperator is now an abstractmethod. Meaning that an operator that inherits from TritonOperator is required to implement the export method.
There was a problem hiding this comment.
Removing the default implemntation here because it wasn't used. Each Triton Operator is expected to export it's own model config that is specific to the backend/model that is being used.
| exportable_node_idx = 0 | ||
| node_id_lookup = {} | ||
| for node in nodes: | ||
| if node.exportable(backend): | ||
| if isinstance(node.op, TritonOperator): |
There was a problem hiding this comment.
We don't need to check the exportable method anymore because every TrtionOperator is expected to implement export to save the operator as a Triton model.
| if node_config is not None: | ||
| node_configs.append(node_config) | ||
|
|
||
| executor_config = self._executor_model_export(path, name, ensemble) | ||
| if hasattr(node.op, "save_artifacts"): | ||
| node.op.save_artifacts(str(artifact_path)) |
There was a problem hiding this comment.
This new method save_artifacts enables any inference operator to save state that can't be pickled, without needing to implement a TritonOperator with a model export. Complements the existing load_artifacts method that is called within the executor_model.py
Documentation preview |
Following the removal of the ensemble runtime export in #255. Further cleanup of methods that are no longer required.
Motivation
Details
One new feature is added by this PR. A
save_artifactsmethod that complements theload_artifactsmethod. And facilitates the removal of theexportmethods on all non-Triton operators.OperatorRunner(and corresponding Triton Python Modeloprunner_model.py)TritonExecutorRuntime(and corresponding Triton Python Modelexecutor_model.py)from_configmethods from non-Triton specific operators (this method was called byoprunner_model.pyto reload an operator from the Triton model config). This was used when we were exporting each operator as a triton model)exportmethods from non-Triton specific operators.exportable_backendsfrom operatorsTritonOperator.exportto save non-pickleable objects for reloading. Added a new method calledsave_artifactsthat serves this purpose and is intended to be implemented on any operator that implementsload_artifacts.PredictImplicitTriton. The regularPredictImplicitis sufficient now that we havesave_artifactsimplemented.