coremltools 5.0b1

To install this version run: pip install coremltools==5.0b1

Whats New

• Added a new kind of Core ML model type, called ML Program. TensorFlow and Pytorch models can now be converted to ML Programs.
  • To learn about ML Programs, how they are different from the classicial Core ML neural network types, and what they offer, please see the documentation here
  • Use the convert_to argument with the unified converter API to indicate the model type of the Core ML model.
    • coremltools.convert(..., convert_to=“mlprogram”) converts to a Core ML model of type ML program.
    • coremltools.convert(..., convert_to=“neuralnetwork”) converts to a Core ML model of type neural network. “Neural network” is the older Core ML format and continues to be supported. Using just coremltools.convert(...) will default to produce a neural network Core ML model.
  • When targeting ML program, there is an additional option available to set the compute precision of the Core ML model to either float 32 or float16. That is,
    • ct.convert(..., convert_to=“mlprogram”, compute_precision=ct.precision.FLOAT32) or ct.convert(..., convert_to=“mlprogram”, compute_precision=ct.precision.FLOAT16)
    • To know more about how this affects the runtime, see the documentation on Typed execution.
• You can save to the new Model Package format through the usual coremltool’s save method. Simply use model.save("<model_name>.mlpackage") instead of the usual model.save(<"model_name>.mlmodel")
  • Core ML is introducing a new model format called model packages. It’s a container that stores each of a model’s components in its own file, separating out its architecture, weights, and metadata. By separating these components, model packages allow you to easily edit metadata and track changes with source control. They also compile more efficiently, and provide more flexibility for tools which read and write models.
  • ML Programs can only be saved in the model package format.
• Several performance improvements by adding new graph passes in the conversion pipeline for deep learning models, including “fuse_gelu”, “replace_stack_reshape”, “concat_to_pixel_shuffle”, “fuse_layernorm_or_instancenorm” etc
• New Translation methods for Torch ops such as “einsum”, “GRU”, “zeros_like” etc
• OS versions supported by coremltools 5.0b1: macOS10.15 and above, Linux with C++17 and above

Deprecations and Removals

• Caffe converter has been removed. If you are still using the Caffe converter, please use coremltools 4.
• Keras.io and ONNX converters will be deprecated in coremltools 6. Users are recommended to transition to the TensorFlow/PyTorch conversion via the unified converter API.
• Methods, such as convert_neural_network_weights_to_fp16(), convert_neural_network_spec_weights_to_fp16() , that had been deprecated in coremltools 4, have been removed.

Known Issues

• The default compute precision for conversion to ML Programs is set to precision.FLOAT32, although it will be updated to precision.FLOAT16 in a later beta release, prior to the official coremltools 5.0 release.
• Core ML may downcast float32 tensors specified in ML Program model types when running on a device with Neural Engine support. Workaround: Restrict compute units to .cpuAndGPU in MLModelConfiguration for seed 1
• Converting some models to ML Program may lead to an error (such as a segmentation fault or “Error in building plan”), due to a bug in the Core ML GPU runtime. Workaround: When using coremltools, you can force the prediction to stay on the CPU, without changing the prediction code, by specifying the useCPUOnly argument during conversion. That is, ct.convert(source_model, convert_to='mlprogram', useCPUOnly=True). And for such models, in your swift code you can use the MLComputeUnits.cpuOnly option at the time of loading the model, to restrict the compute unit to CPU.
• Flexible input shapes, for image inputs have a bug when using with the ML Program type, in seed 1 of Core ML framework. This will be fixed in an upcoming seed release.
• coremltools 5.0b1 supports python versions 3.5, 3.6, 3.7, 3.8. Support for python 3.9 will be enabled in a future beta release.