Dataprep doet not work on debian:buster

[abij]

The Dataprep uses the dotnetcore2 engine. When it start for the first time it downloads stuff from from https://azuremldownloads.azureedge.net/dotnetcore2-dependencies/

I noticed that Debian 10 (released 6 July 2019) is not available:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.7/dist-packages/dotnetcore2/runtime.py", line 181, in ensure_dependencies
    if not attemp_get_deps():
  File "/usr/local/lib/python3.7/dist-packages/dotnetcore2/runtime.py", line 175, in attemp_get_deps
    raise NotImplementedError('Unsupported Linux distribution {0} {1}.{2}'.format(dist, version[0], version[1]))
NotImplementedError: Unsupported Linux distribution debian 10.

Reproduce

# Start linux buster
docker run -it --rm debian:buster-slim

# Install python and pip
apt-get update && apt-get install python3-pip wget curl --no-install-recommends

# --- FIX start: pre-install dotnet-runtime-2.1
wget https://packages.microsoft.com/config/debian/10/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
dpkg -i packages-microsoft-prod.deb
apt-get update && apt-get install -y --no-install-recommends dotnet-runtime-2.1
# --- FIX end

# Install package:
pip3 install dotnetcore2

# Test by running: runtime.ensure_dependencies()
python3 -c 'import logging as l; l.basicConfig(level=l.DEBUG); from dotnetcore2 import runtime; runtime.ensure_dependencies()'

If you replace buster (v10) with stretch (v9) it works, this is the base image from conda/miniconda3 used in the MLOps image.

Can you also include debian:buster, of at least give a matrix of supported versions in the dotnetcore2 pipy repo?

update:
I have found the dotnet-linux-matrix

update 2:
Added wget and curl
Pre install dotnet-runtime-2.1 works!
Enable debug python logging.

[epa095]

Related:
Azure/MachineLearningNotebooks#950
Azure/MachineLearningNotebooks#713
Azure/MachineLearningNotebooks#1003

A ugly workaround worked for me. But MS stated that it "should work" if you install .NET Core 2.1 manually first with whatever means fits your distro.

[abij]

I tried to pre-install dotnetcore 2.1 as mentioned in the MS docs. I can confirm this works, but you do need to install curl as an extra dependency. This should be a transient dependency in my opinion.

# Pre install dotnetcode-2.1 on Debian-10 (note: requires wget)
wget https://packages.microsoft.com/config/debian/10/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
dpkg -i packages-microsoft-prod.deb

# note also curl is required!
apt-get update && apt-get install -y --no-install-recommends curl dotnet-runtime-2.1

In the Dataprep package (azureml.dataprep.api.engineapi.engine.launch_engine) the runtime.ensure_dependencies() is always called. This will check the dotnet libs are executable. Preinstalling the dotnetcode2.1 does work.