Exception: Unable to discover a target process

[steinarox]

Description

Using this example https://github.com/dotnet/dotnet-monitor/blob/main/documentation/kubernetes.md i tried adding dotnetmonitor to one of our dotnet 3.1 services but getting this error. I also tried removing the securityContext with no change in behaviour. Our service do not use IHostedService, it is just a console application. Do we need to make sure that there is some specific nuget package installed in the application?

/metrics is empty

Result from /info

{
    "version": "6.1.0+48853dad0d5d1efd50a90904505f68a137c7b789",
    "runtimeVersion": "6.0.2",
    "diagnosticPortMode": "Listen",
    "diagnosticPortName": "/diag/port"
}

Logmessage on /logs, /livemetrics

{
    "Timestamp": "2022-03-01T22:03:42.3496842Z",
    "EventId": 1,
    "LogLevel": "Error",
    "Category": "Microsoft.Diagnostics.Monitoring.WebApi.Controllers.DiagController",
    "Message": "Request failed.",
    "Exception": "System.ArgumentException: Unable to discover a target process.    at Microsoft.Diagnostics.Monitoring.WebApi.DiagnosticServices.GetProcessAsync(DiagProcessFilter processFilterConfig, CancellationToken token) in /_/src/Microsoft.Diagnostics.Monitoring.WebApi/DiagnosticServices.cs:line 94    at Microsoft.Diagnostics.Monitoring.WebApi.Controllers.DiagController.\u003C\u003Ec__DisplayClass33_0\u00601.\u003C\u003CInvokeForProcess\u003Eb__0\u003Ed.MoveNext() in /_/src/Microsoft.Diagnostics.Monitoring.WebApi/Controllers/DiagController.cs:line 704 --- End of stack trace from previous location ---    at Microsoft.Diagnostics.Monitoring.WebApi.Controllers.DiagControllerExtensions.InvokeService[T](ControllerBase controller, Func\u00601 serviceCall, ILogger logger) in /_/src/Microsoft.Diagnostics.Monitoring.WebApi/Controllers/DiagControllerExtensions.cs:line 57",
    "State": {
        "Message": "Request failed.",
        "{OriginalFormat}": "Request failed."
    },
    "Scopes": [
        {
            "Message": "SpanId:c18db2e0c5361c36, TraceId:c313035cd8e27c421ea1fc0390e5ee1e, ParentId:0000000000000000",
            "SpanId": "c18db2e0c5361c36",
            "TraceId": "c313035cd8e27c421ea1fc0390e5ee1e",
            "ParentId": "0000000000000000"
        },
        {
            "Message": "ConnectionId:0HMFRPSIV145K",
            "ConnectionId": "0HMFRPSIV145K"
        },
        {
            "Message": "RequestPath:/logs RequestId:0HMFRPSIV145K:00000007",
            "RequestId": "0HMFRPSIV145K:00000007",
            "RequestPath": "/logs"
        },
        {
            "Message": "Microsoft.Diagnostics.Monitoring.WebApi.Controllers.DiagController.CaptureLogs (Microsoft.Diagnostics.Monitoring.WebApi)",
            "ActionId": "70802734-2e24-4f0c-aa33-fcbd38df3145",
            "ActionName": "Microsoft.Diagnostics.Monitoring.WebApi.Controllers.DiagController.CaptureLogs (Microsoft.Diagnostics.Monitoring.WebApi)"
        },
        {
            "Message": "ArtifactType:logs",
            "ArtifactType": "logs"
        }
    ]
}

Configuration

# Source: mytestapp/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mytestapp
  labels:
    app: mytestapp
    helm.sh/chart: mytestapp-1.0.0
    app.kubernetes.io/name: mytestapp
    app.kubernetes.io/instance: mytestapp
    app.kubernetes.io/managed-by: Helm
spec:
  replicas: 1
  revisionHistoryLimit: 0
  selector:
    matchLabels:
      app.kubernetes.io/name: mytestapp
      app.kubernetes.io/instance: mytestapp
  template:
    metadata:
      labels:
        app: mytestapp
        helm.sh/chart: mytestapp-1.0.0
        app.kubernetes.io/name: mytestapp
        app.kubernetes.io/instance: mytestapp
        app.kubernetes.io/managed-by: Helm
    spec:
      securityContext:
        seccompProfile:
          type: RuntimeDefault
        runAsUser: 952
        runAsGroup: 952
        runAsNonRoot: true
        fsGroup: 952
      volumes:
        - name: diagvol
          emptyDir: {}
        - name: dumpsvol
          emptyDir: {}
      containers:
        - name: mytestapp
          image: "myrepository.azurecr.io/mytestapp:1.0.0"
          imagePullPolicy: IfNotPresent
          securityContext:
            capabilities:
              drop:
                - ALL
            allowPrivilegeEscalation: false          
         env:
            - name: DOTNET_DiagnosticPorts
              value: /diag/port
          ports:
            - name: testcp
              containerPort: 49706
              protocol: TCP
          volumeMounts:
            - mountPath: /diag
              name: diagvol
            - mountPath: /dumps
              name: dumpsvol
          resources:
            {}
          env: []
        - name: mytestapp-monitor
          image: "mcr.microsoft.com/dotnet/monitor:6.1-alpine"
          # DO NOT use the --no-auth argument for deployments in production; this argument is used for demonstration
          # purposes only in this example. Please continue reading after this example for further details.
          args: [ "--no-auth" ]
          imagePullPolicy: IfNotPresent
          securityContext:
            capabilities:
              drop:
                - ALL
            allowPrivilegeEscalation: false
          ports:
            - name: monitor
              containerPort: 52323
              protocol: TCP
          env:
          - name: DOTNETMONITOR_DiagnosticPort__ConnectionMode
            value: Listen
          - name: DOTNETMONITOR_DiagnosticPort__EndpointName
            value: /diag/port
          - name: DOTNETMONITOR_Storage__DumpTempFolder
            value: /dumps
          # ALWAYS use the HTTPS form of the URL for deployments in production; the removal of HTTPS is done for
          # demonstration purposes only in this example. Please continue reading after this example for further details.
          - name: DOTNETMONITOR_Urls
            value: http://localhost:52323
          volumeMounts:
          - mountPath: /diag
            name: diagvol
          - mountPath: /dumps
            name: dumpsvol
          resources:
            {}

Regression?

Other information

[jander-msft]

Unfortunately, the sample as written only works for .NET 5+. I'll put out a PR to show how to do this for .NET Core 3.1 and link back here. The gist is that you need to mount the /tmp directory between the application container and the dotnet monitor container, remove the DOTNET_DiagnosticPorts env var from the application container, and remove the DOTNETMONITOR_DiagnosticPort__ConnectionMode and DOTNETMONITOR_DiagnosticPort__EndpointName env vars from the dotnet monitor container.

[steinarox]

@jander-msft thanks! after upgrading to .net6 it works 😎. Yes a readme for 3.1 and/or warning in the example for this limitation would me great. I'll let you close this one when you PR lands

[WeihanLi]

You had to use the Connect mode for .NET Core 3.1 apps, I had a .NET Core 3.1 sample, the deployment yaml likes below:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: sample
  name: sample
spec:
  revisionHistoryLimit: 1
  selector:
    matchLabels:
      app: sample
  replicas: 1
  strategy:
    rollingUpdate:
      maxSurge: 50%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
      labels:
        app: sample
    spec:
      containers:
      - name: sample
        ports:
        - containerPort: 80
        env:
        - name: DOTNET_DiagnosticPorts
          value: /diag/port
        volumeMounts:
        - mountPath: /diag
          name: diagvol
        - mountPath: /dumps
          name: dumpsvol
        - mountPath: /tmp
          name: tmpvol
      - name: dotnet-monitor
        image: mcr.microsoft.com/dotnet/monitor:6.0.0
        args: [ "--no-auth" ]
        imagePullPolicy: Always
        ports:
          - containerPort: 52323
        env:
        - name: DOTNETMONITOR_DiagnosticPort__ConnectionMode
          value: Connect
        - name: DOTNETMONITOR_DiagnosticPort__EndpointName
          value: /diag/port
        - name: DOTNETMONITOR_Storage__DumpTempFolder
          value: /dumps
        - name: DOTNETMONITOR_Urls
          value: "http://+:52323"
        volumeMounts:
        - mountPath: /diag
          name: diagvol
        - mountPath: /dumps
          name: dumpsvol
        - mountPath: /tmp
          name: tmpvol
        resources:
          requests:
            cpu: 50m
            memory: 32Mi
          limits:
            cpu: 250m
            memory: 256Mi
      volumes:
      - name: diagvol
        emptyDir: {}
      - name: dumpsvol
        emptyDir: {}
      - name: tmpvol
        emptyDir: {}

[jander-msft]

With .NET Core 3.1 out of support earlier this month, we will likely not invest in updating documentation for .NET Core 3.1 specific deployments.