Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add reversed diagnostics server, endpoints, and endpoint sources. #1303

Merged
merged 55 commits into from
Aug 6, 2020
Merged
Show file tree
Hide file tree
Changes from 54 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
bd9ccc8
Add reversed diagnostics server and connections.
jander-msft Jun 15, 2020
cef68e9
Catch exception when advertise data is incomplete and wait for new co…
jander-msft Jun 30, 2020
abadf40
PR Feedback
jander-msft Jul 1, 2020
f6feb8e
Send ResumeRuntime command for reverse connected runtime instances.
jander-msft Jul 1, 2020
bc11c01
Handle exceptions when waiting for initial transport connection.
jander-msft Jul 1, 2020
58310dd
Rename IIpcEndpoint.CheckTransport to HasTransport and reimplement to…
jander-msft Jul 1, 2020
b038235
Let cancellation propagate out of ReversedDiagnosticsServer.AcceptAsync
jander-msft Jul 1, 2020
4e6a070
Fix transport name calculation.
jander-msft Jul 1, 2020
30bfa2d
Fix transport name calculation (again).
jander-msft Jul 1, 2020
a924d5b
Do not automatically send ResumeRuntime command from within server.
jander-msft Jul 2, 2020
0af15d6
Change back to CheckTransport and update to check the connectivity fo…
jander-msft Jul 2, 2020
eb5caf8
More synchronization of stream within ServerIpcEndpoint.
jander-msft Jul 2, 2020
8301660
Ensure connection has stream before inserting into connections source.
jander-msft Jul 2, 2020
79bc590
PR Feedback
jander-msft Jul 13, 2020
01d1c92
Recreate unix domain socket if cancelled accept.
jander-msft Jul 13, 2020
202ab10
Fix typo
jander-msft Jul 15, 2020
8b1050b
Move ResumeRuntime command into process command set.
jander-msft Jul 15, 2020
5e66596
Refactor CheckTransport into WaitForConnectionAsync.
jander-msft Jul 15, 2020
577a2c2
Increase connection accepter timeout in unit tests.
jander-msft Jul 17, 2020
99b5240
Parse advertise structure async to allow for cancellation to avoid in…
jander-msft Jul 17, 2020
18bc3c3
Refactor socket endpoint creation into UnixDomainSocket class.
jander-msft Jul 17, 2020
707c5c6
Refactor the client IPC endpoints so that PidIpcEndpoint can test the…
jander-msft Jul 18, 2020
6bc2d9b
Refactor IPC transports to test and provide streams consistently.
jander-msft Jul 18, 2020
dcb24ae
Merge branch 'master' of https://github.com/dotnet/diagnostics into d…
jander-msft Jul 21, 2020
3c993a7
Fix newly added API
jander-msft Jul 21, 2020
a9c1543
Fix wait handle notification to avoid race condition.
jander-msft Jul 21, 2020
3cf7d16
Dispose connection and endpoint if failed to add to mapping.
jander-msft Jul 21, 2020
32012be
Add stream semaphore to lock stream field mutation.
jander-msft Jul 21, 2020
0b252ed
Increase event production timeout.
jander-msft Jul 21, 2020
c098773
Fix NRE and possible infinite loop in ReversedDiagnosticsServer.
jander-msft Jul 22, 2020
01c743e
Simplify BaseIpcEndpoint and reduce timeouts.
jander-msft Jul 24, 2020
c131328
Merge branch 'master' of https://github.com/dotnet/diagnostics into d…
jander-msft Jul 24, 2020
a88d464
Refactor IpcEndpoint implementations so that they are stateless and i…
jander-msft Jul 24, 2020
aba487f
Reincorporate cancellation token from ProcessEvents method into stopp…
jander-msft Jul 24, 2020
edd475a
Use Interlocked instead of TaskCompletionSource for reversed server C…
jander-msft Jul 24, 2020
d199874
More tests and minor fixes.
jander-msft Jul 26, 2020
a89ab94
Fix exception type checks
jander-msft Jul 27, 2020
e0ed65e
Throw ObjectDisposedException for pending WaitForConnectionAsync and …
jander-msft Jul 27, 2020
366be3e
Fix another exception test.
jander-msft Jul 27, 2020
1988e98
Use ConfigureAwait(false) in new library code.
jander-msft Jul 28, 2020
5a9df7a
Add cancellation checks to ReversedDiagnosticsServer.AcceptAsync method.
jander-msft Jul 28, 2020
cc20bb8
Force remote process to stop if not gracefully stopped.
jander-msft Jul 28, 2020
4ea37c8
Rename connection source concepts.
jander-msft Jul 28, 2020
3e770a5
Simplify event pipe testing in reversed server tests and add more log…
jander-msft Jul 29, 2020
d2c1967
Fix typo (TaskContinuationOptions to TaskCreationOptions)
jander-msft Jul 29, 2020
d4ecce8
Add ResumeRuntime command fallback for Preview 7 and Preview 8
jander-msft Jul 30, 2020
2f1e4fc
Merge branch 'master' of https://github.com/dotnet/diagnostics into d…
jander-msft Jul 30, 2020
394a509
Prevent tracee from exiting until receive key.
jander-msft Jul 31, 2020
5316a44
Run multi client verification in parallel
jander-msft Jul 31, 2020
3cde474
Remove multi target test.
jander-msft Jul 31, 2020
88a90f4
Consistent naming in comments.
jander-msft Aug 1, 2020
267514b
Add timeout to UnixDomainSocket.Connect method.
jander-msft Aug 3, 2020
b54cf48
Remove try..catch from WaitForConnectionAsync.
jander-msft Aug 3, 2020
508088b
PR Feedback
jander-msft Aug 4, 2020
408904e
Merge branch 'master' of https://github.com/dotnet/diagnostics into d…
jander-msft Aug 5, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions eng/Versions.props
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
<MicrosoftExtensionsConfigurationJsonVersion>2.1.1</MicrosoftExtensionsConfigurationJsonVersion>
<MicrosoftExtensionsConfigurationKeyPerFileVersion>2.1.1</MicrosoftExtensionsConfigurationKeyPerFileVersion>
<MicrosoftExtensionsDependencyInjectionVersion>2.1.1</MicrosoftExtensionsDependencyInjectionVersion>
<MicrosoftExtensionsHostingAbstractionsVersion>2.1.1</MicrosoftExtensionsHostingAbstractionsVersion>
<MicrosoftExtensionsLoggingVersion>2.1.1</MicrosoftExtensionsLoggingVersion>
<MicrosoftExtensionsLoggingConsoleVersion>2.1.1</MicrosoftExtensionsLoggingConsoleVersion>
<!-- We use a newer version of LoggingEventSource due to a bug in an older version-->
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,14 +44,14 @@ public DiagController(ILogger<DiagController> logger, IDiagnosticServices diagno
}

[HttpGet("processes")]
public ActionResult<IEnumerable<ProcessModel>> GetProcesses()
public Task<ActionResult<IEnumerable<ProcessModel>>> GetProcesses()
{
return this.InvokeService(() =>
return this.InvokeService(async () =>
{
IList<ProcessModel> processes = new List<ProcessModel>();
foreach (int pid in _diagnosticServices.GetProcesses())
foreach (IProcessInfo p in await _diagnosticServices.GetProcessesAsync(HttpContext.RequestAborted))
{
processes.Add(new ProcessModel() { Pid = pid });
processes.Add(ProcessModel.FromProcessInfo(p));
}
return new ActionResult<IEnumerable<ProcessModel>>(processes);
});
Expand All @@ -62,8 +62,8 @@ public Task<ActionResult> GetDump(int? pid, [FromQuery] DumpType type = DumpType
{
return this.InvokeService(async () =>
{
int pidValue = _diagnosticServices.ResolveProcess(pid);
Stream result = await _diagnosticServices.GetDump(pidValue, type);
int pidValue = await _diagnosticServices.ResolveProcessAsync(pid, HttpContext.RequestAborted);
Stream result = await _diagnosticServices.GetDump(pidValue, type, HttpContext.RequestAborted);

string dumpFileName = RuntimeInformation.IsOSPlatform(OSPlatform.Windows) ?
FormattableString.Invariant($"dump_{GetFileNameTimeStampUtcNow()}.dmp") :
Expand All @@ -80,7 +80,7 @@ public Task<ActionResult> GetGcDump(int? pid)
{
return this.InvokeService(async () =>
{
int pidValue = _diagnosticServices.ResolveProcess(pid);
int pidValue = await _diagnosticServices.ResolveProcessAsync(pid, HttpContext.RequestAborted);
Stream result = await _diagnosticServices.GetGcDump(pidValue, this.HttpContext.RequestAborted);
return File(result, "application/octet-stream", FormattableString.Invariant($"{GetFileNameTimeStampUtcNow()}_{pidValue}.gcdump"));
});
Expand Down Expand Up @@ -159,12 +159,12 @@ public Task<ActionResult> TraceCustomConfiguration(

[HttpGet("logs/{pid?}")]
[Produces(ContentTypeEventStream, ContentTypeNdJson, ContentTypeJson)]
public ActionResult Logs(int? pid, [FromQuery][Range(-1, int.MaxValue)] int durationSeconds = 30, [FromQuery] LogLevel level = LogLevel.Debug)
public Task<ActionResult> Logs(int? pid, [FromQuery][Range(-1, int.MaxValue)] int durationSeconds = 30, [FromQuery] LogLevel level = LogLevel.Debug)
{
TimeSpan duration = ConvertSecondsToTimeSpan(durationSeconds);
return this.InvokeService(() =>
return this.InvokeService(async () =>
{
int pidValue = _diagnosticServices.ResolveProcess(pid);
int pidValue = await _diagnosticServices.ResolveProcessAsync(pid, HttpContext.RequestAborted);

LogFormat format = ComputeLogFormat(Request.GetTypedHeaders().Accept);
if (format == LogFormat.None)
Expand All @@ -184,7 +184,7 @@ public ActionResult Logs(int? pid, [FromQuery][Range(-1, int.MaxValue)] int dura

private async Task<StreamWithCleanupResult> StartTrace(int? pid, MonitoringSourceConfiguration configuration, TimeSpan duration)
{
int pidValue = _diagnosticServices.ResolveProcess(pid);
int pidValue = await _diagnosticServices.ResolveProcessAsync(pid, HttpContext.RequestAborted);
IStreamWithCleanup result = await _diagnosticServices.StartTrace(pidValue, configuration, duration, this.HttpContext.RequestAborted);
return new StreamWithCleanupResult(result, "application/octet-stream", FormattableString.Invariant($"{GetFileNameTimeStampUtcNow()}_{pidValue}.nettrace"));
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,12 @@
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.

using Microsoft.Diagnostics.Tracing.Analysis;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Options;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Diagnostics.NETCore.Client;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Options;

namespace Microsoft.Diagnostics.Monitoring.RestServer
{
Expand Down Expand Up @@ -46,8 +43,9 @@ protected override Task ExecuteAsync(CancellationToken stoppingToken)
{
//TODO In multi-process scenarios, how do we decide which process to choose?
//One possibility is to enable metrics after a request to begin polling for metrics
int pid = _services.ResolveProcess(pid: null);
await _pipeProcessor.Process(pid, Timeout.InfiniteTimeSpan, stoppingToken);
int pid = await _services.ResolveProcessAsync(pid: null, stoppingToken);
var client = new DiagnosticsClient(pid);
await _pipeProcessor.Process(client, pid, Timeout.InfiniteTimeSpan, stoppingToken);
}
catch(Exception e) when (!(e is OperationCanceledException))
{
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
using System.Runtime.Serialization;
using System;
using System.Runtime.Serialization;

namespace Microsoft.Diagnostics.Monitoring.RestServer.Models
{
Expand All @@ -7,5 +8,13 @@ public class ProcessModel
{
[DataMember(Name = "pid")]
public int Pid { get; set; }

[DataMember(Name = "uid")]
public Guid Uid { get; set; }

public static ProcessModel FromProcessInfo(IProcessInfo processInfo)
{
return new ProcessModel() { Pid = processInfo.Pid, Uid = processInfo.Uid };
}
}
}
44 changes: 44 additions & 0 deletions src/Microsoft.Diagnostics.Monitoring/ClientEndpointInfoSource.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Diagnostics.NETCore.Client;

namespace Microsoft.Diagnostics.Monitoring
{
internal sealed class ClientEndpointInfoSource : IEndpointInfoSourceInternal
{
public Task<IEnumerable<IEndpointInfo>> GetEndpointInfoAsync(CancellationToken token)
{
List<IEndpointInfo> endpointInfos = new List<IEndpointInfo>();
foreach (int pid in DiagnosticsClient.GetPublishedProcesses())
{
// CONSIDER: Generate a "runtime instance identifier" based on the pipe name
// e.g. pid + disambiguator in GUID form.
jander-msft marked this conversation as resolved.
Show resolved Hide resolved
endpointInfos.Add(new EndpointInfo(pid));
}

return Task.FromResult(endpointInfos.AsEnumerable());
}

private class EndpointInfo : IEndpointInfo
{
public EndpointInfo(int processId)
{
Endpoint = new PidIpcEndpoint(processId);
ProcessId = processId;
}

public IpcEndpoint Endpoint { get; }

public int ProcessId { get; }

public Guid RuntimeInstanceCookie => Guid.Empty;
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ public sealed class CpuProfileConfiguration : MonitoringSourceConfiguration
public override IList<EventPipeProvider> GetProviders() =>
new EventPipeProvider[]
{
new EventPipeProvider("Microsoft-DotNETCore-SampleProfiler", System.Diagnostics.Tracing.EventLevel.Informational),
new EventPipeProvider(SampleProfilerProviderName, System.Diagnostics.Tracing.EventLevel.Informational),
new EventPipeProvider("Microsoft-Windows-DotNETRuntime", System.Diagnostics.Tracing.EventLevel.Informational, (long) Tracing.Parsers.ClrTraceEventParser.Keywords.Default)
};
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ public abstract class MonitoringSourceConfiguration
public const string GrpcAspNetCoreServer = "Grpc.AspNetCore.Server";
public const string DiagnosticSourceEventSource = "Microsoft-Diagnostics-DiagnosticSource";
public const string TplEventSource = "System.Threading.Tasks.TplEventSource";
public const string SampleProfilerProviderName = "Microsoft-DotNETCore-SampleProfiler";
public const string EventPipeProviderName = "Microsoft-DotNETCore-EventPipe";

public abstract IList<EventPipeProvider> GetProviders();

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.

using Microsoft.Diagnostics.NETCore.Client;
using System.Collections.Generic;
using System.Diagnostics.Tracing;

namespace Microsoft.Diagnostics.Monitoring
{
public sealed class SampleProfilerConfiguration : MonitoringSourceConfiguration
{
public override IList<EventPipeProvider> GetProviders() =>
new EventPipeProvider[]
{
new EventPipeProvider(SampleProfilerProviderName, EventLevel.Informational)
};

public override int BufferSizeInMB => 1;

public override bool RequestRundown => false;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,11 @@ namespace Microsoft.Diagnostics.Monitoring
/// </summary>
public interface IDiagnosticServices : IDisposable
{
IEnumerable<int> GetProcesses();
Task<IEnumerable<IProcessInfo>> GetProcessesAsync(CancellationToken token);

int ResolveProcess(int? pid);
Task<int> ResolveProcessAsync(int? pid, CancellationToken token);

Task<Stream> GetDump(int pid, DumpType mode);
Task<Stream> GetDump(int pid, DumpType mode, CancellationToken token);

Task<Stream> GetGcDump(int pid, CancellationToken token);

Expand All @@ -35,6 +35,13 @@ public interface IStreamWithCleanup : IAsyncDisposable
Stream Stream { get; }
}

public interface IProcessInfo
{
int Pid { get; }

Guid Uid { get; }
}

public enum DumpType
{
Full = 1,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.

using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Diagnostics.NETCore.Client;

namespace Microsoft.Diagnostics.Monitoring
{
internal interface IEndpointInfo
{
IpcEndpoint Endpoint { get; }

int ProcessId { get; }

Guid RuntimeInstanceCookie { get; }
}

public interface IEndpointInfoSource
{
}

internal interface IEndpointInfoSourceInternal : IEndpointInfoSource
{
Task<IEnumerable<IEndpointInfo>> GetEndpointInfoAsync(CancellationToken token);
}
}
Loading