Skip to content
This repository has been archived by the owner on Jan 23, 2023. It is now read-only.

Reduce Execution Context Save+Restore #15629

Merged
merged 10 commits into from
Jan 31, 2018

Conversation

benaadams
Copy link
Member

@benaadams benaadams commented Dec 24, 2017

x 1.94 speed up for ExecutionContext.Run(ec, (o) => { }, null); where ec == Default
x 1.57 speed up for ExecutionContext.Run(ec, (o) => { }, null); with AsyncLocals (no notifications)

Results #15629 (comment)

Alternative approach to #11100 as suggested by @kouvel #11100 (comment)

When Default context is used in Run use it as null; which also counts as Default - rather than having two forms of Default context internally (Thread starts will null) and saving and restoring between them.

                Method |   Current |       New |    Change |
---------------------- |----------:|----------:|----------:|
        Run Default EC | 14.134 ns |  7.303 ns |    x 1.94 |
 Run w/ AsyncLocals EC | 11.986 ns |  7.631 ns |    x 1.57 | 
                 await |  3.152 ns |  3.158 ns |    x 1.00 | 

ASP.NET Kestrel Plaintext call counts

Pre
image

Post
image

Resolves #11126

@benaadams
Copy link
Member Author

benaadams commented Dec 24, 2017

Total bytes of diff: -1467 (-0.04% of base)
    diff is an improvement.

Total byte diff includes -184 bytes from reconciling methods
        Base had    8 unique methods,      660 unique bytes
        Diff had    3 unique methods,      476 unique bytes

Top file improvements by size (bytes):
       -1467 : System.Private.CoreLib.dasm (-0.04% of base)

1 total files with size differences (1 improved, 0 regressed), 0 unchanged.

Top method regessions by size (bytes):
         452 : System.Private.CoreLib.dasm - ExecutionContext:Restore(byref,ref) (0/1 methods)
          78 : System.Private.CoreLib.dasm - ExecutionContext:SetLocalValue(ref,ref,bool)
          63 : System.Private.CoreLib.dasm - ExecutionContext:Run(ref,ref,ref)
          12 : System.Private.CoreLib.dasm - Thread:get_ExecutionContext():byref:this (0/1 methods)
          12 : System.Private.CoreLib.dasm - Thread:get_SynchronizationContext():byref:this (0/1 methods)
           2 : System.Private.CoreLib.dasm - SynchronizationContext:SetSynchronizationContext(ref)
           2 : System.Private.CoreLib.dasm - SynchronizationContext:get_Current():ref
           2 : System.Private.CoreLib.dasm - ExecutionContext:Capture():ref

Top method improvements by size (bytes):
       -1438 : System.Private.CoreLib.dasm - AsyncTaskMethodBuilder`1:Start(byref):this (12 methods)
        -384 : System.Private.CoreLib.dasm - ExecutionContext:OnContextChanged(ref,ref) (1/0 methods)
        -123 : System.Private.CoreLib.dasm - ExecutionContext:Restore(ref,ref) (1/0 methods)
         -61 : System.Private.CoreLib.dasm - ExecutionContextSwitcher:Undo(ref):this (1/0 methods)
         -40 : System.Private.CoreLib.dasm - ExecutionContext:EstablishCopyOnWriteScope(ref,byref) (1/0 methods)
         -16 : System.Private.CoreLib.dasm - Thread:set_ExecutionContext(ref):this (1/0 methods)
         -16 : System.Private.CoreLib.dasm - Thread:set_SynchronizationContext(ref):this (1/0 methods)
         -10 : System.Private.CoreLib.dasm - Thread:get_ExecutionContext():ref:this (1/0 methods)

21 total methods with size differences (9 improved, 12 regressed), 16738 unchanged.

@jkotas
Copy link
Member

jkotas commented Dec 24, 2017

@dotnet-bot test Windows_NT x64 Checked corefx_baseline
@dotnet-bot test Ubuntu x64 Checked corefx_baseline

@benaadams benaadams changed the title [WIP] Reduce EC Save+Restore for Default context Reduce EC Save+Restore for Default context Dec 24, 2017
@benaadams
Copy link
Member Author

@dotnet-bot test Windows_NT x64 Checked corefx_baseline
@dotnet-bot test Ubuntu x64 Checked corefx_baseline

@benaadams
Copy link
Member Author

benaadams commented Dec 24, 2017

x 1.58 speed up for ExecutionContext.Run(ec, (o) => { }, null); where ec == Default

Pre

 Method |      Mean |     Error |    StdDev |        Op/s |
------- |----------:|----------:|----------:|------------:|
    Run | 14.690 ns | 0.2907 ns | 0.4261 ns |  68,073,519 |

Post

 Method |      Mean |     Error |    StdDev |        Op/s |
------- |----------:|----------:|----------:|------------:|
    Run |  9.263 ns | 0.0123 ns | 0.0115 ns | 107,956,385 |

https://gist.github.com/benaadams/73fedf79313a90bdef4f4f617cbb7aa8

@benaadams
Copy link
Member Author

benaadams commented Dec 25, 2017

Corresponding corert change dotnet/corert#5153

@benaadams
Copy link
Member Author

Previous call tracing
image

Post call tracing (Restore not invoked)
image

@benaadams
Copy link
Member Author

PTAL @stephentoub

Thread currentThread = Thread.CurrentThread;
ExecutionContextSwitcher ecs = default(ExecutionContextSwitcher);
try
// Async state machines are required not to throw, so no need for try/finally here.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As described in #11156, this is debatable. Let's keep PRs focused on one thing rather than trying to cram such changes together.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

@stephentoub
Copy link
Member

x 1.58 speed up for ExecutionContext.Run(ec, (o) => { }, null); where ec == Default

Any impact when not using the default?

@@ -130,126 +108,164 @@ public static bool IsFlowSuppressed()

public static void Run(ExecutionContext executionContext, ContextCallback callback, Object state)
{
// Note: ExecutionContext.Run is an extremly hot function and used by every await, threadpool execution etc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: "threadpool execution etc" => "ThreadPool execution, etc."

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also typo: extremly


Thread currentThread = Thread.CurrentThread;
ExecutionContextSwitcher ecsw = default(ExecutionContextSwitcher);
// Capture references to Thread Contexts
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: move the comment up a line


Thread currentThread = Thread.CurrentThread;
ExecutionContextSwitcher ecsw = default(ExecutionContextSwitcher);
// Capture references to Thread Contexts
ref ExecutionContext current = ref currentThread.ExecutionContext;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: current => currentExecutionCtx

// Store current ExecutionContext and SynchronizationContext as "previous"
// This allows us to restore them and undo any Context changes made in callback.Invoke
// so that they won't "leak" back into caller.
ExecutionContext previous = current;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: previous => previousExecutionCtx

ref ExecutionContext current = ref currentThread.ExecutionContext;
ref SynchronizationContext currentSyncCtx = ref currentThread.SynchronizationContext;

// Store current ExecutionContext and SynchronizationContext as "previous"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: missing period at end of sentence.

}

private static void OnContextChanged(ExecutionContext previous, ExecutionContext current)
internal static void Restore(ref ExecutionContext current, ExecutionContext next)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restore seems like the wrong naming for this. SetCurrentContext?

Copy link
Member

@stephentoub stephentoub Jan 3, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, maybe currentRef instead of current would make this easier to understand when reading the code?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to OnValuesChanged as it only gets triggered if at least once side has notifications

ecsw.Undo(currentThread);
throw;
// context before any of our callers' EH filters run.
edi = ExceptionDispatchInfo.Capture(ex);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this going to change the call stack from what it would have been, adding in the whole "rethrown from" gook? That's going to affect pretty much every async invocation that incurs an exception, no?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its the same in both cases

Copy link
Member

@stephentoub stephentoub Jan 7, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really? This:

using System;
using System.Runtime.ExceptionServices;

class Program
{
    static void Main()
    {
        try { Foo1(); } catch (Exception e) { Console.WriteLine(e.StackTrace); }
        Console.WriteLine();
        try { Foo2(); } catch (Exception e) { Console.WriteLine(e.StackTrace); }
    }

    static void Foo1()
    {
        try { throw new Exception("test"); }
        catch { throw; }
    }

    static void Foo2()
    {
        ExceptionDispatchInfo edi = null;
        try { throw new Exception("test"); }
        catch (Exception e) { edi = ExceptionDispatchInfo.Capture(e); }
        edi.Throw();
    }
}

produces this for me:

>corerun test.exe
   at Program.Foo1() in c:\Users\stoub\Desktop\CoreClrTest\test.cs:line 16
   at Program.Main() in c:\Users\stoub\Desktop\CoreClrTest\test.cs:line 8

   at Program.Foo2() in c:\Users\stoub\Desktop\CoreClrTest\test.cs:line 22
--- End of stack trace from previous location where exception was thrown ---
   at Program.Main() in c:\Users\stoub\Desktop\CoreClrTest\test.cs:line 10

You're saying though that with your change that --- End of stack trace... doesn't show up when an exception emerges from EC.Run?

(It's not a deal-breaker, but given how common this method is, it'd be nice to avoid the extra gook. If it's not possible, ok.)

Copy link
Member Author

@benaadams benaadams Jan 8, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Been trying to get this to show up in async without much luck; but have since realized that async never lets an exception get caught here (i.e. its already caught it to fail the Task)

Copy link
Member Author

@benaadams benaadams Jan 8, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just need to change StackTrace from

if (sf.GetIsLastFrameFromForeignExceptionStackTrace() &&
    !isAsync) // Skip EDI boundary for async

to

if (sf.GetIsLastFrameFromForeignExceptionStackTrace() &&
    !isAsync && // Skip EDI boundary for async
    declaringType != typeof(ExecutionContext)) // Skip EDI boundary for ExecutionContext.Run

However would need to rebase; and also I said I'd finished making changes to this PR 😉

internal static void EstablishCopyOnWriteScope(Thread currentThread, ref ExecutionContextSwitcher ecsw)
{
Debug.Assert(currentThread == Thread.CurrentThread);
if (current != previous)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any benefit to storing a bool earlier when we do the previous != executionContext check and using that bool here rather than redoing the comparison with the ref?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess current could have changed back to previous already, could be if(executionContextChanged && current != previous)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it would be better, however the callee may have changed the EC by setting an AsyncLocal or unpaired SuppressFlow/RestoreFlow call; so it needs to check the Thread's contexts again?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so it needs to check the Thread's contexts again?

Yup, you're right.

foreach (IAsyncLocal local in previous.m_localChangeNotifications)
{
previous.m_localValues.TryGetValue(local, out object previousValue);
object currentValue = null;
Copy link
Member

@stephentoub stephentoub Jan 3, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The = null should not be necessary, in which case it can be out object currentValue as on the previous line.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nevermind, I missed the ? in the next? on the next line.

{
newValues = AsyncLocalValueMap.Empty.Set(local, newValue);;
newChangeNotifications = Array.Empty<IAsyncLocal>();
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you measured the cost of setting async locals with this change?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wasn't entirely happy with this; will see if can improve

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is better now

// Run the MoveNext method within a copy-on-write ExecutionContext scope.
// This allows us to undo any ExecutionContext changes made in MoveNext,
Thread currentThread = Thread.CurrentThread;
// Capture references to Thread Contexts
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: same nits as earlier

Copy link
Member

@kouvel kouvel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM aside from other comments, thanks!

@@ -130,126 +108,164 @@ public static bool IsFlowSuppressed()

public static void Run(ExecutionContext executionContext, ContextCallback callback, Object state)
{
// Note: ExecutionContext.Run is an extremly hot function and used by every await, threadpool execution etc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also typo: extremly

internal static void EstablishCopyOnWriteScope(Thread currentThread, ref ExecutionContextSwitcher ecsw)
{
Debug.Assert(currentThread == Thread.CurrentThread);
if (current != previous)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess current could have changed back to previous already, could be if(executionContextChanged && current != previous)

@benaadams benaadams changed the title Reduce EC Save+Restore for Default context [Wip] Reduce EC Save+Restore for Default context Jan 4, 2018
@benaadams
Copy link
Member Author

benaadams commented Jan 4, 2018

Any impact when not using the default?

Think I have a version that may improve both default and non-default (without notifications); setting up some scenarios to test properly.

@benaadams benaadams changed the title [Wip] Reduce EC Save+Restore for Default context Reduce EC Save+Restore for Default context Jan 6, 2018
@kouvel
Copy link
Member

kouvel commented Jan 30, 2018

@stephentoub, should we go ahead and merge this?

@stephentoub
Copy link
Member

I'll take one more look tomorrow. Thanks.

@stephentoub
Copy link
Member

@dotnet-bot test CentOS7.1 x64 Release Innerloop Build and Test please
@dotnet-bot test Ubuntu arm Cross Debug Innerloop Build please

@benaadams
Copy link
Member Author

@dotnet-bot test CentOS7.1 x64 Release Innerloop Build and Test please
@dotnet-bot test Ubuntu arm Cross Debug Innerloop Build please

No longer exist on CI

@stephentoub
Copy link
Member

No longer exist on CI

Oh, you're right. Ok.

/// <summary>Initiates the builder's execution with the associated state machine.</summary>
/// <typeparam name="TStateMachine">Specifies the type of the state machine.</typeparam>
/// <param name="stateMachine">The state machine instance, passed by reference.</param>
[DebuggerStepThrough]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why DebuggerStepThrough?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, nevermind, I see that it's there on the original for some reason. Not sure why it's there either, but we can leave this for consistency.

Thread.CurrentThread.ExecutionContext =
new ExecutionContext(newValues, newChangeNotifications, current.m_isFlowSuppressed);
Thread.CurrentThread.ExecutionContext =
(!isFlowSuppressed && newValues.GetType() == typeof(AsyncLocalValueMap.EmptyAsyncLocalValueMap)) ?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In what situation could newValues be empty here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, if null was set as the value for an already empty map I guess?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the value in OneElementAsyncLocalValueMap is set to null

@@ -94,7 +94,8 @@ static internal void IOCompletionCallback_Context(Object state)
overlapped = OverlappedData.GetOverlappedFromNative(pOVERLAP).m_overlapped;
helper = overlapped.iocbHelper;

if (helper == null || helper._executionContext == null || helper._executionContext == ExecutionContext.Default)
ExecutionContext context = helper?._executionContext;
if (context == null || context.IsDefault)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If helper is null, don't we end up with an extra null check (checking context for null when we otherwise wouldn't)? Or is the JIT able to remove it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No doesn't, will change

(IThreadPoolWorkItem)new QueueUserWorkItemCallback(callBack, state, context);
IThreadPoolWorkItem tpcallBack = (context == null || !context.IsDefault) ?
new QueueUserWorkItemCallback(callBack, state, context) :
(IThreadPoolWorkItem)new QueueUserWorkItemCallbackDefaultContext(callBack, state);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's fine, but I'm curious why you swapped the polarity here. Did that have an impact on something?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So could combine the context == null || !context.IsDefault tests; otherwise it was a bit ugly

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

context != null && context.IsDefault, no?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may have over thought it 😄

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted

@benaadams
Copy link
Member Author

@dotnet-bot test Windows_NT x64 Checked corefx_baseline
@dotnet-bot test Ubuntu x64 Checked corefx_baseline

@benaadams
Copy link
Member Author

benaadams commented Jan 31, 2018

coreclr and corefx out of sync

MembersMustExist :
 Member 'System.Threading.Tasks.ValueTask.CreateAsyncMethodBuilder()'
 does not exist in the implementation but it does exist in the contract. 

@stephentoub
Copy link
Member

stephentoub commented Jan 31, 2018

I'll fix it. My change hasn't mirrored to corefx yet, and I didn't actually know that these legs did ref validation.

@stephentoub
Copy link
Member

@dotnet-bot test Windows_NT x64 Checked corefx_baseline
@dotnet-bot test Ubuntu x64 Checked corefx_baseline

@stephentoub
Copy link
Member

@benaadams, have you run corefx tests with your change locally? Things are still out-of-sync, e.g.

error : MembersMustExist : Member 'System.Runtime.Intrinsics.X86.Avx.BroadcastElementToVector128(System.Single)' does not exist in the implementation but it does exist in the contract. 

and if you've verified this locally we can just get it merged.

@benaadams
Copy link
Member Author

have you run corefx tests with your change locally?

Yes and on here on coreclr CI prior to the two feedback tweaks

@stephentoub
Copy link
Member

Ok. Thanks!

@stephentoub stephentoub merged commit 1816802 into dotnet:master Jan 31, 2018
@benaadams benaadams deleted the executioncontext branch January 31, 2018 15:25
dotnet-bot pushed a commit to dotnet/corert that referenced this pull request Jan 31, 2018
* Reduce Save+Restore for ExecutionContext

* Use flag rather than comparison to static

* Skip null check for pre-checked EC.Run

* Feedback

* Add static helper lookup for default context for TP

* Add note for enregistering

* Return to Default context when no values

* Default + FlowSuppressed Context

* Move AsyncMethodBuilder.Start to static non-generic

* Feedback

Signed-off-by: dotnet-bot <[email protected]>
dotnet-bot pushed a commit to dotnet/corefx that referenced this pull request Jan 31, 2018
* Reduce Save+Restore for ExecutionContext

* Use flag rather than comparison to static

* Skip null check for pre-checked EC.Run

* Feedback

* Add static helper lookup for default context for TP

* Add note for enregistering

* Return to Default context when no values

* Default + FlowSuppressed Context

* Move AsyncMethodBuilder.Start to static non-generic

* Feedback

Signed-off-by: dotnet-bot-corefx-mirror <[email protected]>
stephentoub pushed a commit to dotnet/corefx that referenced this pull request Jan 31, 2018
* Reduce Save+Restore for ExecutionContext

* Use flag rather than comparison to static

* Skip null check for pre-checked EC.Run

* Feedback

* Add static helper lookup for default context for TP

* Add note for enregistering

* Return to Default context when no values

* Default + FlowSuppressed Context

* Move AsyncMethodBuilder.Start to static non-generic

* Feedback

Signed-off-by: dotnet-bot-corefx-mirror <[email protected]>
jkotas pushed a commit to dotnet/corert that referenced this pull request Feb 2, 2018
* Reduce Save+Restore for ExecutionContext

* Use flag rather than comparison to static

* Skip null check for pre-checked EC.Run

* Feedback

* Add static helper lookup for default context for TP

* Add note for enregistering

* Return to Default context when no values

* Default + FlowSuppressed Context

* Move AsyncMethodBuilder.Start to static non-generic

* Feedback

Signed-off-by: dotnet-bot <[email protected]>
jkotas pushed a commit to dotnet/corert that referenced this pull request Feb 2, 2018
* Reduce Save+Restore for ExecutionContext

* Use flag rather than comparison to static

* Skip null check for pre-checked EC.Run

* Feedback

* Add static helper lookup for default context for TP

* Add note for enregistering

* Return to Default context when no values

* Default + FlowSuppressed Context

* Move AsyncMethodBuilder.Start to static non-generic

* Feedback

Signed-off-by: dotnet-bot <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants