-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
A cheaper linked cancellation token source? #40670
Comments
I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label. |
CancellationToken just references a CancellationTokenSource. Any change that required a CT to be able to refer multiple CTS instances or even multiple value types would bloat the size of a CT non-trivially, and it's stored on tons of types (think of all the async methods that take one as an argument, whch causes it to be on the state machine type), and would thus make all of them larger, and I expect be a significant net negative. If you have a suggestion/prototype for doing this in a way that's pay-for-play, please share. Otherwise, I don't think this is actionable. |
Tagging subscribers to this area: @tarekgh |
@stephentoub, could we reuse One thing is certain: linked |
You can pool your own CTS instances if you want: if you control their cancellation, then you can return them to the pool if they haven't been canceled, and presumably that's the 99% case. For resetting the instance, that has lots of problems, stemming from the fact that there are strong guarantees that once a CT has transitioned to canceled, it must not transition back. |
@stephentoub, thanks man. A question for you: How do I pool these Second token in the example above is the |
CreateLinkedTokenSource is really just a helper that creates a new CTS and does the logical equivalent of CT.Register(CTS.Cancel) on each token. You can do the same, e.g. CancellationTokenSource cts = GetFromPoolOrAllocate();
using (ct1.UnsafeRegister(s => ((CancellationTokenSource)s).Cancel(), cts))
using (ct2.UnsafeRegister(s => ((CancellationTokenSource)s).Cancel(), cts))
await FooAsync(cts.Token);
if (!cts.IsCancellationRequested)
ReturnToPool(cts); |
@stephentoub. Awesome. Giving a try now. |
@stephentoub , I ended up using something like the code below. Perhaps we could have a similar concept in .NET? public static class CancellationScope
{
private static readonly Action<object?> CancelAction = s => ((CancellationTokenSource)s!).Cancel();
private static readonly ConcurrentStack<CancellationTokenSource> Stack;
static CancellationScope()
{
var count = Math.Min(Environment.ProcessorCount * 16, 256);
var sources = Enumerable.Range(0, count).Select(_ => new CancellationTokenSource());
Stack = new ConcurrentStack<CancellationTokenSource>(sources);
}
public static async ValueTask<TResult> ExecuteAsync<TState, TResult>(
CancellationToken token1,
CancellationToken token2,
Func<TState, CancellationToken, ValueTask<TResult>> action,
TState state)
{
if (Stack.TryPop(out var source))
{
try
{
using (token1.Register(CancelAction, source, false))
using (token2.Register(CancelAction, source, false))
return await action(state, source.Token);
}
finally
{
if (source.IsCancellationRequested)
{
source.Dispose();
source = new CancellationTokenSource();
}
Stack.Push(source);
}
}
else
{
using (source = CancellationTokenSource.CreateLinkedTokenSource(token1, token2))
return await action(state, source.Token);
}
} |
Have you measured it and seen that to actually be measurably better? Returning to ConcurrentStack allocates, and if your helper ends up yielding, it'll allocate the state associated with the async frame. On top of that your code is allocating two delegates when passing CancelAction to Register. I don't see us adding this to the core libraries. Obviously feel free to publish your own helpers on nuget for others to use of they find it helpful. |
@stephentoub, the delegate is a
Just to be exact, here is a simple benchmark: [SimpleJob(RuntimeMoniker.NetCoreApp31)]
[MemoryDiagnoser]
public class DelegateBenchmark
{
private static readonly Action<int> FooDelegate = i => Foo(i);
[Benchmark]
public void WithStaticDelegateInstance()
{
for (var i = 0; i < 1000; i++)
Bar(FooDelegate, i);
}
[Benchmark]
public void WithNewDelegateInstance()
{
for (var i = 0; i < 1000; i++)
Bar(Foo, i);
}
private static void Foo(int i)
{
}
private static void Bar(Action<int> action, int i)
=> action(i);
}
|
I must have misread; I thought CancelAction was a method rather than a delegate field. |
@stephentoub , here is the benchmark. I think a
Benchmark[SimpleJob(RuntimeMoniker.NetCoreApp31)]
[MemoryDiagnoser]
public class CancellationScopeBenchmark
{
private static readonly CancellationTokenSource Source1 = new CancellationTokenSource();
private static readonly CancellationTokenSource Source2 = new CancellationTokenSource();
private static readonly Action<CancellationToken> ActionDelegate = ct => ct.ThrowIfCancellationRequested();
[Benchmark]
public void NewLinkedCancellationTokenSource()
{
for (var i = 0; i < 10000; i++)
{
using var source = CancellationTokenSource.CreateLinkedTokenSource(Source1.Token, Source2.Token);
ActionDelegate(source.Token);
}
}
[Benchmark]
public void Stack()
{
for (var i = 0; i < 10000; i++)
CancellationScopeStack.Execute(Source1.Token, Source2.Token, ActionDelegate);
}
[Benchmark]
public void Bag()
{
for (var i = 0; i < 10000; i++)
CancellationScopeBag.Execute(Source1.Token, Source2.Token, ActionDelegate);
}
[Benchmark(Baseline = true)]
public void CompareAndExchange()
{
for (var i = 0; i < 10000; i++)
CancellationScopeCompareAndExchange.Execute(Source1.Token, Source2.Token, ActionDelegate);
}
} Bag implementationinternal static class CancellationScopeBag
{
private static readonly Action<object?> CancelAction = s => ((CancellationTokenSource)s!).Cancel();
private static readonly ConcurrentBag<CancellationTokenSource> Items;
static CancellationScopeBag()
{
var count = Math.Min(Environment.ProcessorCount * 16, 256);
var sources = Enumerable.Range(0, count).Select(_ => new CancellationTokenSource());
Items = new ConcurrentBag<CancellationTokenSource>(sources);
}
internal static void Execute(
CancellationToken token1,
CancellationToken token2,
Action<CancellationToken> action)
{
if (Items.TryTake(out var source))
{
try
{
using (token1.Register(CancelAction, source, false))
using (token2.Register(CancelAction, source, false))
action(source.Token);
}
finally
{
if (source.IsCancellationRequested)
{
source.Dispose();
source = new CancellationTokenSource();
}
Items.Add(source);
}
}
else
{
using (source = CancellationTokenSource.CreateLinkedTokenSource(token1, token2))
action(source.Token);
}
}
} Compare and exchange implementationinternal static class CancellationScopeCompareAndExchange
{
private static readonly Action<object?> CancelAction = s => ((CancellationTokenSource)s!).Cancel();
private static readonly int Count = Math.Min(Environment.ProcessorCount * 16, 256);
private static readonly CancellationTokenSource?[] Items;
static CancellationScopeCompareAndExchange()
{
Items = new CancellationTokenSource[Count];
for (var i = 0; i < Count; i++)
Items[i] = new CancellationTokenSource();
}
internal static void Execute(
CancellationToken token1,
CancellationToken token2,
Action<CancellationToken> action)
{
var source = RentSource();
if (source != null)
{
try
{
using (token1.Register(CancelAction, source, false))
using (token2.Register(CancelAction, source, false))
action(source.Token);
}
finally
{
if (source.IsCancellationRequested)
{
source.Dispose();
source = new CancellationTokenSource();
}
ReturnSource(source);
}
}
else
{
using (source = CancellationTokenSource.CreateLinkedTokenSource(token1, token2))
action(source.Token);
}
}
private static CancellationTokenSource? RentSource()
{
for (var i = 0; i < Count; i++)
{
while (true)
{
var snapshot = Items[i];
if (snapshot is null)
break;
if (Interlocked.CompareExchange(ref Items[i], null, snapshot) == snapshot)
return snapshot;
}
}
return null;
}
private static void ReturnSource(CancellationTokenSource source)
{
var i = 0;
while (true)
{
if (Interlocked.CompareExchange(ref Items[i], source, null) == null)
return;
i = (i + 1) % Count;
}
}
} @stephentoub , are you sure we should not have something like this in the core lib? |
I do not think it's valuable enough to add. |
A few other things to keep in mind about your example. A real-world case will have concurrency and will introduce contention as part of the pooling. Also, cancellation tokens are used much more in async code, so your Execute method would more likely be ExecuteAsync, and introduce another async state machine as part of any operation that yielded. |
@stephentoub, as always, I am grateful to you. Here is a version without the async state machine: Usageusing (var linked = new LinkedCancellationToken(token1, token2))
await Foo(linked.Token); Implementationpublic readonly struct LinkedCancellationToken : IEquatable<LinkedCancellationToken>, IDisposable
{
private static readonly int Count = Math.Min(Environment.ProcessorCount * 16, 256);
private static readonly Action<object?> CancelAction = s => ((CancellationTokenSource)s!).Cancel();
private static readonly CancellationTokenSource?[] Items;
private readonly CancellationTokenSource? source;
private readonly CancellationTokenRegistration reg1;
private readonly CancellationTokenRegistration reg2;
static LinkedCancellationToken()
{
Items = new CancellationTokenSource[Count];
for (var i = 0; i < Count; i++)
Items[i] = new CancellationTokenSource();
}
public LinkedCancellationToken(CancellationToken token1, CancellationToken token2)
{
var source = Rent();
if (source is null)
{
source = CancellationTokenSource.CreateLinkedTokenSource(token1, token2);
reg1 = reg2 = default;
}
else
{
reg1 = token1.Register(CancelAction, source, false);
reg2 = token2.Register(CancelAction, source, false);
}
this.source = source;
}
public CancellationToken Token
=> source?.Token ?? default;
public static bool operator ==(LinkedCancellationToken left, LinkedCancellationToken right)
=> left.Equals(right);
public static bool operator !=(LinkedCancellationToken left, LinkedCancellationToken right)
=> !left.Equals(right);
public override bool Equals(object? obj)
=> obj is LinkedCancellationToken token && Equals(token);
public bool Equals(LinkedCancellationToken other)
=> ReferenceEquals(source, other.source);
public override int GetHashCode()
=> source?.GetHashCode() ?? 0;
public void Dispose()
{
if (source is null)
return;
if (reg1 == default)
{
source.Dispose();
}
else
{
reg1.Dispose();
reg2.Dispose();
if (source.IsCancellationRequested)
{
source.Dispose();
Return(new CancellationTokenSource());
}
else
{
Return(source);
}
}
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
private static CancellationTokenSource? Rent()
{
for (var i = 0; i < Count; i++)
{
while (true)
{
var snapshot = Items[i];
if (snapshot is null)
break;
if (Interlocked.CompareExchange(ref Items[i], null, snapshot) == snapshot)
return snapshot;
}
}
return null;
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
private static void Return(CancellationTokenSource source)
{
var i = 0;
while (true)
{
if (Items[i] is null)
{
if (Interlocked.CompareExchange(ref Items[i], source, null) == null)
return;
}
else
{
if (ReferenceEquals(source, Items[i]))
return;
}
i = (i + 1) % Count;
}
}
} |
You're welcome. :) |
Often, we need to create linked cancellation tokens in the context of an operation. Unfortunatley,
CancellationTokenSource.CreateLinkedTokenSource
requires heap allocation that increases pressure on GC. Can we have a linked cancellation token source that is a value type?This is what I often need to do:
It would be nice if we could do something like this where
LinkedCancellationTokenSource
was a value type with no internal heap allocations:@stephentoub and @ericstj as FYI.
The text was updated successfully, but these errors were encountered: