Component-specific data storage

[alnikola]

There is a need for the storage where different proxy components can store data specific to their internal logic.

At the very least this storage must provide the following capabilities:

1. Storing mutable and immutable data
2. Fast and thread safe retrieval of an entity by key efficiently operating in a highly concurrent environment
3. Thread safe operations to add, update and delete data entities

Optionally it should also provide:

1. Persistent storage preserving data between proxy process restarts
2. Data changes notifications

Examples of the storage use cases:

1. Destination affinity key lookup table for session affinity feature
2. Load balancing counters

See also #87

[Tratcher]

Design review: ClusterInfo.DynamicState is the current location for this class of data, but it needs to be extensible.

This property can be replaced with a ConcurrentDictionary to store arbitrary state. If change notification is needed, an implementer may choose to store a Signal in the dictionary.

[Tratcher]

We ended up using ConditionalWeakTable for this in TransportFailureRateHealthPolicy. It seems like a good compromise for adding state to entities you don't own.

[alnikola]

@Tratcher Should we close it then if there is no actual work left?

[Tratcher]

We should check what other state should be moved to this pattern first:

• ClusterLoadBalancingOptions.RoundRobinState
  

  reverse-proxy/src/ReverseProxy/Service/RuntimeModel/ClusterLoadBalancingOptions.cs

  Line 20 in 3344ec8

  internal AtomicCounter RoundRobinState { get; }

[samsp-msft]

We need to be able to associate the custom metadata with:

• Routes
• Clusters
• Individual destinations

For another example, see #587 where the need is to be able to associate metadata on the routes.