Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unstable test "Open circuit should not effect different route" #1706

Closed
raman-m opened this issue Sep 25, 2023 · 5 comments · Fixed by #1753
Closed

Unstable test "Open circuit should not effect different route" #1706

raman-m opened this issue Sep 25, 2023 · 5 comments · Fixed by #1753
Assignees
Labels
bug Identified as a potential bug low Low priority merged Issue has been merged to dev and is waiting for the next release
Milestone

Comments

@raman-m
Copy link
Member

raman-m commented Sep 25, 2023

Expected Behavior / New Feature

The test Ocelot.AcceptanceTests.PollyQoSTests.Open_circuit_should_not_effect_different_route doesn't fail in 100% of running

Actual Behavior / Motivation for New Feature

Currently the test Open_circuit_should_not_effect_different_route fails sometimes, very rarely.
Possible reason: unknown.

Steps to Reproduce the Problem

Specifications

  • Version: latest, 19.0.3
  • Platform: any
  • Subsystem: CircleCI/Docker, Visual Studio
@raman-m raman-m added bug Identified as a potential bug accepted Bug or feature would be accepted as a PR or is being worked on labels Sep 25, 2023
@raman-m
Copy link
Member Author

raman-m commented Sep 25, 2023

@ggnaegi What about next coding challenge? 😉

@ggnaegi
Copy link
Member

ggnaegi commented Sep 26, 2023

@raman-m Well, it's very odd... At some point, we have a timing problem... It's very difficult to manage. First I thought, ok instead of using the local variable _requestCount, let's inject a singleton IRequestCountService like this:

.ConfigureServices(services =>
                {
                    services.AddSingleton<IRequestCountService, RequestCountService>();
                })

and then refactor the method like that:

private void GivenThereIsAPossiblyBrokenServiceRunningOn(string url, string responseBody)
        {
            _serviceHandler.GivenThereIsAServiceRunningOn(url, async context =>
            {
                var requestCountService = context.RequestServices.GetService<IRequestCountService>();
                context.Response.StatusCode = 200;

                if (requestCountService.GetRequestCount() == 1)
                {
                    await Task.Delay(1000);
                    await context.Response.WriteAsync("delayed service");
                    return;
                }

                await context.Response.WriteAsync(responseBody);
            });
        }

It brings a bit more stability, but that's not it yet, failing after 200-300 iterations I think

@raman-m
Copy link
Member Author

raman-m commented Oct 9, 2023

200-300 iterations

After what number of iterations we have a fail of the test now?

You are assigned! Okay?

@ggnaegi ggnaegi removed their assignment Oct 9, 2023
@ggnaegi
Copy link
Member

ggnaegi commented Oct 13, 2023

@raman-m I will have a go on this next week (wednesday probably)

@raman-m
Copy link
Member Author

raman-m commented Oct 30, 2023

@ggnaegicommented on Oct 13

Ah, never mind!
Assigned to you with low priority, because it is not critical for builds.

We will fix it in November if you'll have a time...
Also, I'm not going to include this to October's release...

raman-m added a commit that referenced this issue Nov 4, 2023
…g issue (#1753)

* When using the QoS option "ExceptionsAllowedBeforeBreaking" the circuit breaker never opens the circuit.

* merge issue, PortFinder

* some code improvements, using httpresponsemessage status codes as a base for circuit breaker

* Adding more unit tests, and trying to mitigate the test issues with the method "GivenThereIsAPossiblyBrokenServiceRunningOn"

* fixing some test issues

* setting timeout value to 5000 to avoid side effects

* again timing issues

* timing issues again

* ok, first one ok

* Revert "ok, first one ok"

This reverts commit 2e4a673.

* inline method

* putting back logging for http request exception

* removing logger configuration, back to default

* adding a bit more tests to check the policy wrap

* Removing TimeoutStrategy from parameters, it's set by default to pessimistic, at least one policy will be returned, so using First() in circuit breaker and removing the branch Policy == null from delegating handler.

* Fix StyleCop warnings

* Format parameters

* Sort usings

* since we might have two policies wrapped,  timeout and circuit breaker, we can't use the name CircuitBreaker for polly qos provider, it's not right. Using PollyPolicyWrapper and AsnycPollyPolicy instead.

* modifying circuit breaker delegating handler name, usin Polly policies instead

* renaming CircuitBreakerFactory to PolicyWrapperFactory in tests

* DRY for FileConfiguration, using FileConfigurationFactory

* Add copy constructor

* Refactor setup

* Use expression body for method

* Fix acceptance test

* IDE1006 Naming rule violation: These words must begin with upper case characters

* CA1816 Change ReturnsErrorTests.Dispose() to call GC.SuppressFinalize(object)

* Sort usings

* Use expression body for method

* Return back named arguments

---------

Co-authored-by: raman-m <[email protected]>
@raman-m raman-m added merged Issue has been merged to dev and is waiting for the next release and removed accepted Bug or feature would be accepted as a PR or is being worked on labels Nov 4, 2023
@raman-m raman-m unpinned this issue Nov 6, 2023
ibnuda pushed a commit to ibnuda/Ocelot that referenced this issue Nov 8, 2023
…ionsAllowedBeforeBreaking issue (ThreeMammals#1753)

* When using the QoS option "ExceptionsAllowedBeforeBreaking" the circuit breaker never opens the circuit.

* merge issue, PortFinder

* some code improvements, using httpresponsemessage status codes as a base for circuit breaker

* Adding more unit tests, and trying to mitigate the test issues with the method "GivenThereIsAPossiblyBrokenServiceRunningOn"

* fixing some test issues

* setting timeout value to 5000 to avoid side effects

* again timing issues

* timing issues again

* ok, first one ok

* Revert "ok, first one ok"

This reverts commit 2e4a673.

* inline method

* putting back logging for http request exception

* removing logger configuration, back to default

* adding a bit more tests to check the policy wrap

* Removing TimeoutStrategy from parameters, it's set by default to pessimistic, at least one policy will be returned, so using First() in circuit breaker and removing the branch Policy == null from delegating handler.

* Fix StyleCop warnings

* Format parameters

* Sort usings

* since we might have two policies wrapped,  timeout and circuit breaker, we can't use the name CircuitBreaker for polly qos provider, it's not right. Using PollyPolicyWrapper and AsnycPollyPolicy instead.

* modifying circuit breaker delegating handler name, usin Polly policies instead

* renaming CircuitBreakerFactory to PolicyWrapperFactory in tests

* DRY for FileConfiguration, using FileConfigurationFactory

* Add copy constructor

* Refactor setup

* Use expression body for method

* Fix acceptance test

* IDE1006 Naming rule violation: These words must begin with upper case characters

* CA1816 Change ReturnsErrorTests.Dispose() to call GC.SuppressFinalize(object)

* Sort usings

* Use expression body for method

* Return back named arguments

---------

Co-authored-by: raman-m <[email protected]>
@raman-m raman-m added the Oct'23 label Nov 10, 2023
@raman-m raman-m added this to the October'23 milestone Nov 18, 2023
raman-m added a commit that referenced this issue Nov 28, 2023
* #1712 Bump to Polly 8.0 (#1714)

* #1712 Migrate to Polly 8.0

* code review post merge

* post PR

* #1712 Migrate to Polly 8.0

* code review post merge

* Update src/Ocelot.Provider.Polly/PollyQoSProvider.cs

Co-authored-by: Raman Maksimchuk <[email protected]>

* namespaces

* Refactor QoS provider

* Refactor AddPolly extension

* Remove single quote because semicolon ends sentence

---------

Co-authored-by: Ray <[email protected]>
Co-authored-by: Raman Maksimchuk <[email protected]>

* Cache by header value: a new Header property in (File)CacheOptions configuration of a route (#1172)

@EngRajabi, Mohsen Rajabi (7):
      add header to file cache option
      fix private set
      fix
      <none>
      <none>
      fix build fail
      fix: fix review comment. add unit test for change

@raman-m, Raman Maksimchuk (1):
      Update caching.rst

@raman-m (7):
      Fix errors
      Fix errors
      Fix styling warnings
      Refactor tests
      Add Delimiter
      Refactor generator
      Add unit tests

* Find available port in integration tests (#1173)

* use random ports in integration tests

* Remove and Sort Usings

* Code modern look

* code review

* code review: fix messages

* code review: fix some messages

* code review: Use simple `using` statement

* Add Ocelot.Testing project

---------

Co-authored-by: raman-m <[email protected]>

* #952 #1174 Merge query strings without duplicate values (#1182)

* Fix issue  #952 and #1174

* Fix compiling errors

* Fix warnings

* Fix errors

* Remove and Sort Usings

* CA1845 Use span-based 'string.Concat' and 'AsSpan' instead of 'Substring'.
Use 'AsSpan' with 'string.Concat'

* IDE1006 Naming rule violation: These words must begin with upper case characters: {should_*}.
Fix name violation

* Add namespace

* Fix build errors

* Test class should match the name of tested class

* Simplify too long class names, and they should match

* Move to the parent folder which was empty

* Fix warnings

* Process dictionaries using LINQ to Objects approach

* Fix code review issues from @RaynaldM

* Remove tiny private helper with one reference

* Fix warning & messages

* Define theory instead of 2 facts

* Add unit test for issue #952

* Add additional unit test for #952 to keep param

* Add tests for issue #1174

* Remove unnecessary parameter

* Copy routing.rst from released version

* Refactor the middleware body for query params

* Update routing.rst: Describe query string user scenarios

---------

Co-authored-by: Stjepan Majdak <[email protected]>
Co-authored-by: raman-m <[email protected]>

* #1550 #1706 Addressing the QoS options ExceptionsAllowedBeforeBreaking issue (#1753)

* When using the QoS option "ExceptionsAllowedBeforeBreaking" the circuit breaker never opens the circuit.

* merge issue, PortFinder

* some code improvements, using httpresponsemessage status codes as a base for circuit breaker

* Adding more unit tests, and trying to mitigate the test issues with the method "GivenThereIsAPossiblyBrokenServiceRunningOn"

* fixing some test issues

* setting timeout value to 5000 to avoid side effects

* again timing issues

* timing issues again

* ok, first one ok

* Revert "ok, first one ok"

This reverts commit 2e4a673.

* inline method

* putting back logging for http request exception

* removing logger configuration, back to default

* adding a bit more tests to check the policy wrap

* Removing TimeoutStrategy from parameters, it's set by default to pessimistic, at least one policy will be returned, so using First() in circuit breaker and removing the branch Policy == null from delegating handler.

* Fix StyleCop warnings

* Format parameters

* Sort usings

* since we might have two policies wrapped,  timeout and circuit breaker, we can't use the name CircuitBreaker for polly qos provider, it's not right. Using PollyPolicyWrapper and AsnycPollyPolicy instead.

* modifying circuit breaker delegating handler name, usin Polly policies instead

* renaming CircuitBreakerFactory to PolicyWrapperFactory in tests

* DRY for FileConfiguration, using FileConfigurationFactory

* Add copy constructor

* Refactor setup

* Use expression body for method

* Fix acceptance test

* IDE1006 Naming rule violation: These words must begin with upper case characters

* CA1816 Change ReturnsErrorTests.Dispose() to call GC.SuppressFinalize(object)

* Sort usings

* Use expression body for method

* Return back named arguments

---------

Co-authored-by: raman-m <[email protected]>

* #1179 Add missing documentation for Secured WebSocket #1180

* Add "WebSocket Secure" and "SSL Errors" sections (#1180)

Co-authored-by: raman-m <[email protected]>

* Resolve issues with projects after auto-merging. Format Document

* #1744 Avoid calls to 'Logger.Log' if LogLevel not enabled in appsettings.json (#1745)

* changing string parameter for IOcelotLogger function to Func<string>, modifying asp dot net logger, only one main method and verifying if LogLevel is enabled. If log level isn't enabled, then return.

    pick 847dac7 changing string parameter for IOcelotLogger function to Func<string>, modifying asp dot net logger, only one main method and verifying if LogLevel is enabled. If log level isn't enabled, then return.
    pick d7a8397 adding back the logger methods with string as parameter, avoiding calling the factory when plain string are used.
    pick d413201 simplify method calls

* adding back the logger methods with string as parameter, avoiding calling the factory when plain string are used.

* simplify method calls

* adding unit test case, If minimum log level not set then no logs are written

* adding logging benchmark

* code cleanup in steps and naming issues fixes

   pick c4f6dc9 adding loglevel acceptance tests, verifying that the logs are returned according to the minimum log level set in appsettings
   pick 478f139 enhanced unit tests, verifying 1) that the log method is only called when log level enabled 2) that the string function is only invoked when log level enabled

* adding loglevel acceptance tests, verifying that the logs are returned according to the minimum log level set in appsettings

* enhanced unit tests, verifying 1) that the log method is only called when log level enabled 2) that the string function is only invoked when log level enabled

* weird issue with the merge.

* adding comment

* Update src/Ocelot/ServiceDiscovery/ServiceDiscoveryProviderFactory.cs

Co-authored-by: Raman Maksimchuk <[email protected]>

* Update src/Ocelot/Claims/Middleware/ClaimsToClaimsMiddleware.cs

Co-authored-by: Raman Maksimchuk <[email protected]>

* Update src/Ocelot/Configuration/Repository/FileConfigurationPoller.cs

Co-authored-by: Raman Maksimchuk <[email protected]>

* Update src/Ocelot/DownstreamRouteFinder/Finder/DownstreamRouteProviderFactory.cs

Co-authored-by: Raman Maksimchuk <[email protected]>

* Update src/Ocelot/Logging/AspDotNetLogger.cs

Co-authored-by: Raman Maksimchuk <[email protected]>

* Update test/Ocelot.AcceptanceTests/LogLevelTests.cs

Co-authored-by: Raman Maksimchuk <[email protected]>

* Update src/Ocelot/Configuration/Repository/FileConfigurationPoller.cs

Co-authored-by: Raman Maksimchuk <[email protected]>

* As mentioned, using OcelotLogger instead of AspDotNeLogger as default logger name

* Some code refactoring and usage of factories in LogLevelTests

* Update src/Ocelot/Claims/Middleware/ClaimsToClaimsMiddleware.cs

Co-authored-by: Raman Maksimchuk <[email protected]>

* using overrided method WriteLog for strings, some changes as requested,

* code changes after review 2

    pick ad0e060 Update test/Ocelot.UnitTests/Middleware/OcelotPiplineBuilderTests.cs

* checking test cases

* adding ms logger benchmarks with console provider. Unfortunately, benchmark.net doesn't support "quiet" mode yet.

* 2 small adjustments

* Adding multi targets support for serilog

* Fix warnings

* Review new logger

* Fix unit tests

* The last change but not least

* Update logging.rst: Add draft

* Update logging.rst: Add RequestId section

* Update logging.rst: "Best Practices" section

* Update logging.rst: "Top Logging Performance?" subsection

* Update logging.rst: Rewrite "Request ID" section

* Update requestid.rst: Review and up to date

* Update logging.rst: "Run Benchmarks" section

---------

Co-authored-by: Raman Maksimchuk <[email protected]>

* #1783 Less logs for circuit breakers (Polly exceptions) (#1786)

* #1783 More accurate logs for circuit breakers (and other "polly" exceptions)
Remove try/catch in PollyPoliciesDelegatingHandler and add a more generic AddPolly<T> to be able to use a specific PollyQoSProvider

* fix should_be_invalid_re_route_using_downstream_http_version UT

* fix remarks on PR

* arrange code

* fix UT

* merge with release/net8 branch

* switch benchmark to Net8

* Fix warnings

* Final review

---------

Co-authored-by: Ray <[email protected]>
Co-authored-by: raman-m <[email protected]>

* Revert #1172 feature (#1807)

* Revert #1172

* Remove Header

* Take actual version of caching.rst and remove Header info

* Release 22.0 | +semver: breaking

---------

Co-authored-by: Raynald Messié <[email protected]>
Co-authored-by: Ray <[email protected]>
Co-authored-by: Mohsen Rajabi <[email protected]>
Co-authored-by: jlukawska <[email protected]>
Co-authored-by: Stjepan <[email protected]>
Co-authored-by: Stjepan Majdak <[email protected]>
Co-authored-by: Guillaume Gnaegi <[email protected]>
Co-authored-by: Samuel Poirier <[email protected]>
raman-m added a commit to ibnuda/Ocelot that referenced this issue Feb 20, 2024
…ionsAllowedBeforeBreaking issue (ThreeMammals#1753)

* When using the QoS option "ExceptionsAllowedBeforeBreaking" the circuit breaker never opens the circuit.

* merge issue, PortFinder

* some code improvements, using httpresponsemessage status codes as a base for circuit breaker

* Adding more unit tests, and trying to mitigate the test issues with the method "GivenThereIsAPossiblyBrokenServiceRunningOn"

* fixing some test issues

* setting timeout value to 5000 to avoid side effects

* again timing issues

* timing issues again

* ok, first one ok

* Revert "ok, first one ok"

This reverts commit 2e4a673.

* inline method

* putting back logging for http request exception

* removing logger configuration, back to default

* adding a bit more tests to check the policy wrap

* Removing TimeoutStrategy from parameters, it's set by default to pessimistic, at least one policy will be returned, so using First() in circuit breaker and removing the branch Policy == null from delegating handler.

* Fix StyleCop warnings

* Format parameters

* Sort usings

* since we might have two policies wrapped,  timeout and circuit breaker, we can't use the name CircuitBreaker for polly qos provider, it's not right. Using PollyPolicyWrapper and AsnycPollyPolicy instead.

* modifying circuit breaker delegating handler name, usin Polly policies instead

* renaming CircuitBreakerFactory to PolicyWrapperFactory in tests

* DRY for FileConfiguration, using FileConfigurationFactory

* Add copy constructor

* Refactor setup

* Use expression body for method

* Fix acceptance test

* IDE1006 Naming rule violation: These words must begin with upper case characters

* CA1816 Change ReturnsErrorTests.Dispose() to call GC.SuppressFinalize(object)

* Sort usings

* Use expression body for method

* Return back named arguments

---------

Co-authored-by: raman-m <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Identified as a potential bug low Low priority merged Issue has been merged to dev and is waiting for the next release
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants