[POC] local/global rate limiting

[strekm]

Description

In scope of this POC is documentation of examples on how to configure Istio EnvoyFilters to configure rate limit. Documented examples later on will be used to design API of new CRD. CRD will be part of Istio module. It should be possible to limit workload access by:

• custom header
• client IP
• client cert

Have in mind that rate limiting should be possible on more than 1 condition. Take into consideration that workload can be exposed and secured with API Gateway APIRule CR or directly using Istio resources. Workload might be secured with external authorizer.

Focus of this POC is local rate limiting but have in mind that in the future global rate limiting also should be possible.

Update on 30.04.
After initial tests for local rate limiting, we realised that capabilities of local rate limiting are very limited. Local rate limiting is limited to very static descriptors, which makes it impossible to support use cases such as rate limiting based on client IP in a good way. As we have not yet spent much time on this task, it was decided to also look at global rate limiting and compare global and local rate limiting.

ACs:

• it is possible to limit access to workload by caller IP, cert or customer header
• it is possible to limit access by more than 1 condition
• existing functionalities (APIRule, external authorizer or Istio equivalent) are not affected
• perf tests executed

DoD:
- [ ] Provide unit and integration tests.

• Provide documentation.
• Verify if the solution works for both open-source Kyma and SAP BTP, Kyma runtime.
  - [ ] If you changed the resource limits, explain why it was needed.
  - [ ] If the default configuration of Istio Operator has been changed, you performed a manual upgrade test to verify that the change can be rolled out correctly.
  - [ ] Verify that your contributions don't decrease code coverage. If they do, explain why this is the case.
  - [ ] Add release notes.

Attachments
part of: kyma-project/api-gateway#1365

[triffer]

PoC documentation can be found in rate-limit-poc branch.

The documentation consist of

• General comparsion between local and global rate limiting and performance test results
• Local Rate Limiting documentation with scenarios and limitations
• Global Rate Limiting documentation with scenarios