Integration tests tracing #1464

adriangonz · 2020-02-19T17:00:07Z

Fixes #977

Changelog

Update retrying to tenacity (retrying is now unmantained).
Add K8s configs to spin up Jaeger in integration tests.
Add helper methods to fetch traces from Jaeger's internal REST API.
Add simple test to assert traces get reported correctly when using the REST API.

adriangonz · 2020-02-19T17:00:21Z

/test integration

adriangonz · 2020-02-19T17:00:31Z

/cc @cliveseldon @axsaucedo

seldondev · 2020-02-19T17:01:20Z

Wed Feb 19 17:01:19 UTC 2020
The logs for [lint] [2] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1464/2.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1464 --build=2

seldondev · 2020-02-19T17:01:28Z

Wed Feb 19 17:01:27 UTC 2020
The logs for [pr-build] [1] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1464/1.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1464 --build=1

seldondev · 2020-02-19T17:01:34Z

Wed Feb 19 17:01:33 UTC 2020
The logs for [integration] [3] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1464/3.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1464 --build=3

adriangonz · 2020-02-20T14:40:25Z

/test integration

seldondev · 2020-02-20T14:41:35Z

Thu Feb 20 14:41:34 UTC 2020
The logs for [integration] [6] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1464/6.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1464 --build=6

seldondev · 2020-02-20T14:42:51Z

Thu Feb 20 14:42:50 UTC 2020
The logs for [pr-build] [4] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1464/4.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1464 --build=4

seldondev · 2020-02-20T14:42:59Z

Thu Feb 20 14:42:58 UTC 2020
The logs for [lint] [5] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1464/5.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1464 --build=5

adriangonz · 2020-02-20T15:48:11Z

/test integration

seldondev · 2020-02-20T15:49:19Z

Thu Feb 20 15:49:18 UTC 2020
The logs for [integration] [7] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1464/7.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1464 --build=7

adriangonz · 2020-02-24T13:20:23Z

/test integration

seldondev · 2020-02-24T13:21:19Z

Mon Feb 24 13:21:18 UTC 2020
The logs for [pr-build] [8] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1464/8.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1464 --build=8

seldondev · 2020-02-24T13:21:36Z

Mon Feb 24 13:21:35 UTC 2020
The logs for [lint] [9] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1464/9.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1464 --build=9

seldondev · 2020-02-24T13:22:10Z

Mon Feb 24 13:22:09 UTC 2020
The logs for [integration] [10] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1464/10.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1464 --build=10

adriangonz · 2020-02-24T14:55:30Z

/test integration

seldondev · 2020-02-24T14:56:52Z

Mon Feb 24 14:56:47 UTC 2020
The logs for [integration] [11] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1464/11.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1464 --build=11

adriangonz · 2020-02-25T11:58:25Z

/test integration

seldondev · 2020-02-25T11:59:23Z

Tue Feb 25 11:59:22 UTC 2020
The logs for [lint] [13] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1464/13.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1464 --build=13

seldondev · 2020-02-25T11:59:27Z

Tue Feb 25 11:59:25 UTC 2020
The logs for [pr-build] [12] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1464/12.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1464 --build=12

seldondev · 2020-02-25T11:59:45Z

Tue Feb 25 11:59:44 UTC 2020
The logs for [integration] [14] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1464/14.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1464 --build=14

adriangonz · 2020-02-25T13:26:18Z

/test integration

seldondev · 2020-02-25T13:27:33Z

Tue Feb 25 13:27:32 UTC 2020
The logs for [integration] [15] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1464/15.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1464 --build=15

axsaucedo

Looks good

axsaucedo · 2020-02-27T11:05:25Z

testing/resources/kind_config.yaml

+        hostPort: 8004
+kubeadmConfigPatches:
+  - |
+    apiVersion: kubelet.config.k8s.io/v1beta1


Not relevant to this PR specifically but just realised we should try to move away from v1beta1 as it was dropped from v1.16 onwards

axsaucedo · 2020-02-27T11:08:34Z

/approve
/lgtm

seldondev · 2020-02-27T11:08:43Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: axsaucedo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [axsaucedo]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Adrian Gonzalez-Martin added 16 commits February 17, 2020 16:13

Add target to install Jaeger and open port in Kind cluster

06695d8

Move configs to /resources

a9030f1

Add Jaeger config

2589098

Tweak config of jaeger

4fb079c

Remove port since Jaeger is exposed through ambassador

031d654

Update install_jaeger for Helm 3.x

f416129

Fix setup

3ace75f

Fix mapping

5d333e7

Get traces through internal REST API

946d540

Filter traces by pod name

9537ced

Add method to get pod names in a deployment

5d68f21

Assert traces by operation order (and referencing)

4f8c056

Add retry if traces are empty

c75fae7

Use tenacity instead of retrying (which is no longer maintained)

2b3c0f4

Fix spanID

0b7b9db

Add some comments on how we fetch traces

2429329

seldondev added the size/L label Feb 19, 2020

seldondev requested review from axsaucedo and ukclivecox February 19, 2020 17:00

Ensure spans are sorted before assertions

0b6eece

Increase waiting time

9df5b49

Always wait a couple seconds to sync with Jaeger

e71614c

axsaucedo reviewed Feb 27, 2020

View reviewed changes

seldondev assigned axsaucedo Feb 27, 2020

seldondev added the lgtm label Feb 27, 2020

seldondev added the approved label Feb 27, 2020

seldondev merged commit 98d9435 into SeldonIO:master Feb 27, 2020

adriangonz mentioned this pull request Feb 27, 2020

Reduce flakiness on tracing tests #1488

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integration tests tracing #1464

Integration tests tracing #1464

adriangonz commented Feb 19, 2020

adriangonz commented Feb 19, 2020

adriangonz commented Feb 19, 2020

seldondev commented Feb 19, 2020

seldondev commented Feb 19, 2020

seldondev commented Feb 19, 2020

adriangonz commented Feb 20, 2020

seldondev commented Feb 20, 2020

seldondev commented Feb 20, 2020

seldondev commented Feb 20, 2020

adriangonz commented Feb 20, 2020

seldondev commented Feb 20, 2020

adriangonz commented Feb 24, 2020

seldondev commented Feb 24, 2020

seldondev commented Feb 24, 2020

seldondev commented Feb 24, 2020

adriangonz commented Feb 24, 2020

seldondev commented Feb 24, 2020

adriangonz commented Feb 25, 2020

seldondev commented Feb 25, 2020

seldondev commented Feb 25, 2020

seldondev commented Feb 25, 2020

adriangonz commented Feb 25, 2020

seldondev commented Feb 25, 2020

axsaucedo left a comment

axsaucedo Feb 27, 2020

axsaucedo commented Feb 27, 2020

seldondev commented Feb 27, 2020

Integration tests tracing #1464

Integration tests tracing #1464

Conversation

adriangonz commented Feb 19, 2020

Changelog

adriangonz commented Feb 19, 2020

adriangonz commented Feb 19, 2020

seldondev commented Feb 19, 2020

seldondev commented Feb 19, 2020

seldondev commented Feb 19, 2020

adriangonz commented Feb 20, 2020

seldondev commented Feb 20, 2020

seldondev commented Feb 20, 2020

seldondev commented Feb 20, 2020

adriangonz commented Feb 20, 2020

seldondev commented Feb 20, 2020

adriangonz commented Feb 24, 2020

seldondev commented Feb 24, 2020

seldondev commented Feb 24, 2020

seldondev commented Feb 24, 2020

adriangonz commented Feb 24, 2020

seldondev commented Feb 24, 2020

adriangonz commented Feb 25, 2020

seldondev commented Feb 25, 2020

seldondev commented Feb 25, 2020

seldondev commented Feb 25, 2020

adriangonz commented Feb 25, 2020

seldondev commented Feb 25, 2020

axsaucedo left a comment

Choose a reason for hiding this comment

axsaucedo Feb 27, 2020

Choose a reason for hiding this comment

axsaucedo commented Feb 27, 2020

seldondev commented Feb 27, 2020