Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial benchmarking and load tests #2075

Merged
merged 9 commits into from
Jul 21, 2020
Merged

Initial benchmarking and load tests #2075

merged 9 commits into from
Jul 21, 2020

Conversation

ukclivecox
Copy link
Contributor

Fixes #1731

Use vegeta. Intial test provide a set of base tests we can expand to our components and are provided as basis for further work and discussion.

For tensorflow with flowers model (medium sized image classifier):

  • single worker throughput latency with and without executor to estimate the extra latency added by executor.
  • Max saturation of single server
  • HPA test for twice max saturation rate.

@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

Review Jupyter notebook visual diffs & provide feedback on notebooks.


Powered by ReviewNB

@ukclivecox ukclivecox requested a review from axsaucedo July 5, 2020 19:27
@ukclivecox ukclivecox requested a review from adriangonz July 5, 2020 19:27
@seldondev
Copy link
Collaborator

Sun Jul 5 19:28:52 UTC 2020
The logs for [pr-build] [1] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/1.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=1

@seldondev
Copy link
Collaborator

Sun Jul 5 19:28:54 UTC 2020
The logs for [lint] [2] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/2.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=2

@seldondev
Copy link
Collaborator

Tue Jul 7 17:25:52 UTC 2020
The logs for [lint] [4] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/4.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=4

@seldondev
Copy link
Collaborator

Tue Jul 7 17:25:54 UTC 2020
The logs for [pr-build] [3] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/3.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=3

@seldondev seldondev added size/XXL and removed size/XL labels Jul 9, 2020
@seldondev
Copy link
Collaborator

Thu Jul 9 17:13:58 UTC 2020
The logs for [lint] [6] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/6.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=6

@seldondev
Copy link
Collaborator

Thu Jul 9 17:14:03 UTC 2020
The logs for [pr-build] [5] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/5.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=5

@seldondev
Copy link
Collaborator

Thu Jul 9 17:34:30 UTC 2020
The logs for [lint] [8] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/8.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=8

@seldondev
Copy link
Collaborator

Thu Jul 9 17:34:32 UTC 2020
The logs for [pr-build] [7] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/7.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=7

@seldondev
Copy link
Collaborator

Thu Jul 9 18:06:43 UTC 2020
The logs for [lint] [10] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/10.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=10

@seldondev
Copy link
Collaborator

Thu Jul 9 18:07:32 UTC 2020
The logs for [pr-build] [9] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/9.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=9

@seldondev
Copy link
Collaborator

Sun Jul 12 18:11:52 UTC 2020
The logs for [pr-build] [11] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/11.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=11

@seldondev
Copy link
Collaborator

Sun Jul 12 18:12:05 UTC 2020
The logs for [lint] [12] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/12.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=12

@seldondev
Copy link
Collaborator

Sun Jul 12 19:15:44 UTC 2020
The logs for [lint] [14] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/14.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=14

@seldondev
Copy link
Collaborator

Sun Jul 12 19:15:55 UTC 2020
The logs for [pr-build] [13] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/13.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=13

@seldondev
Copy link
Collaborator

Tue Jul 14 07:08:20 UTC 2020
The logs for [pr-build] [15] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/15.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=15

@seldondev
Copy link
Collaborator

Tue Jul 14 07:08:22 UTC 2020
The logs for [lint] [16] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/16.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=16

@adriangonz
Copy link
Contributor

/test lint

@seldondev
Copy link
Collaborator

Thu Jul 16 17:25:27 UTC 2020
The logs for [lint] [17] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/17.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=17

@axsaucedo
Copy link
Contributor

/approve

@seldondev
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: axsaucedo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@seldondev
Copy link
Collaborator

Tue Jul 21 20:08:30 UTC 2020
The logs for [pr-build] [18] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/18.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=18

@seldondev
Copy link
Collaborator

Tue Jul 21 20:08:32 UTC 2020
The logs for [lint] [19] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2075/19.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2075 --build=19

@seldondev seldondev merged commit 9d7bc49 into SeldonIO:master Jul 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create a benchmarking framework
4 participants