Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add ActiveDeadlineSeconds and BackoffLimit features #963

Merged
merged 12 commits into from
Mar 26, 2019
Merged

add ActiveDeadlineSeconds and BackoffLimit features #963

merged 12 commits into from
Mar 26, 2019

Conversation

ChanYiLin
Copy link
Member

@ChanYiLin ChanYiLin commented Mar 21, 2019 •
edited by jlewi
Loading

This PR add both ActiveDeadlineSeconds and BackoffLimit features.

For ActiveDeadlineSeconds, once a Job reaches activeDeadlineSeconds, all of its Pods are terminated and the Job status will become type: Failed.

For BackoffLimit, user can set BackoffLimit to specify the number of retries of replicas before considering a Job as failed.

Fixes #550

This change is Reviewable

@k8s-ci-robot k8s-ci-robot requested review from jimexist and yph152 March 21, 2019 13:14
@TravisBuddy
Copy link

Travis tests have failed

Hey @ChanYiLin,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
pkg/controller.v1beta2/tensorflow/job_test.go:1::warning: file is not goimported (goimports)
pkg/controller.v1beta2/tensorflow/job_test.go:678:3:warning: unused variable or constant undeclared name: forget (varcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:682:7:warning: unused variable or constant undeclared name: forget (varcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:683:85:warning: unused variable or constant undeclared name: forget (varcheck)
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.055s
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.052s
=== RUN   TestValidateBetaOneTFJobSpec
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 2.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.049s
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 1.0% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.047s
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 4.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.071s
=== RUN   TestNormalPath
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
--- PASS: TestDeletePodsAndServices (0.00s)
=== RUN   TestCleanupTFJob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestAddPod
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 29.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.910s
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow [build failed]
exit status 2: warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/version
# github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow
pkg/controller.v1beta2/tensorflow/job_test.go:678:3: undefined: forget
pkg/controller.v1beta2/tensorflow/job_test.go:682:7: undefined: forget
pkg/controller.v1beta2/tensorflow/job_test.go:683:85: undefined: forget

2nd Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
pkg/controller.v1beta2/tensorflow/job_test.go:678:3:error: undefined: forget (vet)
pkg/controller.v1beta2/tensorflow/job_test.go:683:85:error: undefined: forget (vet)
pkg/controller.v1beta2/tensorflow/job_test.go:683:85:warning: undeclared name: forget (compile) (staticcheck, staticcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:678:3:warning: unused variable or constant undeclared name: forget (varcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:682:7:warning: unused variable or constant undeclared name: forget (varcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:682:7:error: undefined: forget (vet)
pkg/controller.v1beta2/tensorflow/job_test.go:678:3:warning: undeclared name: forget (compile) (staticcheck, staticcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:682:7:warning: undeclared name: forget (compile) (staticcheck, staticcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:1::warning: file is not goimported (goimports)
pkg/controller.v1beta2/tensorflow/job_test.go:683:85:warning: unused variable or constant undeclared name: forget (varcheck)
travis_time:end:0a939538:start=1553174299365384582,finish=1553174478978567170,duration=179613182588
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.033s	coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.030s	coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestValidateBetaOneTFJobSpec
time="2019-03-21T13:21:35Z" level=warning msg="Image is undefined in the container"
time="2019-03-21T13:21:35Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
time="2019-03-21T13:21:35Z" level=warning msg="Image is undefined in the container"
time="2019-03-21T13:21:35Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.029s	coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.023s	coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
time="2019-03-21T13:21:52Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
time="2019-03-21T13:21:52Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.063s	coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestNormalPath
time="2019-03-21T13:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:21:59Z" level=info msg="Creating Job controller"
time="2019-03-21T13:21:59Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:21:59Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.656293ms)" job=default.test-tfjob
time="2019-03-21T13:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:21:59Z" level=info msg="Creating Job controller"
time="2019-03-21T13:21:59Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T13:21:59Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T13:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T13:21:59Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T13:21:59Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:21:59Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:21:59Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:21:59Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:21:59Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:21:59Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:21:59Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:21:59Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.028346ms)" job=default.test-tfjob
time="2019-03-21T13:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:21:59Z" level=info msg="Creating Job controller"
time="2019-03-21T13:21:59Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (376.598µs)" job=default.test-tfjob
time="2019-03-21T13:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:21:59Z" level=info msg="Creating Job controller"
time="2019-03-21T13:21:59Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T13:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T13:21:59Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:21:59Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:21:59Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:21:59Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (631.673µs)" job=default.test-tfjob
time="2019-03-21T13:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:21:59Z" level=info msg="Creating Job controller"
time="2019-03-21T13:21:59Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T13:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T13:21:59Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:21:59Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (506.317µs)" job=default.test-tfjob
time="2019-03-21T13:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:21:59Z" level=info msg="Creating Job controller"
time="2019-03-21T13:21:59Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (417.619µs)" job=default.test-tfjob
time="2019-03-21T13:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:21:59Z" level=info msg="Creating Job controller"
time="2019-03-21T13:21:59Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T13:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T13:21:59Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:21:59Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.014871ms)" job=default.test-tfjob
time="2019-03-21T13:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:21:59Z" level=info msg="Creating Job controller"
time="2019-03-21T13:21:59Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0321 13:21:59.682824    9794 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc4207c36f0), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"PS":(*v1beta1.ReplicaSpec)(0xc4202a4dc0), "Worker":(*v1beta1.ReplicaSpec)(0xc4202a5340)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"PS":(*v1beta1.ReplicaStatus)(0xc4205cc920), "Worker":(*v1beta1.ReplicaStatus)(0xc4205cc9b0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobSucceeded' 'TFJob test-tfjob successfully completed.'
time="2019-03-21T13:21:59Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (855.916µs)" job=default.test-tfjob
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
time="2019-03-21T13:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:21:59Z" level=info msg="Creating Job controller"
time="2019-03-21T13:21:59Z" level=info msg="Starting TFJob controller"
time="2019-03-21T13:21:59Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T13:21:59Z" level=info msg="Starting 1 workers"
time="2019-03-21T13:21:59Z" level=info msg="Started workers"
time="2019-03-21T13:22:00Z" level=info msg="Shutting down workers"
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
time="2019-03-21T13:22:00Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:00Z" level=info msg="Creating Job controller"
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
time="2019-03-21T13:22:00Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:00Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:00Z" level=info msg="Starting TFJob controller"
time="2019-03-21T13:22:00Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T13:22:00Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-21T13:22:00Z" level=info msg="Starting 1 workers"
time="2019-03-21T13:22:00Z" level=info msg="Started workers"
--- PASS: TestAddTFJob (0.10s)
time="2019-03-21T13:22:00Z" level=info msg="Shutting down workers"
=== RUN   TestCopyLabelsAndAnnotation
time="2019-03-21T13:22:00Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:00Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:00Z" level=info msg="Starting TFJob controller"
time="2019-03-21T13:22:00Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T13:22:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T13:22:00Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:22:00Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:00Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:22:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (344.175µs)" job=default.test-tfjob
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
time="2019-03-21T13:22:00Z" level=info msg="Starting 1 workers"
time="2019-03-21T13:22:00Z" level=info msg="Started workers"
time="2019-03-21T13:22:00Z" level=info msg="Shutting down workers"
time="2019-03-21T13:22:00Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:00Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T13:22:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (251.714µs)" job=default.test-tfjob
time="2019-03-21T13:22:00Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:00Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T13:22:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (253.287µs)" job=default.test-tfjob
time="2019-03-21T13:22:00Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:00Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T13:22:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (225.496µs)" job=default.test-tfjob
time="2019-03-21T13:22:00Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:00Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T13:22:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (504.795µs)" job=default.test-tfjob
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
time="2019-03-21T13:22:00Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:00Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T13:22:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (315.608µs)" job=default.test-tfjob
time="2019-03-21T13:22:00Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:00Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T13:22:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (228.742µs)" job=default.test-tfjob
time="2019-03-21T13:22:00Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:00Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (375.516µs)" job=default.test-tfjob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestAddPod
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="Starting TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T13:22:02Z" level=info msg="Starting 1 workers"
time="2019-03-21T13:22:02Z" level=info msg="Started workers"
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
time="2019-03-21T13:22:02Z" level=info msg="Shutting down workers"
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="Starting TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T13:22:02Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0321 13:22:02.404679    9794 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc4203c56b0), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc4203d7340)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc420044350)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-21T13:22:02Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0321 13:22:02.404909    9794 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc4203c56b0), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc4203d7340)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc420044350)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'TFJobRestarting' 'TFJob test-tfjob is restarting because 1 Worker replica(s) failed.'
time="2019-03-21T13:22:02Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T13:22:02Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (578.128µs)" job=default.test-tfjob
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
time="2019-03-21T13:22:02Z" level=info msg="Starting 1 workers"
time="2019-03-21T13:22:02Z" level=info msg="Started workers"
time="2019-03-21T13:22:02Z" level=info msg="Shutting down workers"
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="Starting TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T13:22:02Z" level=info msg="Starting 1 workers"
time="2019-03-21T13:22:02Z" level=info msg="Started workers"
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
time="2019-03-21T13:22:02Z" level=info msg="Shutting down workers"
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0321 13:22:02.508113    9794 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc4205b62c0)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4204bccd0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="Creating TFJob controller"
time="2019-03-21T13:22:02Z" level=info msg="Creating Job controller"
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T13:22:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
--- PASS: TestStatus (0.02s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.908s	coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow [build failed]
exit status 2: warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/version
# github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow
pkg/controller.v1beta2/tensorflow/job_test.go:678:3: undefined: forget
pkg/controller.v1beta2/tensorflow/job_test.go:682:7: undefined: forget
pkg/controller.v1beta2/tensorflow/job_test.go:683:85: undefined: forget

travis_time:end:11ec99a3:start=1553174478988210765,finish=1553174524246065467,duration=45257854702
TravisBuddy Request Identifier: a60534a0-4bdf-11e9-9767-c9e36d0d3911

@TravisBuddy
Copy link

Travis tests have failed

Hey @ChanYiLin,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.046s
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.046s
=== RUN   TestValidateBetaOneTFJobSpec
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 2.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.047s
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 1.0% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.046s
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 4.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.067s
=== RUN   TestNormalPath
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
--- PASS: TestDeletePodsAndServices (0.00s)
=== RUN   TestCleanupTFJob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestAddPod
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 29.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.908s
=== RUN   TestNormalPath
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
--- PASS: TestDeletePodsAndServices (0.00s)
=== RUN   TestCleanupTFJob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestActiveDeadlineSeconds
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/version
time="2019-03-21T14:43:37Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:37Z" level=info msg="Creating Job controller"
time="2019-03-21T14:43:37Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:43:37Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:43:37Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:37Z" level=inceeded, deletion time <nil>"
time="2019-03-21T14:43:37Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:43:37Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:37Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:43:37Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:43:37Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:37Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:43:37Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (452.076µs)" job=default.test-tfjob
time="2019-03-21T14:43:37Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:37Z" level=info msg="Creating Job controller"
time="2019-03-21T14:43:37Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:43:37Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:43:37Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:43:37Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:37Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:43:37Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:43:37Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:43:37Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:43:37Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:43:37Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:43:37Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:37Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:43:37Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:43:37Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:43:37Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:43:37Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (585.56µs)" job=default.test-tfjob
time="2019-03-21T14:43:37Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:37Z" level=info msg="Creating Job controller"
time="2019-03-21T14:43:37Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:43:37Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:37Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:37Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (315.46µs)" job=default.test-tfjob
time="2019-03-21T14:43:37Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:37Z" level=info msg="Creating Job controller"
time="2019-03-21T14:43:37Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:43:37Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:43:37Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:43:37Z" level=info msg="Started workers"
time="2019-03-21T14:43:37Z" level=info msg="Shutting down workers"
time="2019-03-21T14:43:37Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:37Z" level=info msg="Creating Job controller"
time="2019-03-21T14:43:37Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:37Z" level=info msg="Creating Job controller"
time="2019-03-21T14:43:37Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:43:37Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:43:37Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-21T14:43:37Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:43:37Z" level=info msg="Started workers"
time="2019-03-21T14:43:37Z" level=info
time="2019-03-21T14:43:41Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:43:41Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (444.776µs)" job=default.test-tfjob
time="2019-03-21T14:43:41Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:43:41Z" level=info msg="Started workers"
time="2019-03-21T14:43:41Z" level=info msg="Shutting down workers"
time="2019-03-21T14:43:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:43:41Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:43:41Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:43:41Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:43:41Z" level=info msg="Started workers"
time="2019-03-21T14:43:41Z" level=info msg="Shutting down workers"
time="2019-03-21T14:43:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:43:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0321 14:43:41.994596    6001 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc42088a580)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc420341db0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
time="2019-03-21T14:43:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:43:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:43:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:43:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T14:43:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:43:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T14:43:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:43:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:43:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:43:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:43:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-21T14:43:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:43:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-21T14:43:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:42Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:42Z" level=info msg="Creating Job controller"
time="2019-03-21T14:43:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:43:42Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:43:42Z" level=info msg="Creating Job controller"

2nd Build

View build log

goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.031s	coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.029s	coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestValidateBetaOneTFJobSpec
time="2019-03-21T14:19:41Z" level=warning msg="Image is undefined in the container"
time="2019-03-21T14:19:41Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
time="2019-03-21T14:19:41Z" level=warning msg="Image is undefined in the container"
time="2019-03-21T14:19:41Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.030s	coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.023s	coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
time="2019-03-21T14:20:00Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
time="2019-03-21T14:20:00Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.060s	coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestNormalPath
time="2019-03-21T14:20:07Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:07Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:07Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.561825ms)" job=default.test-tfjob
time="2019-03-21T14:20:07Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:07Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:07Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:07Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:07Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:07Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:07Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:07Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:07Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (627.623µs)" job=default.test-tfjob
time="2019-03-21T14:20:07Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:07Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:07Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:07Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:07Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:07Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:07Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (420.818µs)" job=default.test-tfjob
time="2019-03-21T14:20:07Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:07Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:07Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (376.095µs)" job=default.test-tfjob
time="2019-03-21T14:20:07Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:07Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:07Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:07Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:07Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:07Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:07Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (554.102µs)" job=default.test-tfjob
time="2019-03-21T14:20:07Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:07Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:07Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0321 14:20:07.904313    9778 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc42005b6d0), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc42044fb80), "PS":(*v1beta1.ReplicaSpec)(0xc420754000)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc420632cb0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobSucceeded' 'TFJob test-tfjob successfully completed.'
time="2019-03-21T14:20:07Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (3.703834ms)" job=default.test-tfjob
time="2019-03-21T14:20:07Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:07Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:07Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:07Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:07Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (286.016µs)" job=default.test-tfjob
time="2019-03-21T14:20:07Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:07Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:07Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:07Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:07Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:07Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:07Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:07Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:07Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:07Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:07Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:07Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:07Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:07Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:07Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:07Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (613.162µs)" job=default.test-tfjob
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
time="2019-03-21T14:20:07Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:07Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:07Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:20:07Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:20:08Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:20:08Z" level=info msg="Started workers"
time="2019-03-21T14:20:08Z" level=info msg="Shutting down workers"
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
time="2019-03-21T14:20:08Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:08Z" level=info msg="Creating Job controller"
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
time="2019-03-21T14:20:08Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:08Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:08Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-21T14:20:08Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:20:08Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:20:08Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:20:08Z" level=info msg="Started workers"
--- PASS: TestAddTFJob (0.10s)
time="2019-03-21T14:20:08Z" level=info msg="Shutting down workers"
=== RUN   TestCopyLabelsAndAnnotation
time="2019-03-21T14:20:08Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:08Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:08Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:08Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:08Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:08Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:08Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (349.959µs)" job=default.test-tfjob
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
time="2019-03-21T14:20:08Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:08Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:08Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:08Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (249.047µs)" job=default.test-tfjob
time="2019-03-21T14:20:08Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:08Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:08Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:08Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (367.338µs)" job=default.test-tfjob
time="2019-03-21T14:20:08Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:08Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:08Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:08Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (245.304µs)" job=default.test-tfjob
time="2019-03-21T14:20:08Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:20:08Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:20:08Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:20:08Z" level=info msg="Started workers"
time="2019-03-21T14:20:08Z" level=info msg="Shutting down workers"
time="2019-03-21T14:20:08Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:08Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:08Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:08Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (210.345µs)" job=default.test-tfjob
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
time="2019-03-21T14:20:08Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:08Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:08Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:08Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (205.402µs)" job=default.test-tfjob
time="2019-03-21T14:20:08Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:08Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:08Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:08Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (384.396µs)" job=default.test-tfjob
time="2019-03-21T14:20:08Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:08Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (541.26µs)" job=default.test-tfjob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestAddPod
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:20:10Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:20:10Z" level=info msg="Started workers"
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
time="2019-03-21T14:20:10Z" level=info msg="Shutting down workers"
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:20:10Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0321 14:20:10.634167    9778 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc42041b4c0), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc420388840)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc420633dc0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-21T14:20:10Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0321 14:20:10.634485    9778 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc42041b4c0), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc420388840)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc420633dc0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'TFJobRestarting' 'TFJob test-tfjob is restarting because 1 Worker replica(s) failed.'
time="2019-03-21T14:20:10Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:10Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (709.646µs)" job=default.test-tfjob
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
time="2019-03-21T14:20:10Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:20:10Z" level=info msg="Started workers"
time="2019-03-21T14:20:10Z" level=info msg="Shutting down workers"
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:20:10Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:20:10Z" level=info msg="Started workers"
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
time="2019-03-21T14:20:10Z" level=info msg="Shutting down workers"
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0321 14:20:10.737485    9778 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc420389340)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4204b7da0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:10Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T14:20:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.914s	coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestNormalPath
time="2019-03-21T14:20:16Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:16Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:16Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:16Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:16Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:16Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:16Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:16Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:16Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:16Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:16Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:16Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:16Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:16Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:16Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (2.875596ms)" job=default.test-tfjob
time="2019-03-21T14:20:16Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:16Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:16Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (381.157µs)" job=default.test-tfjob
time="2019-03-21T14:20:16Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:16Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:16Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:16Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:16Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:16Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:16Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:16Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:16Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (523.263µs)" job=default.test-tfjob
time="2019-03-21T14:20:16Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:16Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:16Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:16Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:16Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:16Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:16Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:16Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (535.304µs)" job=default.test-tfjob
time="2019-03-21T14:20:16Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:16Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:16Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (391.386µs)" job=default.test-tfjob
time="2019-03-21T14:20:16Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:16Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:16Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:16Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:20:16Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:16Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:16Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (491.735µs)" job=default.test-tfjob
time="2019-03-21T14:20:16Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:16Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:16Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:16Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:16Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:16Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:16Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:16Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:16Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0321 14:20:16.983825    9888 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc42041e2e0), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"PS":(*v1beta2.ReplicaSpec)(0xc4207702c0), "Worker":(*v1beta2.ReplicaSpec)(0xc420770580)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc4207253f0), "PS":(*v1beta2.ReplicaStatus)(0xc4207253c0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobSucceeded' 'TFJob test-tfjob successfully completed.'
time="2019-03-21T14:20:16Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (2.306399ms)" job=default.test-tfjob
time="2019-03-21T14:20:16Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:16Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:16Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:16Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:16Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:16Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (306.541µs)" job=default.test-tfjob
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
time="2019-03-21T14:20:16Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:16Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:16Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:20:16Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:20:17Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:20:17Z" level=info msg="Started workers"
time="2019-03-21T14:20:17Z" level=info msg="Shutting down workers"
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
time="2019-03-21T14:20:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:17Z" level=info msg="Creating Job controller"
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
time="2019-03-21T14:20:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:17Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:17Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:20:17Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:20:17Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-21T14:20:17Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:20:17Z" level=info msg="Started workers"
--- PASS: TestAddTFJob (0.10s)
time="2019-03-21T14:20:17Z" level=info msg="Shutting down workers"
=== RUN   TestCopyLabelsAndAnnotation
time="2019-03-21T14:20:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:17Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:17Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:20:17Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:20:17Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:17Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:17Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:17Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (411.792µs)" job=default.test-tfjob
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
time="2019-03-21T14:20:17Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:20:17Z" level=info msg="Started workers"
time="2019-03-21T14:20:17Z" level=info msg="Shutting down workers"
time="2019-03-21T14:20:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:17Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:17Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:17Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (289.128µs)" job=default.test-tfjob
time="2019-03-21T14:20:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:17Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:17Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:17Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (292.059µs)" job=default.test-tfjob
time="2019-03-21T14:20:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:17Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:17Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:17Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:17Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:17Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:17Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:17Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:17Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:17Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (274.454µs)" job=default.test-tfjob
time="2019-03-21T14:20:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:17Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:17Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:17Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:17Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:17Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:17Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:17Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:17Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:17Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (285.159µs)" job=default.test-tfjob
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
time="2019-03-21T14:20:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:17Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:17Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:17Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (249.806µs)" job=default.test-tfjob
time="2019-03-21T14:20:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:17Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:17Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:17Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (221.16µs)" job=default.test-tfjob
time="2019-03-21T14:20:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:17Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:19Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:19Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:19Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:19Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:19Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:19Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:19Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T14:20:19Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (2.020238ms)" job=default.test-tfjob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestActiveDeadlineSeconds
time="2019-03-21T14:20:19Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:19Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:19Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:19Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:19Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:19Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.025028ms)" job=default.test-tfjob
time="2019-03-21T14:20:19Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:19Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="Job with ActiveDeadlineSeconds will sync after 2 seconds" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (640.675µs)" job=default.test-tfjob
--- FAIL: TestActiveDeadlineSeconds (2.01s)
	job_test.go:687: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected number of pod deletes.  Expected 6, saw 0
	job_test.go:690: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected number of service deletes.  Expected 6, saw 0
=== RUN   TestAddPod
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:20:21Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:20:21Z" level=info msg="Started workers"
--- PASS: TestAddPod (0.10s)
time="2019-03-21T14:20:21Z" level=info msg="Shutting down workers"
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:20:21Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="Ignoring inactive pod default/worker-0 in state Failed, deletion time <nil>"
time="2019-03-21T14:20:21Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0321 14:20:21.717608    9888 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc42039a850), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc42027d600)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc420725e70)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-21T14:20:21Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0321 14:20:21.717718    9888 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc42039a850), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc42027d600)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc420725e70)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'TFJobRestarting' 'TFJob test-tfjob is restarting because 1 Worker replica(s) failed.'
time="2019-03-21T14:20:21Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:20:21Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (517.495µs)" job=default.test-tfjob
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
time="2019-03-21T14:20:21Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:20:21Z" level=info msg="Started workers"
time="2019-03-21T14:20:21Z" level=info msg="Shutting down workers"
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:20:21Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:20:21Z" level=info msg="Started workers"
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
time="2019-03-21T14:20:21Z" level=info msg="Shutting down workers"
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0321 14:20:21.821097    9888 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc420116b00)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc4204d1e50)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:20:21Z" level=info msg="Creating Job controller"
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T14:20:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
--- PASS: TestStatus (0.02s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
FAIL
coverage: 42.4% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow	4.920s
exit status 1: warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/version

travis_time:end:09bc7df5:start=1553177963166144494,finish=1553178021858227454,duration=58692082960
TravisBuddy Request Identifier: b93e7ec0-4be7-11e9-9767-c9e36d0d3911

@TravisBuddy
Copy link

Travis tests have failed

Hey @ChanYiLin,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
pkg/controller.v1beta2/tensorflow/job_test.go:683:15:warning: unused variable or constant no new variables on left side of := (varcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:23:2:warning: unused variable or constant "github.com/kubeflow/tf-operator/vendor/k8s.io/apimachinery/pkg/apis/meta/v1" imported but not used as metav1 (varcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:1::warning: file is not goimported (goimports)
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.054s
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.047s
=== RUN   TestValidateBetaOneTFJobSpec
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 2.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.044s
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 1.0% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.044s
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 4.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.065s
=== RUN   TestNormalPath
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
--- PASS: TestCleanupTFJob (2.00s)
=== RUN   TestAddPod
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 29.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.910s
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow [build failed]
exit status 2: warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/version
# github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow
pkg/controller.v1beta2/tensorflow/job_test.go:23:2: imported and not used: "github.com/kubeflow/tf-operator/vendor/k8s.io/apimachinery/pkg/apis/meta/v1" as metav1
pkg/controller.v1beta2/tensorflow/job_test.go:683:15: no new variables on left side of :=

2nd Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
pkg/controller.v1beta2/tensorflow/job_test.go:23:2:warning: unused variable or constant "github.com/kubeflow/tf-operator/vendor/k8s.io/apimachinery/pkg/apis/meta/v1" imported but not used as metav1 (varcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:1::warning: file is not goimported (goimports)
pkg/controller.v1beta2/tensorflow/job_test.go:23:2:error: imported and not used: "github.com/kubeflow/tf-operator/vendor/k8s.io/apimachinery/pkg/apis/meta/v1" as metav1 (vet)
pkg/controller.v1beta2/tensorflow/job_test.go:683:15:error: no new variables on left side of := (vet)
pkg/controller.v1beta2/tensorflow/job_test.go:683:15:warning: no new variables on left side of := (compile) (staticcheck, staticcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:23:2:warning: "github.com/kubeflow/tf-operator/vendor/k8s.io/apimachinery/pkg/apis/meta/v1" imported but not used as metav1 (compile) (staticcheck, staticcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:683:15:warning: unused variable or constant no new variables on left side of := (varcheck)
travis_time:end:02f7dc50:start=1553180122329157952,finish=1553180283761485670,duration=161432327718
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.032s	coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.028s	coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestValidateBetaOneTFJobSpec
time="2019-03-21T14:58:18Z" level=warning msg="Image is undefined in the container"
time="2019-03-21T14:58:18Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
time="2019-03-21T14:58:18Z" level=warning msg="Image is undefined in the container"
time="2019-03-21T14:58:18Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.030s	coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.018s	coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
time="2019-03-21T14:58:35Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
time="2019-03-21T14:58:35Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.055s	coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestNormalPath
time="2019-03-21T14:58:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:41Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:41Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.614162ms)" job=default.test-tfjob
time="2019-03-21T14:58:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:41Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:58:41Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:58:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:58:41Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:58:41Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:41Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:41Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:41Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:41Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:41Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:41Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:41Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.120093ms)" job=default.test-tfjob
time="2019-03-21T14:58:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:41Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (488.555µs)" job=default.test-tfjob
time="2019-03-21T14:58:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:41Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:41Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:41Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:41Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:58:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:58:41Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (872.205µs)" job=default.test-tfjob
time="2019-03-21T14:58:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:41Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:58:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:58:41Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:41Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (626.94µs)" job=default.test-tfjob
time="2019-03-21T14:58:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:41Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (906.769µs)" job=default.test-tfjob
time="2019-03-21T14:58:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:41Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:41Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:58:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T14:58:41Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.543121ms)" job=default.test-tfjob
time="2019-03-21T14:58:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:41Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0321 14:58:41.856266    9854 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc4204de730), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc4202acdc0), "PS":(*v1beta1.ReplicaSpec)(0xc4202ad340)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc42036bdf0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobSucceeded' 'TFJob test-tfjob successfully completed.'
time="2019-03-21T14:58:41Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:41Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (937.459µs)" job=default.test-tfjob
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
time="2019-03-21T14:58:41Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:41Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:41Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:58:41Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:58:41Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:58:41Z" level=info msg="Started workers"
time="2019-03-21T14:58:42Z" level=info msg="Shutting down workers"
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
time="2019-03-21T14:58:42Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:42Z" level=info msg="Creating Job controller"
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
time="2019-03-21T14:58:42Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:42Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:42Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:58:42Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:58:42Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-21T14:58:42Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:58:42Z" level=info msg="Started workers"
--- PASS: TestAddTFJob (0.10s)
time="2019-03-21T14:58:42Z" level=info msg="Shutting down workers"
=== RUN   TestCopyLabelsAndAnnotation
time="2019-03-21T14:58:42Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:42Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:42Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:58:42Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:58:42Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:58:42Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:42Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:42Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (294.865µs)" job=default.test-tfjob
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
time="2019-03-21T14:58:42Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:58:42Z" level=info msg="Started workers"
time="2019-03-21T14:58:42Z" level=info msg="Shutting down workers"
time="2019-03-21T14:58:42Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:42Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:42Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:58:42Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (252.388µs)" job=default.test-tfjob
time="2019-03-21T14:58:42Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:42Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:42Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:58:42Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (237.581µs)" job=default.test-tfjob
time="2019-03-21T14:58:42Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:42Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:42Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:58:42Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (229.177µs)" job=default.test-tfjob
time="2019-03-21T14:58:42Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:42Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:42Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:58:42Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (437.908µs)" job=default.test-tfjob
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
time="2019-03-21T14:58:42Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:42Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:42Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:58:42Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (230.818µs)" job=default.test-tfjob
time="2019-03-21T14:58:42Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:42Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:42Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:58:42Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (281.925µs)" job=default.test-tfjob
time="2019-03-21T14:58:42Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:42Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (402.221µs)" job=default.test-tfjob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestAddPod
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:58:44Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:58:44Z" level=info msg="Started workers"
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
time="2019-03-21T14:58:44Z" level=info msg="Shutting down workers"
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:58:44Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0321 14:58:44.577930    9854 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc420296c00), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc42043e000)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4205cde60)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-21T14:58:44Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0321 14:58:44.578033    9854 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc420296c00), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc42043e000)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4205cde60)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'TFJobRestarting' 'TFJob test-tfjob is restarting because 1 Worker replica(s) failed.'
time="2019-03-21T14:58:44Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T14:58:44Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (528.063µs)" job=default.test-tfjob
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
time="2019-03-21T14:58:44Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:58:44Z" level=info msg="Started workers"
time="2019-03-21T14:58:44Z" level=info msg="Shutting down workers"
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="Starting TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T14:58:44Z" level=info msg="Starting 1 workers"
time="2019-03-21T14:58:44Z" level=info msg="Started workers"
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
time="2019-03-21T14:58:44Z" level=info msg="Shutting down workers"
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0321 14:58:44.680924    9854 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc42034cb00)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4200c6170)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="Creating TFJob controller"
time="2019-03-21T14:58:44Z" level=info msg="Creating Job controller"
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T14:58:44Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
--- PASS: TestStatus (0.02s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.913s	coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow [build failed]
exit status 2: warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/version
# github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow
pkg/controller.v1beta2/tensorflow/job_test.go:23:2: imported and not used: "github.com/kubeflow/tf-operator/vendor/k8s.io/apimachinery/pkg/apis/meta/v1" as metav1
pkg/controller.v1beta2/tensorflow/job_test.go:683:15: no new variables on left side of :=

travis_time:end:008bdb38:start=1553180283767999787,finish=1553180326359354588,duration=42591354801
TravisBuddy Request Identifier: 23228cf0-4bed-11e9-9767-c9e36d0d3911

@TravisBuddy
Copy link

Travis tests have failed

Hey @ChanYiLin,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
pkg/controller.v1beta2/tensorflow/job_test.go:23:2:warning: unused variable or constant "github.com/kubeflow/tf-operator/vendor/k8s.io/apimachinery/pkg/apis/meta/v1" imported but not used as metav1 (varcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:1::warning: file is not goimported (goimports)
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.053s
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.045s
=== RUN   TestValidateBetaOneTFJobSpec
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 2.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.044s
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 1.0% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.044s
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 4.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.064s
=== RUN   TestNormalPath
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
--- PASS: TestDeletePodsAndServices (0.00s)
=== RUN   TestCleanupTFJob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestAddPod
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 29.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.909s
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow [build failed]
exit status 2: warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/version
# github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow
pkg/controller.v1beta2/tensorflow/job_test.go:23:2: imported and not used: "github.com/kubeflow/tf-operator/vendor/k8s.io/apimachinery/pkg/apis/meta/v1" as metav1

2nd Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
pkg/controller.v1beta2/tensorflow/job_test.go:23:2:error: imported and not used: "github.com/kubeflow/tf-operator/vendor/k8s.io/apimachinery/pkg/apis/meta/v1" as metav1 (vet)
pkg/controller.v1beta2/tensorflow/job_test.go:23:2:warning: unused variable or constant "github.com/kubeflow/tf-operator/vendor/k8s.io/apimachinery/pkg/apis/meta/v1" imported but not used as metav1 (varcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:1::warning: file is not goimported (goimports)
pkg/controller.v1beta2/tensorflow/job_test.go:23:2:warning: "github.com/kubeflow/tf-operator/vendor/k8s.io/apimachinery/pkg/apis/meta/v1" imported but not used as metav1 (compile) (staticcheck, staticcheck)
travis_time:end:18eb1dbb:start=1553180559368770437,finish=1553180732919812370,duration=173551041933
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.031s	coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.030s	coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestValidateBetaOneTFJobSpec
time="2019-03-21T15:05:49Z" level=warning msg="Image is undefined in the container"
time="2019-03-21T15:05:49Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
time="2019-03-21T15:05:49Z" level=warning msg="Image is undefined in the container"
time="2019-03-21T15:05:49Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.031s	coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.020s	coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
time="2019-03-21T15:06:07Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
time="2019-03-21T15:06:07Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.056s	coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestNormalPath
time="2019-03-21T15:06:14Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:14Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:14Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:06:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:06:14Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:14Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:14Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:14Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (2.114753ms)" job=default.test-tfjob
time="2019-03-21T15:06:14Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:14Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:14Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:06:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:06:14Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:14Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (613.317µs)" job=default.test-tfjob
time="2019-03-21T15:06:14Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:14Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:14Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (341.333µs)" job=default.test-tfjob
time="2019-03-21T15:06:14Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:14Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:14Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:06:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:06:14Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:14Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (595.818µs)" job=default.test-tfjob
time="2019-03-21T15:06:14Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:14Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:14Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0321 15:06:14.851716    9765 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc420272520), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"PS":(*v1beta1.ReplicaSpec)(0xc420630000), "Worker":(*v1beta1.ReplicaSpec)(0xc4206302c0)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"PS":(*v1beta1.ReplicaStatus)(0xc420366e50), "Worker":(*v1beta1.ReplicaStatus)(0xc420366eb0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobSucceeded' 'TFJob test-tfjob successfully completed.'
time="2019-03-21T15:06:14Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.27666ms)" job=default.test-tfjob
time="2019-03-21T15:06:14Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:14Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:14Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:14Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (841.049µs)" job=default.test-tfjob
time="2019-03-21T15:06:14Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:14Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:14Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:14Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:14Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:14Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:14Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:14Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:14Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:14Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:06:14Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:06:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:06:14Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:06:14Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (964.194µs)" job=default.test-tfjob
time="2019-03-21T15:06:14Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:14Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:14Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:14Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (404.313µs)" job=default.test-tfjob
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
time="2019-03-21T15:06:14Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:14Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:14Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:06:14Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:06:14Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:06:14Z" level=info msg="Started workers"
time="2019-03-21T15:06:15Z" level=info msg="Shutting down workers"
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
time="2019-03-21T15:06:15Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:15Z" level=info msg="Creating Job controller"
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
time="2019-03-21T15:06:15Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:15Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:15Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:06:15Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:06:15Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-21T15:06:15Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:06:15Z" level=info msg="Started workers"
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
time="2019-03-21T15:06:15Z" level=info msg="Shutting down workers"
time="2019-03-21T15:06:15Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:15Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:15Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:06:15Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:06:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:06:15Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:15Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (354.036µs)" job=default.test-tfjob
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
time="2019-03-21T15:06:15Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:06:15Z" level=info msg="Started workers"
time="2019-03-21T15:06:15Z" level=info msg="Shutting down workers"
time="2019-03-21T15:06:15Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:15Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:06:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (272.919µs)" job=default.test-tfjob
time="2019-03-21T15:06:15Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:15Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:06:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (256.961µs)" job=default.test-tfjob
time="2019-03-21T15:06:15Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:15Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:06:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (245.283µs)" job=default.test-tfjob
time="2019-03-21T15:06:15Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:15Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:06:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (231.484µs)" job=default.test-tfjob
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
time="2019-03-21T15:06:15Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:15Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:06:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (308.106µs)" job=default.test-tfjob
time="2019-03-21T15:06:15Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:15Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:06:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (222.42µs)" job=default.test-tfjob
time="2019-03-21T15:06:15Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:15Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (587.517µs)" job=default.test-tfjob
--- PASS: TestCleanupTFJob (2.00s)
=== RUN   TestAddPod
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:06:17Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:06:17Z" level=info msg="Started workers"
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
time="2019-03-21T15:06:17Z" level=info msg="Shutting down workers"
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:06:17Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0321 15:06:17.579349    9765 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc420422980), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc420110840)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4207f6f50)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-21T15:06:17Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0321 15:06:17.579487    9765 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc420422980), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc420110840)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4207f6f50)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'TFJobRestarting' 'TFJob test-tfjob is restarting because 1 Worker replica(s) failed.'
time="2019-03-21T15:06:17Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:06:17Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (437.251µs)" job=default.test-tfjob
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
time="2019-03-21T15:06:17Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:06:17Z" level=info msg="Started workers"
time="2019-03-21T15:06:17Z" level=info msg="Shutting down workers"
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:06:17Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:06:17Z" level=info msg="Started workers"
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
time="2019-03-21T15:06:17Z" level=info msg="Shutting down workers"
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0321 15:06:17.682738    9765 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc4209bc2c0)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc420460f50)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:06:17Z" level=info msg="Creating Job controller"
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T15:06:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.904s	coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow [build failed]
exit status 2: warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/version
# github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow
pkg/controller.v1beta2/tensorflow/job_test.go:23:2: imported and not used: "github.com/kubeflow/tf-operator/vendor/k8s.io/apimachinery/pkg/apis/meta/v1" as metav1

travis_time:end:069f7972:start=1553180732927061345,finish=1553180779472501847,duration=46545440502
TravisBuddy Request Identifier: 283396c0-4bee-11e9-9767-c9e36d0d3911

@TravisBuddy
Copy link

Travis tests have failed

Hey @ChanYiLin,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.057s
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.047s
=== RUN   TestValidateBetaOneTFJobSpec
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 2.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.045s
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 1.0% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.046s
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 4.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.064s
=== RUN   TestNormalPath
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestAddPod
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 29.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.904s
=== RUN   TestNormalPath
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
--- PASS: TestDeletePodsAndServices (0.00s)
=== RUN   TestCleanupTFJob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestActiveDeadlineSeconds
--- FAIL: TestActiveDeadlineSeconds (2.00s)
	job_test.go:691: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected number of pod deletes.  Expected 6, saw 0
	job_test.go:694: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected number of service deletes.  Expected 6, saw 0
=== RUN   TestAddPod
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
FAIL
coverage: 31.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
exit status 1
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow	4.911s
exit status 1: warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/version
time="2019-03-21T15:39:54Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:54Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:54Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:39:54Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:39:54Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:54Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:54Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:54Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:54Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (2.104936ms)" job=default.test-tfjob
time="2019-03-21T15:39:54Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:54Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:54Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:54Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:54Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:54Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:54Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:54Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:54Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0321 15:39:54.602946    6058 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc4204ceae0), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"PS":(*v1beta2.ReplicaSpec)(0xc420283340), "Worker":(*v1beta2.ReplicaSpec)(0xc420283600)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"PS":(*v1beta2.ReplicaStatus)(0xc4203a6790), "Worker":(*v1beta2.ReplicaStatus)(0xc4203a6850)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobSucceeded' 'TFJob test-tfjob successfully completed.'
time="2019-03-21T15:39:54Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.836806ms)" job=default.test-tfjob
time="2019-03-21T15:39:54Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:54Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:54Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (742.52µs)" job=default.test-tfjob
time="2019-03-21T15:39:54Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:54Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:54Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (610.735µs)" job=default.test-tfjob
time="2019-03-21T15:39:54Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:54Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:54Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:39:54Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:39:54Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:54Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:54Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (420.027µs)" job=default.test-tfjob
time="2019-03-21T15:39:54Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:54Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:54Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:54Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:39:54Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:39:54Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:54Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:54Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (436.12µs)" job=default.test-tfjob
time="2019-03-21T15:39:54Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:54Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:54Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:54Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:54Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (202.552µs)" job=default.test-tfjob
time="2019-03-21T15:39:54Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:54Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:54Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:39:54Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:39:54Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:39:54Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:39:54Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:54Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:54Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:54Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:54Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:54Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:54Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:54Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:54Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:54Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (589.897µs)" job=default.test-tfjob
time="2019-03-21T15:39:54Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:54Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:54Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:39:54Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:39:54Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:39:54Z" level=info msg="Started workers"
time="2019-03-21T15:39:55Z" level=info msg="Shutting down workers"
time="2019-03-21T15:39:55Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:55Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:55Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:55Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:55Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:39:55Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:39:55Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-21T15:39:55Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:39:55Z" level=info msg="Started workers"
time="2019-03-21T15:39:55Z" level=info msg="Shutting down workers"
time="2019-03-21T15:39:55Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:55Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:55Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:55Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:55Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:55Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:55Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (336.702µs)" job=default.test-tfjob
time="2019-03-21T15:39:55Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:39:55Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:39:55Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:39:55Z" level=info msg="Started workers"
time="2019-03-21T15:39:55Z" level=info msg="Shutting down workers"
time="2019-03-21T15:39:55Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:55Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:55Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:55Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (261.246µs)" job=default.test-tfjob
time="2019-03-21T15:39:55Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:55Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:55Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:55Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (250.864µs)" job=default.test-tfjob
time="2019-03-21T15:39:55Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:55Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:55Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:55Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:55Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:55Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:55Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:55Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:55Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:55Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (263.739µs)" job=default.test-tfjob
time="2019-03-21T15:39:55Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:55Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:55Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:55Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:55Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:55Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:55Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:55Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:55Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:55Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (291.73µs)" job=default.test-tfjob
time="2019-03-21T15:39:55Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:55Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:55Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:55Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (426.753µs)" job=default.test-tfjob
time="2019-03-21T15:39:55Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:55Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:55Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:55Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (428.175µs)" job=default.test-tfjob
time="2019-03-21T15:39:55Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:55Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:57Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:57Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:57Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:57Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:57Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:57Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:57Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:39:57Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (603.059µs)" job=default.test-tfjob
time="2019-03-21T15:39:57Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:57Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:57Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:57Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:57Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:57Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (453.314µs)" job=default.test-tfjob
time="2019-03-21T15:39:57Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:57Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:57Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:57Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (335.792µs)" job=default.test-tfjob
time="2019-03-21T15:39:57Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:57Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:57Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:57Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:57Z" level=info msg="Job with ActiveDeadlineSeconds will sync after 2 seconds" job=default.test-tfjob uid=
time="2019-03-21T15:39:57Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:57Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (394.859µs)" job=default.test-tfjob
time="2019-03-21T15:39:59Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="Job with ActiveDeadlineSeconds will sync after 2 seconds" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (598.078µs)" job=default.test-tfjob
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:39:59Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:39:59Z" level=info msg="Started workers"
time="2019-03-21T15:39:59Z" level=info msg="Shutting down workers"
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:39:59Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="Ignoring inactive pod default/worker-0 in state Failed, deletion time <nil>"
time="2019-03-21T15:39:59Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0321 15:39:59.335896    6058 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc42028ba20), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc42039b8c0)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc420519590)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-21T15:39:59Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0321 15:39:59.335991    6058 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc42028ba20), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc42039b8c0)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc420519590)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'TFJobRestarting' 'TFJob test-tfjob is restarting because 1 Worker replica(s) failed.'
time="2019-03-21T15:39:59Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:39:59Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (457.666µs)" job=default.test-tfjob
time="2019-03-21T15:39:59Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:39:59Z" level=info msg="Started workers"
time="2019-03-21T15:39:59Z" level=info msg="Shutting down workers"
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:39:59Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:39:59Z" level=info msg="Started workers"
time="2019-03-21T15:39:59Z" level=info msg="Shutting down workers"
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0321 15:39:59.438662    6058 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc4200c6b00)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc4203c8e10)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:39:59Z" level=info msg="Creating Job controller"
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T15:39:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=

2nd Build

View build log

goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.032s	coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.031s	coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestValidateBetaOneTFJobSpec
time="2019-03-21T15:14:57Z" level=warning msg="Image is undefined in the container"
time="2019-03-21T15:14:57Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
time="2019-03-21T15:14:57Z" level=warning msg="Image is undefined in the container"
time="2019-03-21T15:14:57Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.031s	coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.022s	coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
time="2019-03-21T15:15:17Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
time="2019-03-21T15:15:17Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.063s	coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestNormalPath
time="2019-03-21T15:15:24Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:24Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:24Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:24Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:24Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:24Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:24Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:24Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:24Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (2.497897ms)" job=default.test-tfjob
time="2019-03-21T15:15:24Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:24Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:24Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:24Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:24Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:24Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:24Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (875.347µs)" job=default.test-tfjob
time="2019-03-21T15:15:24Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:24Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:24Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (411.428µs)" job=default.test-tfjob
time="2019-03-21T15:15:24Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:24Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:24Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:24Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:24Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:24Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:24Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (561.642µs)" job=default.test-tfjob
time="2019-03-21T15:15:24Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:24Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:24Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0321 15:15:24.900251    9773 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc42005a3a0), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"PS":(*v1beta1.ReplicaSpec)(0xc42070c000), "Worker":(*v1beta1.ReplicaSpec)(0xc42070c2c0)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"PS":(*v1beta1.ReplicaStatus)(0xc420369950), "Worker":(*v1beta1.ReplicaStatus)(0xc420369980)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobSucceeded' 'TFJob test-tfjob successfully completed.'
time="2019-03-21T15:15:24Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (5.060086ms)" job=default.test-tfjob
time="2019-03-21T15:15:24Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:24Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:24Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:24Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:24Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (448.847µs)" job=default.test-tfjob
time="2019-03-21T15:15:24Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:24Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:24Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:24Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:24Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:24Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:24Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:24Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:24Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:24Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:24Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:24Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:24Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:24Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:24Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.243162ms)" job=default.test-tfjob
time="2019-03-21T15:15:24Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:24Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:24Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:24Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (415.696µs)" job=default.test-tfjob
--- PASS: TestNormalPath (0.03s)
=== RUN   TestRun
time="2019-03-21T15:15:24Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:24Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:24Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:15:24Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:15:25Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:15:25Z" level=info msg="Started workers"
time="2019-03-21T15:15:25Z" level=info msg="Shutting down workers"
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
time="2019-03-21T15:15:25Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:25Z" level=info msg="Creating Job controller"
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
time="2019-03-21T15:15:25Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:25Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:25Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-21T15:15:25Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:15:25Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:15:25Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:15:25Z" level=info msg="Started workers"
--- PASS: TestAddTFJob (0.10s)
time="2019-03-21T15:15:25Z" level=info msg="Shutting down workers"
=== RUN   TestCopyLabelsAndAnnotation
time="2019-03-21T15:15:25Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:25Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:25Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:15:25Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:15:25Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:25Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:25Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:25Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (296.356µs)" job=default.test-tfjob
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
time="2019-03-21T15:15:25Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:15:25Z" level=info msg="Started workers"
time="2019-03-21T15:15:25Z" level=info msg="Shutting down workers"
time="2019-03-21T15:15:25Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:25Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:25Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:25Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (274.47µs)" job=default.test-tfjob
time="2019-03-21T15:15:25Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:25Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:25Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:25Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (218.793µs)" job=default.test-tfjob
time="2019-03-21T15:15:25Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:25Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:25Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:25Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (238.326µs)" job=default.test-tfjob
time="2019-03-21T15:15:25Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:25Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:25Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:25Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (294.572µs)" job=default.test-tfjob
--- PASS: TestDeletePodsAndServices (0.00s)
=== RUN   TestCleanupTFJob
time="2019-03-21T15:15:25Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:25Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:25Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:25Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (219.685µs)" job=default.test-tfjob
time="2019-03-21T15:15:25Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:25Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:25Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:25Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (205.021µs)" job=default.test-tfjob
time="2019-03-21T15:15:25Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:25Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (584.795µs)" job=default.test-tfjob
--- PASS: TestCleanupTFJob (2.00s)
=== RUN   TestAddPod
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:15:27Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:15:27Z" level=info msg="Started workers"
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
time="2019-03-21T15:15:27Z" level=info msg="Shutting down workers"
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:15:27Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0321 15:15:27.635470    9773 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc4203c4990), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc420130840)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4207faf80)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-21T15:15:27Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0321 15:15:27.635855    9773 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc4203c4990), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc420130840)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4207faf80)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'TFJobRestarting' 'TFJob test-tfjob is restarting because 1 Worker replica(s) failed.'
time="2019-03-21T15:15:27Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:27Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (782.795µs)" job=default.test-tfjob
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
time="2019-03-21T15:15:27Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:15:27Z" level=info msg="Started workers"
time="2019-03-21T15:15:27Z" level=info msg="Shutting down workers"
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:15:27Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:15:27Z" level=info msg="Started workers"
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
time="2019-03-21T15:15:27Z" level=info msg="Shutting down workers"
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0321 15:15:27.739457    9773 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc4209d22c0)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4205ced70)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:27Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T15:15:27Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.922s	coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestNormalPath
time="2019-03-21T15:15:34Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:34Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:34Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:34Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:34Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:34Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:34Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:34Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0321 15:15:34.377047    9890 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc4205d84e0), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"PS":(*v1beta2.ReplicaSpec)(0xc420346b00), "Worker":(*v1beta2.ReplicaSpec)(0xc420346dc0)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"PS":(*v1beta2.ReplicaStatus)(0xc4205ccf80), "Worker":(*v1beta2.ReplicaStatus)(0xc4205ccfb0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobSucceeded' 'TFJob test-tfjob successfully completed.'
time="2019-03-21T15:15:34Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (2.316334ms)" job=default.test-tfjob
time="2019-03-21T15:15:34Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:34Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:34Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (361.075µs)" job=default.test-tfjob
time="2019-03-21T15:15:34Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:34Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:34Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:34Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:34Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:34Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (665.876µs)" job=default.test-tfjob
time="2019-03-21T15:15:34Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:34Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:34Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (855.08µs)" job=default.test-tfjob
time="2019-03-21T15:15:34Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:34Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:34Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:34Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (941.08µs)" job=default.test-tfjob
time="2019-03-21T15:15:34Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:34Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:34Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:34Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:34Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (513.183µs)" job=default.test-tfjob
time="2019-03-21T15:15:34Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:34Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:34Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (388.817µs)" job=default.test-tfjob
time="2019-03-21T15:15:34Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:34Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:34Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-21T15:15:34Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (484.521µs)" job=default.test-tfjob
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
time="2019-03-21T15:15:34Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:34Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:34Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:15:34Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:15:34Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:15:34Z" level=info msg="Started workers"
time="2019-03-21T15:15:34Z" level=info msg="Shutting down workers"
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
time="2019-03-21T15:15:34Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:34Z" level=info msg="Creating Job controller"
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
time="2019-03-21T15:15:34Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:34Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:34Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:15:34Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:15:34Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:15:34Z" level=info msg="Started workers"
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
time="2019-03-21T15:15:34Z" level=info msg="Shutting down workers"
time="2019-03-21T15:15:34Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:34Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:34Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:34Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (357.565µs)" job=default.test-tfjob
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
time="2019-03-21T15:15:34Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:15:34Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:15:34Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:15:34Z" level=info msg="Started workers"
time="2019-03-21T15:15:34Z" level=info msg="Shutting down workers"
time="2019-03-21T15:15:34Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:34Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:34Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (308.581µs)" job=default.test-tfjob
time="2019-03-21T15:15:34Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:34Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:34Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:34Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (304.435µs)" job=default.test-tfjob
time="2019-03-21T15:15:34Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:34Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:35Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:35Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:35Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:35Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:35Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:35Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:35Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:35Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (299.304µs)" job=default.test-tfjob
time="2019-03-21T15:15:35Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:35Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:35Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:35Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:35Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:35Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:35Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:35Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:35Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:35Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (320.431µs)" job=default.test-tfjob
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
time="2019-03-21T15:15:35Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:35Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:35Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:35Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (399.941µs)" job=default.test-tfjob
time="2019-03-21T15:15:35Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:35Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:35Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:35Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (532.033µs)" job=default.test-tfjob
time="2019-03-21T15:15:35Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:35Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:37Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:37Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:37Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:37Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:37Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:37Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:37Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-21T15:15:37Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.013069ms)" job=default.test-tfjob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestActiveDeadlineSeconds
time="2019-03-21T15:15:37Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:37Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:37Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:37Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:37Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:37Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (441.833µs)" job=default.test-tfjob
time="2019-03-21T15:15:37Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:37Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:37Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:37Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (542.217µs)" job=default.test-tfjob
time="2019-03-21T15:15:37Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:37Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:37Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:37Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:37Z" level=info msg="Job with ActiveDeadlineSeconds will sync after 2 seconds" job=default.test-tfjob uid=
time="2019-03-21T15:15:37Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:37Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (460.321µs)" job=default.test-tfjob
time="2019-03-21T15:15:39Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="Job with ActiveDeadlineSeconds will sync after 2 seconds" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (874.698µs)" job=default.test-tfjob
--- FAIL: TestActiveDeadlineSeconds (2.00s)
	job_test.go:691: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected number of pod deletes.  Expected 6, saw 0
	job_test.go:694: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected number of service deletes.  Expected 6, saw 0
=== RUN   TestAddPod
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:15:39Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:15:39Z" level=info msg="Started workers"
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
time="2019-03-21T15:15:39Z" level=info msg="Shutting down workers"
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:15:39Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="Ignoring inactive pod default/worker-0 in state Failed, deletion time <nil>"
time="2019-03-21T15:15:39Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0321 15:15:39.121380    9890 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc42005a5d0), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc42037cb00)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc420045700)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-21T15:15:39Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0321 15:15:39.121575    9890 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc42005a5d0), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc42037cb00)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc420045700)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'TFJobRestarting' 'TFJob test-tfjob is restarting because 1 Worker replica(s) failed.'
time="2019-03-21T15:15:39Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-21T15:15:39Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (906.247µs)" job=default.test-tfjob
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
time="2019-03-21T15:15:39Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:15:39Z" level=info msg="Started workers"
time="2019-03-21T15:15:39Z" level=info msg="Shutting down workers"
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="Starting TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-21T15:15:39Z" level=info msg="Starting 1 workers"
time="2019-03-21T15:15:39Z" level=info msg="Started workers"
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
time="2019-03-21T15:15:39Z" level=info msg="Shutting down workers"
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0321 15:15:39.225096    9890 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc42037d600)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc420392df0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="Creating TFJob controller"
time="2019-03-21T15:15:39Z" level=info msg="Creating Job controller"
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-21T15:15:39Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
--- PASS: TestStatus (0.02s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
FAIL
coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow	4.924s
exit status 1: warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/version

travis_time:end:26a15ea0:start=1553181279975880720,finish=1553181339261795918,duration=59285915198
TravisBuddy Request Identifier: 9630d380-4bef-11e9-9767-c9e36d0d3911

@richardsliu
Copy link
Contributor

/retest

1 similar comment
@richardsliu
Copy link
Contributor

/retest

@richardsliu
Copy link
Contributor

/test all

@TravisBuddy
Copy link

Travis tests have failed

Hey @ChanYiLin,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
pkg/controller.v1beta2/tensorflow/job_test.go:1::warning: file is not goimported (goimports)
pkg/controller.v1beta2/tensorflow/job_test.go:672:10:warning: unused variable or constant no new variables on left side of := (varcheck)
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.050s
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.048s
=== RUN   TestValidateBetaOneTFJobSpec
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 2.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.046s
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 1.0% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.047s
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 4.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.066s
=== RUN   TestNormalPath
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
--- PASS: TestCleanupTFJob (2.00s)
=== RUN   TestAddPod
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 29.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.926s
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow [build failed]
exit status 2: warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/version
# github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow
pkg/controller.v1beta2/tensorflow/job_test.go:672:10: no new variables on left side of :=

2nd Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
pkg/controller.v1beta2/tensorflow/job_test.go:672:10:warning: unused variable or constant no new variables on left side of := (varcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:1::warning: file is not goimported (goimports)
pkg/controller.v1beta2/tensorflow/job_test.go:672:10:warning: no new variables on left side of := (compile) (staticcheck, staticcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:672:10:error: no new variables on left side of := (vet)
travis_time:end:07c19718:start=1553216792449342608,finish=1553216963026982119,duration=170577639511
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.031s	coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.029s	coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestValidateBetaOneTFJobSpec
time="2019-03-22T01:09:39Z" level=warning msg="Image is undefined in the container"
time="2019-03-22T01:09:39Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
time="2019-03-22T01:09:39Z" level=warning msg="Image is undefined in the container"
time="2019-03-22T01:09:39Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.029s	coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.021s	coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
time="2019-03-22T01:09:56Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
time="2019-03-22T01:09:56Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.071s	coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestNormalPath
time="2019-03-22T01:10:03Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:03Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:03Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:10:03Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:10:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:10:03Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:10:03Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:03Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:03Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:03Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:03Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:03Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:03Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:03Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.922839ms)" job=default.test-tfjob
time="2019-03-22T01:10:03Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:03Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:03Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (402.984µs)" job=default.test-tfjob
time="2019-03-22T01:10:03Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:03Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:03Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:10:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:10:03Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:03Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:03Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:03Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (489.042µs)" job=default.test-tfjob
time="2019-03-22T01:10:03Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:03Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:03Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:10:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:10:03Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:03Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (484.019µs)" job=default.test-tfjob
time="2019-03-22T01:10:03Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:03Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:03Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (368.082µs)" job=default.test-tfjob
time="2019-03-22T01:10:03Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:03Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:03Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:10:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:10:03Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:03Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (504.965µs)" job=default.test-tfjob
time="2019-03-22T01:10:03Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:03Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:03Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0322 01:10:03.562787    9833 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc420520cf0), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc4207c42c0), "PS":(*v1beta1.ReplicaSpec)(0xc4207c4580)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc42001df20)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobSucceeded' 'TFJob test-tfjob successfully completed.'
time="2019-03-22T01:10:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (2.390786ms)" job=default.test-tfjob
time="2019-03-22T01:10:03Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:03Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:03Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:03Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:03Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (400.356µs)" job=default.test-tfjob
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
time="2019-03-22T01:10:03Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:03Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:03Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:10:03Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:10:03Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:10:03Z" level=info msg="Started workers"
time="2019-03-22T01:10:04Z" level=info msg="Shutting down workers"
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
time="2019-03-22T01:10:04Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:04Z" level=info msg="Creating Job controller"
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
time="2019-03-22T01:10:04Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:04Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:04Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-22T01:10:04Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:10:04Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:10:04Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:10:04Z" level=info msg="Started workers"
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
time="2019-03-22T01:10:04Z" level=info msg="Shutting down workers"
time="2019-03-22T01:10:04Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:04Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:04Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:10:04Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:10:04Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:10:04Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:04Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:04Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:04Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (694.781µs)" job=default.test-tfjob
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
time="2019-03-22T01:10:04Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:10:04Z" level=info msg="Started workers"
time="2019-03-22T01:10:04Z" level=info msg="Shutting down workers"
time="2019-03-22T01:10:04Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:04Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:04Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:10:04Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (249.245µs)" job=default.test-tfjob
time="2019-03-22T01:10:04Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:04Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:04Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:10:04Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (207.578µs)" job=default.test-tfjob
time="2019-03-22T01:10:04Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:04Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:04Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:10:04Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (186.909µs)" job=default.test-tfjob
time="2019-03-22T01:10:04Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:04Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:04Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:10:04Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (221.933µs)" job=default.test-tfjob
--- PASS: TestDeletePodsAndServices (0.00s)
=== RUN   TestCleanupTFJob
time="2019-03-22T01:10:04Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:04Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:04Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:10:04Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (325.616µs)" job=default.test-tfjob
time="2019-03-22T01:10:04Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:04Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:04Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:10:04Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (222.337µs)" job=default.test-tfjob
time="2019-03-22T01:10:04Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:04Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (587.725µs)" job=default.test-tfjob
--- PASS: TestCleanupTFJob (2.00s)
=== RUN   TestAddPod
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:10:06Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:10:06Z" level=info msg="Started workers"
--- PASS: TestAddPod (0.11s)
=== RUN   TestClusterSpec
time="2019-03-22T01:10:06Z" level=info msg="Shutting down workers"
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:10:06Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0322 01:10:06.288950    9833 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc420458880), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc42034d600)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4207caf50)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-22T01:10:06Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0322 01:10:06.289209    9833 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc420458880), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc42034d600)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4207caf50)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'TFJobRestarting' 'TFJob test-tfjob is restarting because 1 Worker replica(s) failed.'
time="2019-03-22T01:10:06Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:10:06Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (730.987µs)" job=default.test-tfjob
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
time="2019-03-22T01:10:06Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:10:06Z" level=info msg="Started workers"
time="2019-03-22T01:10:06Z" level=info msg="Shutting down workers"
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:10:06Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:10:06Z" level=info msg="Started workers"
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
time="2019-03-22T01:10:06Z" level=info msg="Shutting down workers"
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0322 01:10:06.392339    9833 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc42098a2c0)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4205daee0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:10:06Z" level=info msg="Creating Job controller"
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-22T01:10:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.911s	coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow [build failed]
exit status 2: warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/version
# github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow
pkg/controller.v1beta2/tensorflow/job_test.go:672:10: no new variables on left side of :=

travis_time:end:271f93bc:start=1553216963034745032,finish=1553217008161483499,duration=45126738467
TravisBuddy Request Identifier: 9c38c730-4c42-11e9-9767-c9e36d0d3911

@TravisBuddy
Copy link

Travis tests have failed

Hey @ChanYiLin,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.048s
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.053s
=== RUN   TestValidateBetaOneTFJobSpec
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 2.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.049s
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 1.0% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.052s
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 4.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.069s
=== RUN   TestNormalPath
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
--- PASS: TestDeletePodsAndServices (0.00s)
=== RUN   TestCleanupTFJob
--- PASS: TestCleanupTFJob (2.00s)
=== RUN   TestAddPod
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 29.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.910s
=== RUN   TestNormalPath
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
--- PASS: TestCleanupTFJob (2.00s)
=== RUN   TestActiveDeadlineSeconds
--- FAIL: TestActiveDeadlineSeconds (2.00s)
	job_test.go:674: 4 workers and 2 ps is running, ActiveDeadlineSeconds unset: unexpected error when updating status of the jobs Put http://localhost/apis/kubeflow.org/v1beta2/namespaces/default/tfjobs/test-tfjob/status: dial tcp 127.0.0.1:80: getsockopt: connection refused
	job_test.go:674: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected error when updating status of the jobs Put http://localhost/apis/kubeflow.org/v1beta2/namespaces/default/tfjobs/test-tfjob/status: dial tcp 127.0.0.1:80: getsockopt: connection refused
	job_test.go:692: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected number of pod deletes.  Expected 6, saw 0
	job_test.go:695: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected number of service deletes.  Expected 6, saw 0
=== RUN   TestAddPod
nternalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
exit status 1
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow	4.919s
exit status 1: warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/version
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:18Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:44:18Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:44:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:44:18Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:44:18Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (2.397571ms)" job=default.test-tfjob
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:18Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (316.035µs)" job=default.test-tfjob
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:18Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (339.153µs)" job=default.test-tfjob
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:18Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (247.317µs)" job=default.test-tfjob
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:18Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:44:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:44:18Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.124207ms)" job=default.test-tfjob
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:18Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:44:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:44:18Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.138891ms)" job=default.test-tfjob
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:18Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:44:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:44:18Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (464.049µs)" job=default.test-tfjob
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:18Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0322 01:44:18.219362    5959 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc4203a1310), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc420329080), "PS":(*v1beta2.ReplicaSpec)(0xc420328dc0)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"PS":(*v1beta2.ReplicaStatus)(0xc4202f3f70), "Worker":(*v1beta2.ReplicaStatus)(0xc42001a280)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobSucceeded' 'TFJob test-tfjob successfully completed.'
time="2019-03-22T01:44:18Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (839.414µs)" job=default.test-tfjob
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:18Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:44:18Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:44:18Z" level=info msg="Started workers"
time="2019-03-22T01:44:18Z" level=info msg="Shutting down workers"
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:18Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:44:18Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:44:18Z" level=info msg="Started workers"
time="2019-03-22T01:44:18Z" level=info msg="Shutting down workers"
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:18Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:44:18Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:18Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (323.008µs)" job=default.test-tfjob
time="2019-03-22T01:44:18Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:44:18Z" level=info msg="Started workers"
time="2019-03-22T01:44:18Z" level=info msg="Shutting down workers"
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:18Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (297.157µs)" job=default.test-tfjob
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:18Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (260.438µs)" job=default.test-tfjob
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:18Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (787.284µs)" job=default.test-tfjob
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:18Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:18Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (366.737µs)" job=default.test-tfjob
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:18Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (202.9µs)" job=default.test-tfjob
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:18Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:44:18Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (189.237µs)" job=default.test-tfjob
time="2019-03-22T01:44:18Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:18Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:20Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:44:20Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:44:20Z" level=info msg="Ignoring inaReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:22Z" level=info msg="Job with ActiveDeadlineSeconds will sync after 2 seconds" job=default.test-tfjob uid=
time="2019-03-22T01:44:22Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:22Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (604.124µs)" job=default.test-tfjob
time="2019-03-22T01:44:22Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:22Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:22Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:44:22Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:44:22Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:44:22Z" level=info msg="Started workers"
time="2019-03-22T01:44:22Z" level=info msg="Shutting down workers"
time="2019-03-22T01:44:22Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:22Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:22Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:44:22Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:44:22Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:44:22Z" level=info msg="Ignoring inactive pod default/worker-0 in state Failed, deletion time <nil>"
time="2019-03-22T01:44:22Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0322 01:44:22.945052    5959 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc4208ba880), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc4208b6840)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc420415590)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-22T01:44:22Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:44:22Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0322 01:44:22.945167    5959 event.go:259] Couldime="2019-03-22T01:44:22Z" level=info msg="Started workers"
time="2019-03-22T01:44:22Z" level=info msg="Shutting down workers"
time="2019-03-22T01:44:22Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:22Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:22Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:44:22Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:44:23Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:44:23Z" level=info msg="Started workers"
time="2019-03-22T01:44:23Z" level=info msg="Shutting down workers"
time="2019-03-22T01:44:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0322 01:44:23.048203    5959 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc4209de2c0)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc4209ee740)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
time="2019-03-22T01:44:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:44:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-22T01:44:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=

2nd Build

View build log

goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.032s	coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.030s	coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestValidateBetaOneTFJobSpec
time="2019-03-22T01:18:45Z" level=warning msg="Image is undefined in the container"
time="2019-03-22T01:18:45Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
time="2019-03-22T01:18:45Z" level=warning msg="Image is undefined in the container"
time="2019-03-22T01:18:45Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.031s	coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.023s	coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
time="2019-03-22T01:19:04Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
time="2019-03-22T01:19:04Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.060s	coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestNormalPath
time="2019-03-22T01:19:11Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:11Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:11Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.947684ms)" job=default.test-tfjob
time="2019-03-22T01:19:11Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:11Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:11Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:11Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:11Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:11Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:11Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:11Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:11Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (636.501µs)" job=default.test-tfjob
time="2019-03-22T01:19:11Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:11Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:11Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:11Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:11Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:11Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:11Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (487.193µs)" job=default.test-tfjob
time="2019-03-22T01:19:11Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:11Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:11Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (346.959µs)" job=default.test-tfjob
time="2019-03-22T01:19:11Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:11Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:11Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:11Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:11Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:11Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:11Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (467.499µs)" job=default.test-tfjob
time="2019-03-22T01:19:11Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:11Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:11Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0322 01:19:11.540978    9816 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc42005bb60), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"PS":(*v1beta1.ReplicaSpec)(0xc42075e000), "Worker":(*v1beta1.ReplicaSpec)(0xc42075e2c0)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"PS":(*v1beta1.ReplicaStatus)(0xc420135080), "Worker":(*v1beta1.ReplicaStatus)(0xc4201350b0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobSucceeded' 'TFJob test-tfjob successfully completed.'
time="2019-03-22T01:19:11Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (2.095646ms)" job=default.test-tfjob
time="2019-03-22T01:19:11Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:11Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:11Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:11Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:11Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (367.774µs)" job=default.test-tfjob
time="2019-03-22T01:19:11Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:11Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:11Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:11Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:11Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:11Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:11Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:11Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:11Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:11Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:11Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:11Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:11Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:11Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:11Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:11Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (808.297µs)" job=default.test-tfjob
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
time="2019-03-22T01:19:11Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:11Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:11Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:19:11Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:19:11Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:19:11Z" level=info msg="Started workers"
time="2019-03-22T01:19:12Z" level=info msg="Shutting down workers"
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
time="2019-03-22T01:19:12Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:12Z" level=info msg="Creating Job controller"
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
time="2019-03-22T01:19:12Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:12Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:12Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:19:12Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:19:12Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-22T01:19:12Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:19:12Z" level=info msg="Started workers"
--- PASS: TestAddTFJob (0.10s)
time="2019-03-22T01:19:12Z" level=info msg="Shutting down workers"
=== RUN   TestCopyLabelsAndAnnotation
time="2019-03-22T01:19:12Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:12Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:12Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:19:12Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:19:12Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:12Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:12Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:12Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:12Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (293.539µs)" job=default.test-tfjob
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
time="2019-03-22T01:19:12Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:19:12Z" level=info msg="Started workers"
time="2019-03-22T01:19:12Z" level=info msg="Shutting down workers"
time="2019-03-22T01:19:12Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:12Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:12Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:12Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (259.779µs)" job=default.test-tfjob
time="2019-03-22T01:19:12Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:12Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:12Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:12Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (238.865µs)" job=default.test-tfjob
time="2019-03-22T01:19:12Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:12Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:12Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:12Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (221.38µs)" job=default.test-tfjob
time="2019-03-22T01:19:12Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:12Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:12Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:12Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (184.786µs)" job=default.test-tfjob
--- PASS: TestDeletePodsAndServices (0.00s)
=== RUN   TestCleanupTFJob
time="2019-03-22T01:19:12Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:12Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:12Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:12Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (212.342µs)" job=default.test-tfjob
time="2019-03-22T01:19:12Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:12Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:12Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:12Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (338.827µs)" job=default.test-tfjob
time="2019-03-22T01:19:12Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:12Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (428.864µs)" job=default.test-tfjob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestAddPod
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:19:14Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:19:14Z" level=info msg="Started workers"
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
time="2019-03-22T01:19:14Z" level=info msg="Shutting down workers"
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:19:14Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0322 01:19:14.268325    9816 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc42041a320), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc4208e1340)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4201353f0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-22T01:19:14Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0322 01:19:14.268491    9816 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc42041a320), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc4208e1340)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4201353f0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'TFJobRestarting' 'TFJob test-tfjob is restarting because 1 Worker replica(s) failed.'
time="2019-03-22T01:19:14Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:14Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (542.401µs)" job=default.test-tfjob
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
time="2019-03-22T01:19:14Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:19:14Z" level=info msg="Started workers"
time="2019-03-22T01:19:14Z" level=info msg="Shutting down workers"
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:19:14Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:19:14Z" level=info msg="Started workers"
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
time="2019-03-22T01:19:14Z" level=info msg="Shutting down workers"
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0322 01:19:14.371529    9816 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc42097cb00)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4205af0c0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:14Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-22T01:19:14Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.911s	coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestNormalPath
time="2019-03-22T01:19:20Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:20Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:20Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:20Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:20Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:20Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:20Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.896487ms)" job=default.test-tfjob
time="2019-03-22T01:19:20Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:20Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:20Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:20Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:20Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:20Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (479.166µs)" job=default.test-tfjob
time="2019-03-22T01:19:20Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:20Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:20Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (314.795µs)" job=default.test-tfjob
time="2019-03-22T01:19:20Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:20Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:20Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:20Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:20Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (582.353µs)" job=default.test-tfjob
time="2019-03-22T01:19:20Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:20Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:20Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:20Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:20Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:20Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:20Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:20Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0322 01:19:20.566133    9924 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc42005ae00), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"PS":(*v1beta2.ReplicaSpec)(0xc4204982c0), "Worker":(*v1beta2.ReplicaSpec)(0xc420498580)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"PS":(*v1beta2.ReplicaStatus)(0xc4204e5c30), "Worker":(*v1beta2.ReplicaStatus)(0xc4204e5c60)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobSucceeded' 'TFJob test-tfjob successfully completed.'
time="2019-03-22T01:19:20Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (2.399919ms)" job=default.test-tfjob
time="2019-03-22T01:19:20Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:20Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:20Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:20Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (479.103µs)" job=default.test-tfjob
time="2019-03-22T01:19:20Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:20Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:20Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:20Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:20Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-22T01:19:20Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:20Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:20Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:20Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:20Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:20Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:20Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:20Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (794.172µs)" job=default.test-tfjob
time="2019-03-22T01:19:20Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:20Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:20Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:20Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (405.562µs)" job=default.test-tfjob
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
time="2019-03-22T01:19:20Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:20Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:20Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:19:20Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:19:20Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:19:20Z" level=info msg="Started workers"
time="2019-03-22T01:19:21Z" level=info msg="Shutting down workers"
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
time="2019-03-22T01:19:21Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:21Z" level=info msg="Creating Job controller"
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
time="2019-03-22T01:19:21Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:21Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:21Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-22T01:19:21Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:19:21Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:19:21Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:19:21Z" level=info msg="Started workers"
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
time="2019-03-22T01:19:21Z" level=info msg="Shutting down workers"
time="2019-03-22T01:19:21Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:21Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:21Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:19:21Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:19:21Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:21Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:21Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:21Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:21Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (426.835µs)" job=default.test-tfjob
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
time="2019-03-22T01:19:21Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:19:21Z" level=info msg="Started workers"
time="2019-03-22T01:19:21Z" level=info msg="Shutting down workers"
time="2019-03-22T01:19:21Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:21Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:21Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:21Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (328.934µs)" job=default.test-tfjob
time="2019-03-22T01:19:21Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:21Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:21Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:21Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (276.166µs)" job=default.test-tfjob
time="2019-03-22T01:19:21Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:21Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:21Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:21Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:21Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:21Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:21Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:21Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:21Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:21Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (304.971µs)" job=default.test-tfjob
time="2019-03-22T01:19:21Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:21Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:21Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:21Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:21Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:21Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:21Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:21Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:21Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:21Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (340.022µs)" job=default.test-tfjob
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
time="2019-03-22T01:19:21Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:21Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:21Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:21Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (224.978µs)" job=default.test-tfjob
time="2019-03-22T01:19:21Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:21Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:21Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:21Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (437.233µs)" job=default.test-tfjob
time="2019-03-22T01:19:21Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:21Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:23Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:23Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:23Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:23Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:23Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:23Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:23Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-22T01:19:23Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (847.165µs)" job=default.test-tfjob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestActiveDeadlineSeconds
time="2019-03-22T01:19:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:23Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:23Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:23Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (442.699µs)" job=default.test-tfjob
time="2019-03-22T01:19:23Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:23Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="Job with ActiveDeadlineSeconds will sync after 2 seconds" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (552.165µs)" job=default.test-tfjob
--- FAIL: TestActiveDeadlineSeconds (2.00s)
	job_test.go:674: 4 workers and 2 ps is running, ActiveDeadlineSeconds unset: unexpected error when updating status of the jobs Put http://localhost/apis/kubeflow.org/v1beta2/namespaces/default/tfjobs/test-tfjob/status: dial tcp 127.0.0.1:80: connect: connection refused
	job_test.go:674: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected error when updating status of the jobs Put http://localhost/apis/kubeflow.org/v1beta2/namespaces/default/tfjobs/test-tfjob/status: dial tcp 127.0.0.1:80: connect: connection refused
	job_test.go:692: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected number of pod deletes.  Expected 6, saw 0
	job_test.go:695: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected number of service deletes.  Expected 6, saw 0
=== RUN   TestAddPod
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:19:25Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:19:25Z" level=info msg="Started workers"
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
time="2019-03-22T01:19:25Z" level=info msg="Shutting down workers"
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="Ignoring inactive pod default/worker-0 in state Failed, deletion time <nil>"
time="2019-03-22T01:19:25Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0322 01:19:25.302052    9924 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc42005ba20), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc4204d9600)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc4204104b0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-22T01:19:25Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0322 01:19:25.302171    9924 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc42005ba20), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc4204d9600)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc4204104b0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'TFJobRestarting' 'TFJob test-tfjob is restarting because 1 Worker replica(s) failed.'
time="2019-03-22T01:19:25Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-22T01:19:25Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (487.207µs)" job=default.test-tfjob
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:19:25Z" level=info msg="Starting TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-22T01:19:25Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:19:25Z" level=info msg="Started workers"
time="2019-03-22T01:19:25Z" level=info msg="Shutting down workers"
time="2019-03-22T01:19:25Z" level=info msg="Starting 1 workers"
time="2019-03-22T01:19:25Z" level=info msg="Started workers"
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
time="2019-03-22T01:19:25Z" level=info msg="Shutting down workers"
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0322 01:19:25.405413    9924 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc4202702c0)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc4201c2c80)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="Creating TFJob controller"
time="2019-03-22T01:19:25Z" level=info msg="Creating Job controller"
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-22T01:19:25Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
--- PASS: TestStatus (0.02s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
FAIL
coverage: 42.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow	4.917s
exit status 1: warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/version

travis_time:end:035a70d1:start=1553217507581800405,finish=1553217565439749181,duration=57857948776
TravisBuddy Request Identifier: 054ed3d0-4c44-11e9-9767-c9e36d0d3911

@TravisBuddy
Copy link

Travis tests have failed

Hey @ChanYiLin,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.044s
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.047s
=== RUN   TestValidateBetaOneTFJobSpec
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 2.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.044s
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 1.0% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.043s
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 4.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.063s
=== RUN   TestNormalPath
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
--- PASS: TestCleanupTFJob (2.00s)
=== RUN   TestAddPod
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 29.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.903s
=== RUN   TestNormalPath
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
--- PASS: TestDeletePodsAndServices (0.00s)
=== RUN   TestCleanupTFJob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestActiveDeadlineSeconds
--- FAIL: TestActiveDeadlineSeconds (2.00s)
	job_test.go:685: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected number of pod deletes.  Expected 6, saw 0
	job_test.go:688: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected number of service deletes.  Expected 6, saw 0
=== RUN   TestAddPod
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
FAIL
coverage: 32.0% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
exit status 1
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow	4.907s
exit status 1: aType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (2.205611ms)" job=default.test-tfjob
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (318.167µs)" job=default.test-tfjob
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="TFsfully completed.'
time="2019-03-26T17:13:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (797.382µs)" job=default.test-tfjob
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (269.857µs)" job=default.test-tfjob
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (314.669µs)" job=default.test-tfjob
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="ob=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (490.329µs)" job=default.test-tfjob
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="Needlevel=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (304.58µs)" job=default.test-tfjob
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (320.78µs)" job=default.test-tfjob
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (197.962µs)" job=default.test-tfjob
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (454.898µs)" job=default.test-tfjob
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:15Z" level=info msg="Shutting down workers"
time="2019-03-26T17:13:15Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:13:15Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:13:15Z" level=info msg="Started workers"
time="2019-03-26T17:13:15Z" level=info msg="Shutting down workers"
time="2019-03-26T17:13:17Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:17Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:17Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:17Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:17Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:17Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:17Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:17Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (552.113µs)" job=default.test-tfjob
time="2019-03-26T17:13:17Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:17Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:17Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:17Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:17Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:17Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:19Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:19Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:19Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:19Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:13:19Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:13:20Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:13:20Z" level=info msg="Started workers"
time="2019-03-26T17:13:20Z" level=info msg="Shutting down workers"
time="2019-03-26T17:13:20Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:20Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:20Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="Ignoring inactive pod default/worker-0 in state Failed, deletion time <nil>"
time="2019-03-26T17:13:20Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0326 17:13:20.043226    6106 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc42026c100), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc420a25340)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc42054d580)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-26T17:13:20Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0326 17:13:20.043328    6106 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc42026c100), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc420a25340)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc42054d580)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'TFJobRestarting' 'TFJob test-tfjob is restarting because 1 Worker replica(s) failed.'
time="2019-03-26T17:13:20Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:20Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (445.138µs)" job=default.test-tfjob
time="2019-03-26T17:13:20Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:20Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:20Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:13:20Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:13:20Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:13:20Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:13:20Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:13:20Z" level=info msg="Started workers"
time="2019-03-26T17:13:20Z" level=info msg="Shutting down workers"
time="2019-03-26T17:13:20Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:13:20Z" level=info msg="Started workers"
time="2019-03-26T17:13:20Z" level=info msg="Shutting down workers"
time="2019-03-26T17:13:20Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:20Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0326 17:13:20.146018    6106 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc420974000)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xZ" level=info msg="Creating Job controller"
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:20Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:20Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:20Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:20Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:20Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:20Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:20Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:20Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:20Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:20Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:20Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:20Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T17:13:20Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=

2nd Build

View build log

goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.031s	coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.029s	coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestValidateBetaOneTFJobSpec
time="2019-03-26T16:49:34Z" level=warning msg="Image is undefined in the container"
time="2019-03-26T16:49:34Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
time="2019-03-26T16:49:34Z" level=warning msg="Image is undefined in the container"
time="2019-03-26T16:49:34Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.031s	coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.021s	coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
time="2019-03-26T16:49:53Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
time="2019-03-26T16:49:53Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.055s	coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestNormalPath
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:00Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:00Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.896673ms)" job=default.test-tfjob
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:00Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (578.304µs)" job=default.test-tfjob
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (421.833µs)" job=default.test-tfjob
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:00Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:00Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (465.649µs)" job=default.test-tfjob
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0326 16:50:00.190250    9746 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc42005a870), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"PS":(*v1beta1.ReplicaSpec)(0xc4204f22c0), "Worker":(*v1beta1.ReplicaSpec)(0xc4204f2580)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"PS":(*v1beta1.ReplicaStatus)(0xc420375b80), "Worker":(*v1beta1.ReplicaStatus)(0xc420375bb0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobSucceeded' 'TFJob test-tfjob successfully completed.'
time="2019-03-26T16:50:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (2.166314ms)" job=default.test-tfjob
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (217.341µs)" job=default.test-tfjob
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:00Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:00Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:00Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:00Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (590.283µs)" job=default.test-tfjob
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (323.199µs)" job=default.test-tfjob
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:00Z" level=info msg="Starting TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T16:50:00Z" level=info msg="Starting 1 workers"
time="2019-03-26T16:50:00Z" level=info msg="Started workers"
time="2019-03-26T16:50:00Z" level=info msg="Shutting down workers"
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:00Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="Starting TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T16:50:00Z" level=info msg="Starting 1 workers"
time="2019-03-26T16:50:00Z" level=info msg="Started workers"
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
time="2019-03-26T16:50:00Z" level=info msg="Shutting down workers"
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:00Z" level=info msg="Starting TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T16:50:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (501.16µs)" job=default.test-tfjob
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
time="2019-03-26T16:50:00Z" level=info msg="Starting 1 workers"
time="2019-03-26T16:50:00Z" level=info msg="Started workers"
time="2019-03-26T16:50:00Z" level=info msg="Shutting down workers"
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (334.994µs)" job=default.test-tfjob
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (234.836µs)" job=default.test-tfjob
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (256.972µs)" job=default.test-tfjob
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (227.819µs)" job=default.test-tfjob
--- PASS: TestDeletePodsAndServices (0.00s)
=== RUN   TestCleanupTFJob
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (3.001257ms)" job=default.test-tfjob
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:00Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:00Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (297.319µs)" job=default.test-tfjob
time="2019-03-26T16:50:00Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:00Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:02Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:02Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (622.221µs)" job=default.test-tfjob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestAddPod
time="2019-03-26T16:50:02Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:02Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:02Z" level=info msg="Starting TFJob controller"
time="2019-03-26T16:50:02Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T16:50:02Z" level=info msg="Starting 1 workers"
time="2019-03-26T16:50:02Z" level=info msg="Started workers"
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
time="2019-03-26T16:50:02Z" level=info msg="Shutting down workers"
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
time="2019-03-26T16:50:02Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:02Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:02Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:02Z" level=info msg="Starting TFJob controller"
time="2019-03-26T16:50:02Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T16:50:02Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0326 16:50:02.919899    9746 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc42034b9e0), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc420345080)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4204ba1e0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-26T16:50:02Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:02Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0326 16:50:02.920012    9746 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc42034b9e0), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc420345080)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4204ba1e0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'TFJobRestarting' 'TFJob test-tfjob is restarting because 1 Worker replica(s) failed.'
time="2019-03-26T16:50:02Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:02Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (634.402µs)" job=default.test-tfjob
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
time="2019-03-26T16:50:02Z" level=info msg="Starting 1 workers"
time="2019-03-26T16:50:02Z" level=info msg="Started workers"
time="2019-03-26T16:50:02Z" level=info msg="Shutting down workers"
time="2019-03-26T16:50:02Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:02Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:02Z" level=info msg="Starting TFJob controller"
time="2019-03-26T16:50:02Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T16:50:03Z" level=info msg="Starting 1 workers"
time="2019-03-26T16:50:03Z" level=info msg="Started workers"
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
time="2019-03-26T16:50:03Z" level=info msg="Shutting down workers"
time="2019-03-26T16:50:03Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:03Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0326 16:50:03.022880    9746 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc420131080)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4207e7af0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
time="2019-03-26T16:50:03Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:03Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:03Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:03Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:03Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:03Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:03Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:03Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:03Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:03Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:03Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:03Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:03Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:03Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:03Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:03Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:03Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T16:50:03Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.907s	coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestNormalPath
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:09Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:09Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:09Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (2.537884ms)" job=default.test-tfjob
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:09Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (365.326µs)" job=default.test-tfjob
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:09Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:09Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:09Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (434.38µs)" job=default.test-tfjob
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:09Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0326 16:50:09.014090    9855 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc42031fd30), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"PS":(*v1beta2.ReplicaSpec)(0xc4203f2dc0), "Worker":(*v1beta2.ReplicaSpec)(0xc4203f3080)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc42046cae0), "PS":(*v1beta2.ReplicaStatus)(0xc42046ca10)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobSucceeded' 'TFJob test-tfjob successfully completed.'
time="2019-03-26T16:50:09Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (973.633µs)" job=default.test-tfjob
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:09Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (683.786µs)" job=default.test-tfjob
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:09Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:09Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:09Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:09Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:09Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (879.846µs)" job=default.test-tfjob
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:09Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (406.839µs)" job=default.test-tfjob
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:09Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:09Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T16:50:09Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (648.892µs)" job=default.test-tfjob
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:09Z" level=info msg="Starting TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T16:50:09Z" level=info msg="Starting 1 workers"
time="2019-03-26T16:50:09Z" level=info msg="Started workers"
time="2019-03-26T16:50:09Z" level=info msg="Shutting down workers"
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:09Z" level=info msg="Starting TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T16:50:09Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="Starting 1 workers"
time="2019-03-26T16:50:09Z" level=info msg="Started workers"
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
time="2019-03-26T16:50:09Z" level=info msg="Shutting down workers"
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:09Z" level=info msg="Starting TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T16:50:09Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:09Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (425.712µs)" job=default.test-tfjob
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
time="2019-03-26T16:50:09Z" level=info msg="Starting 1 workers"
time="2019-03-26T16:50:09Z" level=info msg="Started workers"
time="2019-03-26T16:50:09Z" level=info msg="Shutting down workers"
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:09Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (266.045µs)" job=default.test-tfjob
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:09Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (301.777µs)" job=default.test-tfjob
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:09Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (320.464µs)" job=default.test-tfjob
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:09Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:09Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (319.474µs)" job=default.test-tfjob
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:09Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (223.902µs)" job=default.test-tfjob
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:09Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:09Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (233.485µs)" job=default.test-tfjob
time="2019-03-26T16:50:09Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:09Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:11Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:11Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:11Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:11Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:11Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:11Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:11Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T16:50:11Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (583.241µs)" job=default.test-tfjob
--- PASS: TestCleanupTFJob (2.00s)
=== RUN   TestActiveDeadlineSeconds
time="2019-03-26T16:50:11Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:11Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:11Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:11Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:11Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:11Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:11Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
--- FAIL: TestActiveDeadlineSeconds (2.00s)
	job_test.go:685: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected number of pod deletes.  Expected 6, saw 0
	job_test.go:688: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected number of service deletes.  Expected 6, saw 0
=== RUN   TestAddPod
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="Starting TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T16:50:13Z" level=info msg="Starting 1 workers"
time="2019-03-26T16:50:13Z" level=info msg="Started workers"
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
time="2019-03-26T16:50:13Z" level=info msg="Shutting down workers"
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="Ignoring inactive pod default/worker-0 in state Failed, deletion time <nil>"
time="2019-03-26T16:50:13Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0326 16:50:13.746087    9855 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc42027f450), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc4200d7080)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc4204410c0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-26T16:50:13Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0326 16:50:13.746196    9855 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc42027f450), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc4200d7080)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc4204410c0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'TFJobRestarting' 'TFJob test-tfjob is restarting because 1 Worker replica(s) failed.'
time="2019-03-26T16:50:13Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T16:50:13Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (432.066µs)" job=default.test-tfjob
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
time="2019-03-26T16:50:13Z" level=info msg="Starting TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T16:50:13Z" level=info msg="Starting 1 workers"
time="2019-03-26T16:50:13Z" level=info msg="Started workers"
time="2019-03-26T16:50:13Z" level=info msg="Shutting down workers"
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="Starting TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T16:50:13Z" level=info msg="Starting 1 workers"
time="2019-03-26T16:50:13Z" level=info msg="Started workers"
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
time="2019-03-26T16:50:13Z" level=info msg="Shutting down workers"
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0326 16:50:13.849064    9855 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc4208782c0)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc4203e0b60)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="Creating TFJob controller"
time="2019-03-26T16:50:13Z" level=info msg="Creating Job controller"
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T16:50:13Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
FAIL
coverage: 42.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow	4.910s
exit status 1: warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/version

travis_time:end:041b7133:start=1553618957583219014,finish=1553619013882227303,duration=56299008289
TravisBuddy Request Identifier: 749c6960-4fea-11e9-bd36-ef1a269a9fbb

@TravisBuddy
Copy link

Travis tests have failed

Hey @ChanYiLin,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.049s
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.047s
=== RUN   TestValidateBetaOneTFJobSpec
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 2.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.045s
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 1.0% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.046s
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 4.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.065s
=== RUN   TestNormalPath
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
--- PASS: TestCleanupTFJob (2.00s)
=== RUN   TestAddPod
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 29.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.905s
=== RUN   TestNormalPath
--- FAIL: TestNormalPath (0.01s)
exit status 2
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow	0.056s
exit status 1: warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/version
time="2019-03-26T17:37:08Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:37:08Z" level=info msg="Creating Job controller"
time="2019-03-26T17:37:08Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:37:08Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:37:08Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:37:08Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:37:08Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:37:08Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:37:08Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:37:08Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.900716ms)" job=default.test-tfjob
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x120bc32]

goroutine 9 [running]:
testing.tRunner.func1(0xc4204c61e0)
	/home/travis/.gimme/versions/go1.9.linux.amd64/src/testing/testing.go:711 +0x2d2
panic(0x1390fe0, 0x1f82ef0)
	/home/travis/.gimme/versions/go1.9.linux.amd64/src/runtime/panic.go:491 +0x283
github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow.(*TFController).reconcileTFJobs(0xc42012f200, 0xc4200c2300, 0xc4200c2300, 0xc420372ee8)
	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow/_test/_obj_test/controller.go:389 +0x742
github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow.(*TFController).syncTFJob(0xc42012f200, 0xc420372ee0, 0x12, 0x0, 0x0, 0x0)
	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow/_test/_obj_test/controller.go:318 +0x599
github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow.TestNormalPath(0xc4204c61e0)
	/home/travis/gopath/src/github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow/controller_test.go:269 +0x1774
testing.tRunner(0xc4204c61e0, 0x15ac7c0)
	/home/travis/.gimme/versions/go1.9.linux.amd64/src/testing/testing.go:746 +0xd0
created by testing.(*T).Run
	/home/travis/.gimme/versions/go1.9.linux.amd64/src/testing/testing.go:789 +0x2de

2nd Build

View build log

goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.029s	coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.031s	coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestValidateBetaOneTFJobSpec
time="2019-03-26T17:12:49Z" level=warning msg="Image is undefined in the container"
time="2019-03-26T17:12:49Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
time="2019-03-26T17:12:49Z" level=warning msg="Image is undefined in the container"
time="2019-03-26T17:12:49Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.030s	coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.022s	coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
time="2019-03-26T17:13:08Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
time="2019-03-26T17:13:08Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.062s	coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestNormalPath
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.685585ms)" job=default.test-tfjob
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (594.625µs)" job=default.test-tfjob
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (459.494µs)" job=default.test-tfjob
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (435.115µs)" job=default.test-tfjob
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:13:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (550.484µs)" job=default.test-tfjob
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0326 17:13:15.496453    9759 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc4205dc180), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"PS":(*v1beta1.ReplicaSpec)(0xc420758000), "Worker":(*v1beta1.ReplicaSpec)(0xc4207582c0)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"PS":(*v1beta1.ReplicaStatus)(0xc4206f9150), "Worker":(*v1beta1.ReplicaStatus)(0xc4206f9180)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobSucceeded' 'TFJob test-tfjob successfully completed.'
time="2019-03-26T17:13:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.727056ms)" job=default.test-tfjob
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (311.559µs)" job=default.test-tfjob
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:15Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:13:15Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:13:15Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:13:15Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:13:15Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (656.342µs)" job=default.test-tfjob
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
time="2019-03-26T17:13:15Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:15Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:13:15Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:13:15Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:13:15Z" level=info msg="Started workers"
time="2019-03-26T17:13:16Z" level=info msg="Shutting down workers"
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
time="2019-03-26T17:13:16Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:16Z" level=info msg="Creating Job controller"
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
time="2019-03-26T17:13:16Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:16Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:16Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-26T17:13:16Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:13:16Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:13:16Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:13:16Z" level=info msg="Started workers"
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
time="2019-03-26T17:13:16Z" level=info msg="Shutting down workers"
time="2019-03-26T17:13:16Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:16Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:16Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:13:16Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:13:16Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:16Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:16Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:16Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:16Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (294.258µs)" job=default.test-tfjob
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
time="2019-03-26T17:13:16Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:13:16Z" level=info msg="Started workers"
time="2019-03-26T17:13:16Z" level=info msg="Shutting down workers"
time="2019-03-26T17:13:16Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:16Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:16Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:16Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (245.547µs)" job=default.test-tfjob
time="2019-03-26T17:13:16Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:16Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:16Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:16Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (243.053µs)" job=default.test-tfjob
time="2019-03-26T17:13:16Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:16Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:16Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:16Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (215.814µs)" job=default.test-tfjob
time="2019-03-26T17:13:16Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:16Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:16Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:16Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (239.867µs)" job=default.test-tfjob
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
time="2019-03-26T17:13:16Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:16Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:16Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:16Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (375.898µs)" job=default.test-tfjob
time="2019-03-26T17:13:16Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:16Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:16Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:16Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (332.028µs)" job=default.test-tfjob
time="2019-03-26T17:13:16Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:16Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (632.804µs)" job=default.test-tfjob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestAddPod
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:13:18Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:13:18Z" level=info msg="Started workers"
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
time="2019-03-26T17:13:18Z" level=info msg="Shutting down workers"
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:13:18Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0326 17:13:18.222949    9759 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc4202b1a10), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc4208a4840)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4206f9e70)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-26T17:13:18Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0326 17:13:18.223065    9759 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc4202b1a10), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc4208a4840)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4206f9e70)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'TFJobRestarting' 'TFJob test-tfjob is restarting because 1 Worker replica(s) failed.'
time="2019-03-26T17:13:18Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:13:18Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (474.897µs)" job=default.test-tfjob
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
time="2019-03-26T17:13:18Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:13:18Z" level=info msg="Started workers"
time="2019-03-26T17:13:18Z" level=info msg="Shutting down workers"
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:13:18Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:13:18Z" level=info msg="Started workers"
--- PASS: TestAddService (0.10s)
time="2019-03-26T17:13:18Z" level=info msg="Shutting down workers"
=== RUN   TestFailed
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0326 17:13:18.326325    9759 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc4202722c0)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc420308cc0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:18Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T17:13:18Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
--- PASS: TestStatus (0.02s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.907s	coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestNormalPath
time="2019-03-26T17:13:24Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:13:24Z" level=info msg="Creating Job controller"
time="2019-03-26T17:13:24Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:13:24Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:24Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:24Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:24Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:24Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:24Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:13:24Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.731437ms)" job=default.test-tfjob
--- FAIL: TestNormalPath (0.01s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x11241e9]

goroutine 23 [running]:
testing.tRunner.func1(0xc4204ba1e0)
	/home/travis/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:742 +0x29d
panic(0x129d500, 0x1e4b9d0)
	/home/travis/.gimme/versions/go1.10.linux.amd64/src/runtime/panic.go:505 +0x229
github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow.(*TFController).reconcileTFJobs(0xc42019d440, 0xc4200ad080, 0xc4200ad080, 0xc420042ee8)
	/home/travis/gopath/src/github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow/controller.go:384 +0x699
github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow.(*TFController).syncTFJob(0xc42019d440, 0xc420042ee0, 0x12, 0x0, 0x0, 0x0)
	/home/travis/gopath/src/github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow/controller.go:321 +0x4f2
github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow.TestNormalPath(0xc4204ba1e0)
	/home/travis/gopath/src/github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow/controller_test.go:269 +0x166e
testing.tRunner(0xc4204ba1e0, 0x14bb628)
	/home/travis/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:777 +0xd0
created by testing.(*T).Run
	/home/travis/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:824 +0x2e0
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow	0.057s
exit status 1: warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/version

travis_time:end:19991b60:start=1553620352156744181,finish=1553620404444574388,duration=52287830207
TravisBuddy Request Identifier: c7f90520-4fed-11e9-bd36-ef1a269a9fbb

@TravisBuddy
Copy link

Travis tests have failed

Hey @ChanYiLin,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.047s
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.045s
=== RUN   TestValidateBetaOneTFJobSpec
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 2.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.046s
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 1.0% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.046s
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 4.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.063s
=== RUN   TestNormalPath
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestAddPod
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 29.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.905s
=== RUN   TestNormalPath
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
--- PASS: TestDeletePodsAndServices (0.00s)
=== RUN   TestCleanupTFJob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestActiveDeadlineSeconds
--- FAIL: TestActiveDeadlineSeconds (2.00s)
	job_test.go:680: >>>>>> TIME AFTER SYNC: <nil>
	job_test.go:681: >>>>>> TIME AFTER SYNC: 2019-03-26 17:45:40.7368759 +0000 UTC m=+2.678349615
	job_test.go:687: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected number of pod deletes.  Expected 6, saw 0
	job_test.go:690: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected number of service deletes.  Expected 6, saw 0
=== RUN   TestAddPod
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
FAIL
coverage: 32.1% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.co"Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:45:38Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:45:38Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:38Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0326 17:45:38.108522    6177 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc4204362a0), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"PS":(*v1beta2.ReplicaSpec)(0xc420331b80), "Worker":(*v1beta2.ReplicaSpec)(0xc4202ee000)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc420552e10), "PS":(*v1beta29-03-26T17:45:38Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:38Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:45:38Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:38Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:38Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (337.764µs)" job=default.test-tfjob
time="2019-03-26T17:45:38Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:38Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:38Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:45:38Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:45:38Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:38Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:45:38Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:45:38Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:45:38Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:38Z" level=info msg="need to createZ" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:45:38Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:45:38Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:38Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:45:38Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (201.121µs)" job=default.test-tfjob
time="2019-03-26T17:45:38Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:38Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:38Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:45:38Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:45:38Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:45:38Z" level=info msg="Started workers"
time="2019-03-26T17:45:38Z" level=info msg="Shutting down workers"
time="2019-03-26T17:45:38Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:38Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:38Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:38Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:38Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-26T17:45:38Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:45:38Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-2"2019-03-26T17:45:38Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:45:38Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:45:38Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:45:38Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:45:38Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:45:38Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:45:38Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:45:38Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (282.821µs)" job=default.test-tfjob
time="2019-03-26T17:45:38Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:38Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:38Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:45:38Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (203.985µs)" job=default.test-tfjob
time="2019-03-26T17:45:38Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:38Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:38Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:45:38Z" level=info msg="Finished syncing tfjob \"das active longer than specified deadline  test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg=">>>>>> TFJobs was ExceedsLimit  test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:45:42Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:45:42Z" level=info msg="Started workers"
time="2019-03-26T17:45:42Z" level=info msg="Shutting down workers"
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:45:42Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="Ignoring inactive pod default/worker-0 in state Failed, deletion time <nil>"
time="2019-03-26T17:45:42Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0326 17:45:42.841029    6177 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc420291b70), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc420684000)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc420553a10)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-26T17:45:42Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0326 17:45:42.841136    6177 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:19-03-26T17:45:42Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:45:42Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:45:42Z" level=info msg="Started workers"
time="2019-03-26T17:45:42Z" level=info msg="Shutting down workers"
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0326 17:45:42.943968    6177 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc4203f42c0)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc42051a820)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:45:42Z" level=info msg="Creating Job controller"
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T17:45:42Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=

2nd Build

View build log

goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.036s	coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.028s	coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestValidateBetaOneTFJobSpec
time="2019-03-26T17:21:31Z" level=warning msg="Image is undefined in the container"
time="2019-03-26T17:21:31Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
time="2019-03-26T17:21:31Z" level=warning msg="Image is undefined in the container"
time="2019-03-26T17:21:31Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.030s	coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.019s	coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
time="2019-03-26T17:21:49Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
time="2019-03-26T17:21:49Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.059s	coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestNormalPath
time="2019-03-26T17:21:56Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:56Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:56Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0326 17:21:56.928217    9735 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc42052ed20), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"PS":(*v1beta1.ReplicaSpec)(0xc420342b00), "Worker":(*v1beta1.ReplicaSpec)(0xc420342dc0)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"PS":(*v1beta1.ReplicaStatus)(0xc4201d7260), "Worker":(*v1beta1.ReplicaStatus)(0xc4201d72a0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobSucceeded' 'TFJob test-tfjob successfully completed.'
time="2019-03-26T17:21:56Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (2.314941ms)" job=default.test-tfjob
time="2019-03-26T17:21:56Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:56Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:56Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:56Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:56Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (434.478µs)" job=default.test-tfjob
time="2019-03-26T17:21:56Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:56Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:56Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:56Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:56Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:56Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:56Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:56Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:56Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:56Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:56Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:21:56Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:21:56Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:21:56Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:21:56Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.391911ms)" job=default.test-tfjob
time="2019-03-26T17:21:56Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:56Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:56Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (496.549µs)" job=default.test-tfjob
time="2019-03-26T17:21:56Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:56Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:56Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:56Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:56Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:56Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:56Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:21:56Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:21:56Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (513.309µs)" job=default.test-tfjob
time="2019-03-26T17:21:56Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:56Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:56Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:21:56Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:21:56Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:56Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:56Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (494.453µs)" job=default.test-tfjob
time="2019-03-26T17:21:56Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:56Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:56Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (324.522µs)" job=default.test-tfjob
time="2019-03-26T17:21:56Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:56Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:56Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:21:56Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:21:56Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:56Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:56Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:56Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (519.888µs)" job=default.test-tfjob
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
time="2019-03-26T17:21:56Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:56Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:56Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:21:56Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:21:57Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:21:57Z" level=info msg="Started workers"
time="2019-03-26T17:21:57Z" level=info msg="Shutting down workers"
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
time="2019-03-26T17:21:57Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:57Z" level=info msg="Creating Job controller"
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
time="2019-03-26T17:21:57Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:57Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:57Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:21:57Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:21:57Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-26T17:21:57Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:21:57Z" level=info msg="Started workers"
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
time="2019-03-26T17:21:57Z" level=info msg="Shutting down workers"
time="2019-03-26T17:21:57Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:57Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:57Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:21:57Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:21:57Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:21:57Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:57Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:57Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:57Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (419.615µs)" job=default.test-tfjob
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
time="2019-03-26T17:21:57Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:21:57Z" level=info msg="Started workers"
time="2019-03-26T17:21:57Z" level=info msg="Shutting down workers"
time="2019-03-26T17:21:57Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:57Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:57Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:21:57Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (328.756µs)" job=default.test-tfjob
time="2019-03-26T17:21:57Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:57Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:57Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:21:57Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (281.558µs)" job=default.test-tfjob
time="2019-03-26T17:21:57Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:57Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:57Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:21:57Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (215.395µs)" job=default.test-tfjob
time="2019-03-26T17:21:57Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:57Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:57Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:21:57Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (200.112µs)" job=default.test-tfjob
--- PASS: TestDeletePodsAndServices (0.00s)
=== RUN   TestCleanupTFJob
time="2019-03-26T17:21:57Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:57Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:57Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:21:57Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (315.295µs)" job=default.test-tfjob
time="2019-03-26T17:21:57Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:57Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:57Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:21:57Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (399.55µs)" job=default.test-tfjob
time="2019-03-26T17:21:57Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:57Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (444.488µs)" job=default.test-tfjob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestAddPod
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:21:59Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:21:59Z" level=info msg="Started workers"
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
time="2019-03-26T17:21:59Z" level=info msg="Shutting down workers"
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:21:59Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0326 17:21:59.662577    9735 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc4203c4260), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc4208f9080)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc420a25e00)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-26T17:21:59Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0326 17:21:59.662718    9735 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc4203c4260), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc4208f9080)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc420a25e00)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'TFJobRestarting' 'TFJob test-tfjob is restarting because 1 Worker replica(s) failed.'
time="2019-03-26T17:21:59Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:21:59Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (574.529µs)" job=default.test-tfjob
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
time="2019-03-26T17:21:59Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:21:59Z" level=info msg="Started workers"
time="2019-03-26T17:21:59Z" level=info msg="Shutting down workers"
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:21:59Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:21:59Z" level=info msg="Started workers"
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
time="2019-03-26T17:21:59Z" level=info msg="Shutting down workers"
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0326 17:21:59.765861    9735 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc4203438c0)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4200442f0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:21:59Z" level=info msg="Creating Job controller"
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T17:21:59Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
--- PASS: TestStatus (0.02s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.915s	coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestNormalPath
time="2019-03-26T17:22:05Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:05Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:05Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.846688ms)" job=default.test-tfjob
time="2019-03-26T17:22:05Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:05Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:05Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:22:05Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:22:05Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:05Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:05Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:05Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:05Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (752.749µs)" job=default.test-tfjob
time="2019-03-26T17:22:05Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:05Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:05Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:05Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:22:05Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:22:05Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:05Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:05Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (465.618µs)" job=default.test-tfjob
time="2019-03-26T17:22:05Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:05Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:05Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (347.243µs)" job=default.test-tfjob
time="2019-03-26T17:22:05Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:05Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:05Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:05Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:05Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:22:05Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:22:05Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (568.871µs)" job=default.test-tfjob
time="2019-03-26T17:22:05Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:05Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:05Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:05Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:05Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:05Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:05Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:05Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:05Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0326 17:22:05.744319    9843 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc4204e1fc0), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"PS":(*v1beta2.ReplicaSpec)(0xc4201318c0), "Worker":(*v1beta2.ReplicaSpec)(0xc4202a5340)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"PS":(*v1beta2.ReplicaStatus)(0xc42035a5a0), "Worker":(*v1beta2.ReplicaStatus)(0xc42035a5d0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobSucceeded' 'TFJob test-tfjob successfully completed.'
time="2019-03-26T17:22:05Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (913.833µs)" job=default.test-tfjob
time="2019-03-26T17:22:05Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:05Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:05Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:05Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:05Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (313.048µs)" job=default.test-tfjob
time="2019-03-26T17:22:05Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:05Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:05Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:05Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:05Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:05Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:05Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:05Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:05Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:05Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:05Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:22:05Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:22:05Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:05Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:22:05Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:22:05Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (726.478µs)" job=default.test-tfjob
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
time="2019-03-26T17:22:05Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:05Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:05Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:22:05Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:22:05Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:22:05Z" level=info msg="Started workers"
time="2019-03-26T17:22:06Z" level=info msg="Shutting down workers"
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
time="2019-03-26T17:22:06Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:06Z" level=info msg="Creating Job controller"
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
time="2019-03-26T17:22:06Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:06Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:06Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-26T17:22:06Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:22:06Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:22:06Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:22:06Z" level=info msg="Started workers"
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
time="2019-03-26T17:22:06Z" level=info msg="Shutting down workers"
time="2019-03-26T17:22:06Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:06Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:06Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:22:06Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:22:06Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:06Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:06Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:06Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:06Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (376.233µs)" job=default.test-tfjob
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
time="2019-03-26T17:22:06Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:22:06Z" level=info msg="Started workers"
time="2019-03-26T17:22:06Z" level=info msg="Shutting down workers"
time="2019-03-26T17:22:06Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:06Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:06Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:06Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (243.196µs)" job=default.test-tfjob
time="2019-03-26T17:22:06Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:06Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:06Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:06Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (434.723µs)" job=default.test-tfjob
time="2019-03-26T17:22:06Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:06Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:06Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:06Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:06Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:06Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:06Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:06Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:06Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:06Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (603.594µs)" job=default.test-tfjob
time="2019-03-26T17:22:06Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:06Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:06Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:06Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:06Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:06Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:06Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:06Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:06Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:06Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (247.078µs)" job=default.test-tfjob
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
time="2019-03-26T17:22:06Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:06Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:06Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:06Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (184.316µs)" job=default.test-tfjob
time="2019-03-26T17:22:06Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:06Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:06Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:06Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (222.535µs)" job=default.test-tfjob
time="2019-03-26T17:22:06Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:06Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:08Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:08Z" level=info msg="Ignoring inactive pod default/ps-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:08Z" level=info msg="Ignoring inactive pod default/ps-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:08Z" level=info msg="Ignoring inactive pod default/worker-0 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:08Z" level=info msg="Ignoring inactive pod default/worker-1 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:08Z" level=info msg="Ignoring inactive pod default/worker-2 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:08Z" level=info msg="Ignoring inactive pod default/worker-3 in state Succeeded, deletion time <nil>"
time="2019-03-26T17:22:08Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (550.326µs)" job=default.test-tfjob
--- PASS: TestCleanupTFJob (2.00s)
=== RUN   TestActiveDeadlineSeconds
time="2019-03-26T17:22:08Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:08Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg=">>>>>> TFJobs start time 2019-03-26 17:22:08.366799362 +0000 UTC m=+2.682288700 and deadline 0xc420411ce8" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg=">>>>>> TFJobs was active longer than specified deadline  test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg=">>>>>> TFJobs was ExceedsLimit  test-tfjob" job=default.test-tfjob uid=
--- FAIL: TestActiveDeadlineSeconds (2.00s)
	job_test.go:680: >>>>>> TIME AFTER SYNC: <nil>
	job_test.go:681: >>>>>> TIME AFTER SYNC: 2019-03-26 17:22:08.366799362 +0000 UTC m=+2.682288700
	job_test.go:687: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected number of pod deletes.  Expected 6, saw 0
	job_test.go:690: 4 workers and 2 ps is running, ActiveDeadlineSeconds is 2: unexpected number of service deletes.  Expected 6, saw 0
=== RUN   TestAddPod
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:22:10Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:22:10Z" level=info msg="Started workers"
--- PASS: TestAddPod (0.10s)
time="2019-03-26T17:22:10Z" level=info msg="Shutting down workers"
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:22:10Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="Ignoring inactive pod default/worker-0 in state Failed, deletion time <nil>"
time="2019-03-26T17:22:10Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0326 17:22:10.471179    9843 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc420902940), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc4208f58c0)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc4203084b0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-26T17:22:10Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0326 17:22:10.471364    9843 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(0xc420902940), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc4208f58c0)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc4203084b0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'TFJobRestarting' 'TFJob test-tfjob is restarting because 1 Worker replica(s) failed.'
time="2019-03-26T17:22:10Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:22:10Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (692.94µs)" job=default.test-tfjob
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
time="2019-03-26T17:22:10Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:22:10Z" level=info msg="Started workers"
time="2019-03-26T17:22:10Z" level=info msg="Shutting down workers"
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:22:10Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:22:10Z" level=info msg="Started workers"
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
time="2019-03-26T17:22:10Z" level=info msg="Shutting down workers"
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0326 17:22:10.574636    9843 event.go:259] Could not construct reference to: '&v1beta2.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta2.TFJobSpec{ActiveDeadlineSeconds:(*int64)(nil), BackoffLimit:(*int32)(nil), CleanPodPolicy:(*v1beta2.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta2.TFReplicaType]*v1beta2.ReplicaSpec{"Worker":(*v1beta2.ReplicaSpec)(0xc420341b80)}}, Status:v1beta2.JobStatus{Conditions:[]v1beta2.JobCondition(nil), ReplicaStatuses:map[v1beta2.ReplicaType]*v1beta2.ReplicaStatus{"Worker":(*v1beta2.ReplicaStatus)(0xc420411e50)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:22:10Z" level=info msg="Creating Job controller"
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T17:22:10Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
--- PASS: TestStatus (0.02s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
FAIL
coverage: 42.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow	4.912s
exit status 1: warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/version

travis_time:end:0025fcb6:start=1553620874987885016,finish=1553620930606327727,duration=55618442711
TravisBuddy Request Identifier: fab29890-4fee-11e9-bd36-ef1a269a9fbb

@TravisBuddy
Copy link

Travis tests have failed

Hey @ChanYiLin,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
pkg/controller.v1beta2/tensorflow/job_test.go:728:17:error: missing ',' before newline in composite literal (vet)
pkg/controller.v1beta2/tensorflow/job_test.go:728:17:warning: unused variable or constant missing ',' before newline in composite literal (varcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:721:69:warning: unused variable or constant undeclared name: adsTest2 (varcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:728:4:warning: unused variable or constant mixture of field:value and value elements in struct literal (varcheck)
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.047s
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.048s
=== RUN   TestValidateBetaOneTFJobSpec
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 2.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.046s
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 1.0% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.047s
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 4.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.067s
=== RUN   TestNormalPath
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestAddPod
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 29.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.909s
exit status 1: warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/version
# github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow
pkg/controller.v1beta2/tensorflow/job_test.go:728:17: missing ',' before newline in composite literal
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow [setup failed]

2nd Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
pkg/controller.v1beta2/tensorflow/job_test.go:721:69:warning: unused variable or constant undeclared name: adsTest2 (varcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:728:4:warning: unused variable or constant mixture of field:value and value elements in struct literal (varcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:728:17:warning: missing ',' before newline in composite literal (compile) (staticcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:721:69:warning: undeclared name: adsTest2 (compile) (staticcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:728:4:warning: mixture of field:value and value elements in struct literal (compile) (staticcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:728:17:error: syntax error: unexpected newline, expecting comma or } (vet)
pkg/controller.v1beta2/tensorflow/job_test.go:728:17:warning: unused variable or constant missing ',' before newline in composite literal (varcheck)
travis_time:end:1e5a96ce:start=1553621457388289113,finish=1553621632061712786,duration=174673423673
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.030s	coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.029s	coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestValidateBetaOneTFJobSpec
time="2019-03-26T17:34:08Z" level=warning msg="Image is undefined in the container"
time="2019-03-26T17:34:08Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
time="2019-03-26T17:34:08Z" level=warning msg="Image is undefined in the container"
time="2019-03-26T17:34:08Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.028s	coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.019s	coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
time="2019-03-26T17:34:25Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
time="2019-03-26T17:34:25Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.056s	coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestNormalPath
time="2019-03-26T17:34:31Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:31Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:31Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=4, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.631577ms)" job=default.test-tfjob
time="2019-03-26T17:34:31Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:31Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:31Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:34:31Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:34:31Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:31Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:31Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (705.198µs)" job=default.test-tfjob
time="2019-03-26T17:34:31Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:31Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:31Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
E0326 17:34:31.888965    9716 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc4203c4510), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"PS":(*v1beta1.ReplicaSpec)(0xc420131080), "Worker":(*v1beta1.ReplicaSpec)(0xc420400000)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"PS":(*v1beta1.ReplicaStatus)(0xc4204bd630), "Worker":(*v1beta1.ReplicaStatus)(0xc4204bd660)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobSucceeded' 'TFJob test-tfjob successfully completed.'
time="2019-03-26T17:34:31Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (903.1µs)" job=default.test-tfjob
time="2019-03-26T17:34:31Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:31Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:31Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:31Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:31Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (240.331µs)" job=default.test-tfjob
time="2019-03-26T17:34:31Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:31Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:31Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="Need to create new pod: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:34:31Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:34:31Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="need to create new service: ps-0" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:34:31Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:34:31Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:31Z" level=info msg="Need to create new pod: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:31Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:31Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:31Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:31Z" level=info msg="need to create new service: worker-1" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:31Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:31Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:31Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (1.621193ms)" job=default.test-tfjob
time="2019-03-26T17:34:31Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:31Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:31Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (388.784µs)" job=default.test-tfjob
time="2019-03-26T17:34:31Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:31Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:31Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:34:31Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:34:31Z" level=info msg="Need to create new pod: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:31Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:31Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="need to create new service: worker-2" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:31Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:31Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (611.026µs)" job=default.test-tfjob
time="2019-03-26T17:34:31Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:31Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:31Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="Need to create new pod: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:31Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="need to create new service: worker-3" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:31Z" level=info msg="Need to create new pod: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:34:31Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:31Z" level=info msg="need to create new service: ps-1" job=default.test-tfjob replica-type=ps uid=
time="2019-03-26T17:34:31Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (485.456µs)" job=default.test-tfjob
--- PASS: TestNormalPath (0.02s)
=== RUN   TestRun
time="2019-03-26T17:34:31Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:31Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:31Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:34:31Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:34:32Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:34:32Z" level=info msg="Started workers"
time="2019-03-26T17:34:32Z" level=info msg="Shutting down workers"
--- PASS: TestRun (0.50s)
=== RUN   TestSyncPdb
time="2019-03-26T17:34:32Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:32Z" level=info msg="Creating Job controller"
--- PASS: TestSyncPdb (0.00s)
=== RUN   TestAddTFJob
time="2019-03-26T17:34:32Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:32Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:32Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:34:32Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:34:32Z" level=info msg="TFJob test-tfjob is created." job=default.test-tfjob uid=
time="2019-03-26T17:34:32Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:34:32Z" level=info msg="Started workers"
--- PASS: TestAddTFJob (0.10s)
=== RUN   TestCopyLabelsAndAnnotation
time="2019-03-26T17:34:32Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:32Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:32Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:34:32Z" level=info msg="Need to create new pod: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:32Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:32Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:32Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (258.106µs)" job=default.test-tfjob
--- PASS: TestCopyLabelsAndAnnotation (0.00s)
=== RUN   TestDeletePodsAndServices
time="2019-03-26T17:34:32Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:32Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:32Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:34:32Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (243.368µs)" job=default.test-tfjob
time="2019-03-26T17:34:32Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:32Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:32Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:34:32Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (222.463µs)" job=default.test-tfjob
time="2019-03-26T17:34:32Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:32Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:32Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:34:32Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (260.462µs)" job=default.test-tfjob
time="2019-03-26T17:34:32Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:32Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:32Z" level=info msg="Shutting down workers"
time="2019-03-26T17:34:32Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:34:32Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:34:32Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:34:32Z" level=info msg="Started workers"
time="2019-03-26T17:34:32Z" level=info msg="Shutting down workers"
time="2019-03-26T17:34:32Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:34:32Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (320.072µs)" job=default.test-tfjob
--- PASS: TestDeletePodsAndServices (0.01s)
=== RUN   TestCleanupTFJob
time="2019-03-26T17:34:32Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:32Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:32Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:34:32Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (228.392µs)" job=default.test-tfjob
time="2019-03-26T17:34:32Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:32Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:32Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:34:32Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (182.075µs)" job=default.test-tfjob
time="2019-03-26T17:34:32Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:32Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (564.02µs)" job=default.test-tfjob
--- PASS: TestCleanupTFJob (2.01s)
=== RUN   TestAddPod
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:34:34Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:34:34Z" level=info msg="Started workers"
--- PASS: TestAddPod (0.10s)
=== RUN   TestClusterSpec
--- PASS: TestClusterSpec (0.00s)
=== RUN   TestRestartPolicy
time="2019-03-26T17:34:34Z" level=info msg="Shutting down workers"
--- PASS: TestRestartPolicy (0.00s)
=== RUN   TestExitCode
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:34:34Z" level=info msg="Reconcile TFJobs test-tfjob" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="Pod: default.worker-0 exited with code 130" job=default.test-tfjob replica-type=worker uid=
E0326 17:34:34.621286    9716 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc420450a80), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc4209282c0)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4209453a0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'ExitedWithCode' 'Pod: default.worker-0 exited with code 130'
time="2019-03-26T17:34:34Z" level=info msg="Need to restart the pod: default.worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
E0326 17:34:34.621465    9716 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(0xc420450a80), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc4209282c0)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc4209453a0)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'TFJobRestarting' 'TFJob test-tfjob is restarting because 1 Worker replica(s) failed.'
time="2019-03-26T17:34:34Z" level=info msg="need to create new service: worker-0" job=default.test-tfjob replica-type=worker uid=
time="2019-03-26T17:34:34Z" level=info msg="Finished syncing tfjob \"default/test-tfjob\" (678.853µs)" job=default.test-tfjob
--- PASS: TestExitCode (0.00s)
=== RUN   TestAddService
time="2019-03-26T17:34:34Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:34:34Z" level=info msg="Started workers"
time="2019-03-26T17:34:34Z" level=info msg="Shutting down workers"
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="Starting TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Waiting for informer caches to sync"
time="2019-03-26T17:34:34Z" level=info msg="Starting 1 workers"
time="2019-03-26T17:34:34Z" level=info msg="Started workers"
--- PASS: TestAddService (0.10s)
=== RUN   TestFailed
time="2019-03-26T17:34:34Z" level=info msg="Shutting down workers"
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=0, failed=1" job=default.test-tfjob uid=
E0326 17:34:34.725061    9716 event.go:259] Could not construct reference to: '&v1beta1.TFJob{TypeMeta:v1.TypeMeta{Kind:"TFJob", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"test-tfjob", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1beta1.TFJobSpec{CleanPodPolicy:(*v1beta1.CleanPodPolicy)(nil), TTLSecondsAfterFinished:(*int32)(nil), TFReplicaSpecs:map[v1beta1.TFReplicaType]*v1beta1.ReplicaSpec{"Worker":(*v1beta1.ReplicaSpec)(0xc4207fe000)}}, Status:v1beta1.JobStatus{Conditions:[]v1beta1.JobCondition(nil), ReplicaStatuses:map[v1beta1.ReplicaType]*v1beta1.ReplicaStatus{"Worker":(*v1beta1.ReplicaStatus)(0xc420361630)}, StartTime:(*v1.Time)(nil), CompletionTime:(*v1.Time)(nil), LastReconcileTime:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'TFJobFailed' 'TFJob test-tfjob has failed because 1 Worker replica(s) failed.'
--- PASS: TestFailed (0.00s)
=== RUN   TestStatus
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=2, failed=2" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=2, running=0, failed=2" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=3, running=3, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=1, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=1, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=0, running=0, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="Creating TFJob controller"
time="2019-03-26T17:34:34Z" level=info msg="Creating Job controller"
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Chief expected=1, running=0, failed=1" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=Worker expected=4, running=0, failed=4" job=default.test-tfjob uid=
time="2019-03-26T17:34:34Z" level=info msg="TFJob=test-tfjob, ReplicaType=PS expected=2, running=2, failed=0" job=default.test-tfjob uid=
--- PASS: TestStatus (0.01s)
=== RUN   TestGenOwnerReference
--- PASS: TestGenOwnerReference (0.00s)
=== RUN   TestGenLabels
--- PASS: TestGenLabels (0.00s)
=== RUN   TestConvertTFJobToUnstructured
--- PASS: TestConvertTFJobToUnstructured (0.00s)
PASS
coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow	2.911s	coverage: 42.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
exit status 1: warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/version
# github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow
pkg/controller.v1beta2/tensorflow/job_test.go:728:17: missing ',' before newline in composite literal
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow [setup failed]

travis_time:end:10c34d08:start=1553621632067902869,finish=1553621675136858436,duration=43068955567
TravisBuddy Request Identifier: 305a7250-4ff0-11e9-bd36-ef1a269a9fbb

@TravisBuddy
Copy link

Travis tests have failed

Hey @ChanYiLin,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
WARNING: failed to make error: failed to check packages: errors while loading package github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil: [/home/travis/gopath/src/github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil/tfjob.go a relative path: lstat error: failed to check packages: errors while loading package github.com: no such file or directory
pkg/common/util/v1beta2/testutil/tfjob.go:79:65:warning: undeclared name: RestartPolicyOnFailure (gotype, unconvert)
pkg/controller.v1beta1/tensorflow/controller_test.go:262:162:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:319:165:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:320:145:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:480:165:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:481:145:warning: unused variable or constant too many arguments (varcheck)
pkg/common/util/v1beta2/testutil/tfjob.go:72:66:warning: undeclared name: RestartPolicyOnFailure (gotype, unconvert)
error: failed to check packages: errors while loading package github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil: [/home/travis/gopath/src/github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil/tfjob.go:72:66:warning: error return value not checked (undeclared name: RestartPolicyOnFailure /home/travis/gopath/src/github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil/tfjob.go:79:65: undeclared name: RestartPolicyOnFailure]) (errcheck)
pkg/common/util/v1beta2/testutil/tfjob.go:72:66:warning: unused variable or constant undeclared name: RestartPolicyOnFailure (varcheck)
pkg/common/util/v1beta2/testutil/tfjob.go:79:65:warning: unused variable or constant undeclared name: RestartPolicyOnFailure (varcheck)
pkg/controller.v1beta1/tensorflow/controller_test.go:263:142:warning: unused variable or constant too many arguments (varcheck)
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
FAIL	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1 [build failed]
exit status 2: warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/informers/externalversions
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/jobcontroller
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/control
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/logger
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util/k8sutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util/train
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/version
# github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
/tmp/go-build804657653/github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil/_obj/tfjob.go:83:29: undefined: TFReplicaTypeWorker
/tmp/go-build804657653/github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil/_obj/tfjob.go:83:66: undefined: RestartPolicyOnFailure
/tmp/go-build804657653/github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil/_obj/tfjob.go:91:28: undefined: TFReplicaTypeWorker
/tmp/go-build804657653/github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil/_obj/tfjob.go:91:65: undefined: RestartPolicyOnFailure

2nd Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
WARNING: failed to make error: failed to check packages: errors while loading package github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil: [/home/travis/gopath/src/github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil/tfjob.go a relative path: lstat error: failed to check packages: errors while loading package github.com: no such file or directory
pkg/controller.v1beta1/tensorflow/job_test.go:319:27:error: too many arguments in call to testutil.SetPodsStatuses (vet)
pkg/common/util/v1beta2/testutil/tfjob.go:79:65:warning: unused variable or constant undeclared name: RestartPolicyOnFailure (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:481:145:warning: unused variable or constant too many arguments (varcheck)
pkg/common/util/v1beta2/testutil/tfjob.go:79:28:warning: undeclared name: TFReplicaTypeWorker (gotype, unconvert)
pkg/controller.v1beta1/tensorflow/controller_test.go:263:142:warning: too many arguments (compile) (staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:480:165:warning: too many arguments (compile) (staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:481:145:warning: too many arguments (compile) (staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/controller_test.go:262:162:warning: too many arguments (compile) (staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/controller_test.go:262:27:error: too many arguments in call to testutil.SetPodsStatuses (vet)
pkg/common/util/v1beta2/testutil/tfjob.go:72:66:warning: unused variable or constant undeclared name: RestartPolicyOnFailure (varcheck)
pkg/common/util/v1beta2/testutil/tfjob.go:72:29:warning: unused variable or constant undeclared name: TFReplicaTypeWorker (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:480:165:warning: unused variable or constant too many arguments (varcheck)
pkg/common/util/v1beta2/testutil/tfjob.go:79:28:warning: undeclared name: TFReplicaTypeWorker (compile) (staticcheck, staticcheck, staticcheck, staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/controller_test.go:263:27:error: too many arguments in call to testutil.SetPodsStatuses (vet)
pkg/controller.v1beta1/tensorflow/job_test.go:320:27:error: too many arguments in call to testutil.SetPodsStatuses (vet)
pkg/controller.v1beta1/tensorflow/job_test.go:480:27:error: too many arguments in call to testutil.SetPodsStatuses (vet)
pkg/common/util/v1beta2/testutil/tfjob.go:79:28:error: undefined: TFReplicaTypeWorker (vet)
pkg/common/util/v1beta2/testutil/tfjob.go:79:65:error: undefined: RestartPolicyOnFailure (vet)
pkg/common/util/v1beta2/testutil/tfjob.go:79:28:warning: unused variable or constant undeclared name: TFReplicaTypeWorker (varcheck)
pkg/controller.v1beta1/tensorflow/controller_test.go:262:162:warning: unused variable or constant too many arguments (varcheck)
error: failed to check packages: errors while loading package github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil: [/home/travis/gopath/src/github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil/tfjob.go:72:66:warning: error return value not checked (undeclared name: RestartPolicyOnFailure /home/travis/gopath/src/github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil/tfjob.go:72:29: undeclared name: TFReplicaTypeWorker /home/travis/gopath/src/github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil/tfjob.go:79:65: undeclared name: RestartPolicyOnFailure /home/travis/gopath/src/github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil/tfjob.go:79:28: undeclared name: TFReplicaTypeWorker]) (errcheck)
pkg/common/util/v1beta2/testutil/tfjob.go:72:66:warning: undeclared name: RestartPolicyOnFailure (gotype, unconvert)
pkg/common/util/v1beta2/testutil/tfjob.go:72:29:warning: undeclared name: TFReplicaTypeWorker (gotype, unconvert)
pkg/common/util/v1beta2/testutil/tfjob.go:72:29:warning: undeclared name: TFReplicaTypeWorker (compile) (staticcheck, staticcheck, staticcheck, staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:319:165:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:320:145:warning: unused variable or constant too many arguments (varcheck)
pkg/common/util/v1beta2/testutil/tfjob.go:79:65:warning: undeclared name: RestartPolicyOnFailure (gotype, unconvert)
pkg/common/util/v1beta2/testutil/tfjob.go:72:66:error: undefined: RestartPolicyOnFailure (vet)
pkg/controller.v1beta1/tensorflow/job_test.go:481:27:error: too many arguments in call to testutil.SetPodsStatuses (vet)
pkg/controller.v1beta1/tensorflow/controller_test.go:263:142:warning: unused variable or constant too many arguments (varcheck)
pkg/common/util/v1beta2/testutil/tfjob.go:79:65:warning: undeclared name: RestartPolicyOnFailure (compile) (staticcheck, staticcheck, staticcheck, staticcheck, staticcheck)
pkg/common/util/v1beta2/testutil/tfjob.go:72:29:error: undefined: TFReplicaTypeWorker (vet)
pkg/common/util/v1beta2/testutil/tfjob.go:72:66:warning: undeclared name: RestartPolicyOnFailure (compile) (staticcheck, staticcheck, staticcheck, staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:319:165:warning: too many arguments (compile) (staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:320:145:warning: too many arguments (compile) (staticcheck, staticcheck)
travis_time:end:0fa06494:start=1553623270329153756,finish=1553623413583630429,duration=143254476673
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.031s	coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.030s	coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestValidateBetaOneTFJobSpec
time="2019-03-26T18:03:50Z" level=warning msg="Image is undefined in the container"
time="2019-03-26T18:03:50Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
time="2019-03-26T18:03:50Z" level=warning msg="Image is undefined in the container"
time="2019-03-26T18:03:50Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.029s	coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.022s	coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
exit status 2: warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/informers/externalversions
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/jobcontroller
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/control
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/logger
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util/k8sutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util/train
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/version
# github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
pkg/common/util/v1beta2/testutil/tfjob.go:72: undefined: TFReplicaTypeWorker
pkg/common/util/v1beta2/testutil/tfjob.go:72: undefined: RestartPolicyOnFailure
pkg/common/util/v1beta2/testutil/tfjob.go:79: undefined: TFReplicaTypeWorker
pkg/common/util/v1beta2/testutil/tfjob.go:79: undefined: RestartPolicyOnFailure

travis_time:end:0549e308:start=1553623413589956751,finish=1553623444234701649,duration=30644744898
TravisBuddy Request Identifier: 4fe728a0-4ff2-11e9-bd36-ef1a269a9fbb

@TravisBuddy
Copy link

Travis tests have failed

Hey @ChanYiLin,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
pkg/controller.v1beta2/tensorflow/job_test.go:721:69:warning: unused variable or constant undeclared name: adsTest2 (varcheck)
pkg/controller.v1beta1/tensorflow/controller_test.go:262:162:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/controller_test.go:263:142:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:319:165:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:320:145:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:480:165:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:481:145:warning: unused variable or constant too many arguments (varcheck)
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.053s
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.048s
=== RUN   TestValidateBetaOneTFJobSpec
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 2.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.046s
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 1.0% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.046s
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 4.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.066s
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow [build failed]
exit status 2: warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on github.com/kubeflow/tf-operator/pkg/version
# github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
pkg/controller.v1beta1/tensorflow/controller_test.go:262:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, r.v1beta1/tensorflow/job_test.go:481:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, *"testing".T)

2nd Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
pkg/controller.v1beta1/tensorflow/job_test.go:480:165:warning: too many arguments (compile) (staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:481:145:warning: too many arguments (compile) (staticcheck, staticcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:721:69:warning: undeclared name: adsTest2 (compile) (staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:319:165:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:481:145:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/controller_test.go:262:27:error: too many arguments in call to testutil.SetPodsStatuses (vet)
pkg/controller.v1beta1/tensorflow/job_test.go:319:27:error: too many arguments in call to testutil.SetPodsStatuses (vet)
pkg/controller.v1beta1/tensorflow/job_test.go:320:27:error: too many arguments in call to testutil.SetPodsStatuses (vet)
pkg/controller.v1beta1/tensorflow/job_test.go:481:27:error: too many arguments in call to testutil.SetPodsStatuses (vet)
pkg/controller.v1beta1/tensorflow/controller_test.go:262:162:warning: too many arguments (compile) (staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:320:145:warning: too many arguments (compile) (staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/controller_test.go:262:162:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/controller_test.go:263:27:error: too many arguments in call to testutil.SetPodsStatuses (vet)
pkg/controller.v1beta2/tensorflow/job_test.go:721:69:warning: unused variable or constant undeclared name: adsTest2 (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:320:145:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta2/tensorflow/job_test.go:721:69:error: undefined: adsTest2 (vet)
pkg/controller.v1beta1/tensorflow/controller_test.go:263:142:warning: too many arguments (compile) (staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:319:165:warning: too many arguments (compile) (staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/controller_test.go:263:142:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:480:165:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:480:27:error: too many arguments in call to testutil.SetPodsStatuses (vet)
travis_time:end:073710bd:start=1553622581467050247,finish=1553622751966794712,duration=170499744465
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.030s	coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.030s	coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestValidateBetaOneTFJobSpec
time="2019-03-26T17:52:48Z" level=warning msg="Image is undefined in the container"
time="2019-03-26T17:52:48Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
time="2019-03-26T17:52:48Z" level=warning msg="Image is undefined in the container"
time="2019-03-26T17:52:48Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.029s	coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.021s	coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
time="2019-03-26T17:53:06Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
time="2019-03-26T17:53:06Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.055s	coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow [build failed]
exit status 2: warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/version
# github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
pkg/controller.v1beta1/tensorflow/controller_test.go:262:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, *"testing".T)
pkg/controller.v1beta1/tensorflow/controller_test.go:263:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, *"testing".T)
pkg/controller.v1beta1/tensorflow/job_test.go:319:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, *"testing".T)
pkg/controller.v1beta1/tensorflow/job_test.go:320:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, *"testing".T)
pkg/controller.v1beta1/tensorflow/job_test.go:480:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, *"testing".T)
pkg/controller.v1beta1/tensorflow/job_test.go:481:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, *"testing".T)

travis_time:end:36b6511b:start=1553622751973382163,finish=1553622788988430548,duration=37015048385
TravisBuddy Request Identifier: 1b614ec0-4ff3-11e9-bd36-ef1a269a9fbb

@TravisBuddy
Copy link

Travis tests have failed

Hey @ChanYiLin,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
pkg/controller.v1beta1/tensorflow/job_test.go:480:165:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:481:145:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/controller_test.go:262:162:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/controller_test.go:263:142:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:319:165:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:320:145:warning: unused variable or constant too many arguments (varcheck)
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.2% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.066s
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 2.2% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.062s
=== RUN   TestValidateBetaOneTFJobSpec
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 2.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.048s
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 1.0% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.050s
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 4.3% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.067s
int32, int32, *"testing".T)
pkg/controller.v1beta1/tensorflow/controller_test.go:263:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, *"testing".T)
pkg/controller.v1beta1/tensorflow/job_test.go:319:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, *"testing".T)
pkg/controller.v1beta1/tensorflow/job_test.go:320:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, *"testing".T)
pkg/controller.v1beta1/tensorflow/job_test.go:480:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, *"testing".T)
pkg/controller.v1beta1/tensorflow/job_test.go:481:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, *"testing".T)

2nd Build

View build log

gometalinter --config=linter_config.json --vendor ./... 
pkg/controller.v1beta1/tensorflow/job_test.go:481:27:error: too many arguments in call to testutil.SetPodsStatuses (vet)
pkg/controller.v1beta1/tensorflow/job_test.go:481:145:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/controller_test.go:262:162:warning: too many arguments (compile) (staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/controller_test.go:263:142:warning: too many arguments (compile) (staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:480:165:warning: too many arguments (compile) (staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:320:27:error: too many arguments in call to testutil.SetPodsStatuses (vet)
pkg/controller.v1beta1/tensorflow/job_test.go:319:165:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/controller_test.go:262:27:error: too many arguments in call to testutil.SetPodsStatuses (vet)
pkg/controller.v1beta1/tensorflow/job_test.go:319:27:error: too many arguments in call to testutil.SetPodsStatuses (vet)
pkg/controller.v1beta1/tensorflow/job_test.go:480:165:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:481:145:warning: too many arguments (compile) (staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/controller_test.go:263:27:error: too many arguments in call to testutil.SetPodsStatuses (vet)
pkg/controller.v1beta1/tensorflow/job_test.go:480:27:error: too many arguments in call to testutil.SetPodsStatuses (vet)
pkg/controller.v1beta1/tensorflow/controller_test.go:262:162:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/controller_test.go:263:142:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:320:145:warning: unused variable or constant too many arguments (varcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:319:165:warning: too many arguments (compile) (staticcheck, staticcheck)
pkg/controller.v1beta1/tensorflow/job_test.go:320:145:warning: too many arguments (compile) (staticcheck, staticcheck)
travis_time:end:065081ff:start=1553623803478801161,finish=1553623972486534516,duration=169007733355
goveralls -service=travis-ci -v -package ./pkg/... -ignore "pkg/client/*/*.go,pkg/client/*/*/*.go,pkg/client/*/*/*/*.go,pkg/client/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*.go,pkg/client/*/*/*/*/*/*/*.go,pkg/util/testutil/*.go,pkg/apis/tensorflow/*/zz_generated.*.go,pkg/apis/tensorflow/*/*_generated.go,pkg/apis/common/*/zz_generated.*.go,pkg/apis/common/*/*_generated.go" 
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2	[no test files]
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1	0.029s	coverage: 21.8% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestSetTypeNames
--- PASS: TestSetTypeNames (0.00s)
=== RUN   TestSetDefaultTFJob
--- PASS: TestSetDefaultTFJob (0.00s)
=== RUN   TestIsChieforMaster
--- PASS: TestIsChieforMaster (0.00s)
PASS
coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2	0.029s	coverage: 20.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
=== RUN   TestValidateBetaOneTFJobSpec
time="2019-03-26T18:13:08Z" level=warning msg="Image is undefined in the container"
time="2019-03-26T18:13:08Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaOneTFJobSpec (0.00s)
=== RUN   TestValidateBetaTwoTFJobSpec
time="2019-03-26T18:13:08Z" level=warning msg="Image is undefined in the container"
time="2019-03-26T18:13:08Z" level=warning msg="There is no container named tensorflow in Worker"
--- PASS: TestValidateBetaTwoTFJobSpec (0.00s)
PASS
coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation	0.030s	coverage: 13.7% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2	[no test files]
=== RUN   TestGenGeneralName
--- PASS: TestGenGeneralName (0.00s)
PASS
coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/common/jobcontroller	0.020s	coverage: 0.5% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil	[no test files]
?   	github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured	[no test files]
=== RUN   TestCreatePods
--- PASS: TestCreatePods (0.01s)
=== RUN   TestCreateService
time="2019-03-26T18:13:26Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateService (0.00s)
=== RUN   TestCreateServicesWithControllerRef
time="2019-03-26T18:13:26Z" level=info msg="Controller test-tfjob created service empty_service"
--- PASS: TestCreateServicesWithControllerRef (0.00s)
=== RUN   TestClaimServices
--- PASS: TestClaimServices (0.00s)
PASS
coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
ok  	github.com/kubeflow/tf-operator/pkg/control	0.056s	coverage: 44.9% of statements in github.com/kubeflow/tf-operator/pkg/apis/common/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/common/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/apis/tensorflow/validation, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/scheme, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/internalinterfaces, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/informers/externalversions/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta1, github.com/kubeflow/tf-operator/pkg/client/listers/tensorflow/v1beta2, github.com/kubeflow/tf-operator/pkg/common/jobcontroller, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/unstructured, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil, github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured, github.com/kubeflow/tf-operator/pkg/control, github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow, github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow, github.com/kubeflow/tf-operator/pkg/logger, github.com/kubeflow/tf-operator/pkg/util, github.com/kubeflow/tf-operator/pkg/util/k8sutil, github.com/kubeflow/tf-operator/pkg/util/signals, github.com/kubeflow/tf-operator/pkg/util/train, github.com/kubeflow/tf-operator/pkg/version
FAIL	github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow [build failed]
exit status 2: warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta1/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/client/clientset/versioned/typed/tensorflow/v1beta2/fake
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta1/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/testutil
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/common/util/v1beta2/unstructured
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/controller.v1beta2/tensorflow
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/util/signals
warning: no packages being tested depend on matches for pattern github.com/kubeflow/tf-operator/pkg/version
# github.com/kubeflow/tf-operator/pkg/controller.v1beta1/tensorflow
pkg/controller.v1beta1/tensorflow/controller_test.go:262:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, *"testing".T)
pkg/controller.v1beta1/tensorflow/controller_test.go:263:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, *"testing".T)
pkg/controller.v1beta1/tensorflow/job_test.go:319:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, *"testing".T)
pkg/controller.v1beta1/tensorflow/job_test.go:320:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, *"testing".T)
pkg/controller.v1beta1/tensorflow/job_test.go:480:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, *"testing".T)
pkg/controller.v1beta1/tensorflow/job_test.go:481:27: too many arguments in call to testutil.SetPodsStatuses
	have (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, nil, *"testing".T)
	want (cache.Indexer, *"github.com/kubeflow/tf-operator/pkg/apis/tensorflow/v1beta1".TFJob, string, int32, int32, int32, int32, *"testing".T)

travis_time:end:0076e8dc:start=1553623972494183155,finish=1553624009734186378,duration=37240003223
TravisBuddy Request Identifier: 40763ab0-4ff6-11e9-bd36-ef1a269a9fbb

@coveralls
Copy link

Coverage Status

Coverage remained the same at 76.744% when pulling 7649e1d on ChanYiLin:master into 4689a23 on kubeflow:master.

@richardsliu
Copy link
Contributor

/test all

@richardsliu
Copy link
Contributor

/retest

@richardsliu
Copy link
Contributor

/lgtm
/approve

@ChanYiLin
Copy link
Member Author

/assign @richardsliu
All the tests are passed. Thanks for your help!

@k8s-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: richardsliu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit aa322c7 into kubeflow:master Mar 26, 2019
@richardsliu richardsliu mentioned this pull request Mar 26, 2019
Closed
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jimexist jimexist Awaiting requested review from jimexist

@yph152 yph152 Awaiting requested review from yph152

Assignees

@richardsliu richardsliu

Labels
approved lgtm size/XL
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@ChanYiLin @TravisBuddy @richardsliu @coveralls @k8s-ci-robot