Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug] unit test is broken in github.com/kubeflow/katib/pkg/db/v1alpha2 #523

Closed
gaocegege opened this issue May 15, 2019 · 0 comments
Closed
Labels

Comments

@gaocegege
Copy link
Member

/kind bug

What steps did you take and what happened:
[A clear and concise description of what the bug is.]

error `SELECT 1` probing: Error `SELECT 1` probing: all expectations were already fulfilled, call to ExecQuery 'SELECT 1' with args [] was not expected
--- FAIL: TestRegisterExperiment (0.00s)
    interface_test.go:143: RegisterExperiment failed: ExecQuery 'INSERT INTO experiments (
        			name, 
        			parameters, 
        			objective, 
        			algorithm, 
        			trial_template,
        			metrics_collector_spec,
        			parallel_trial_count, 
        			max_trial_count,
        			status,
        			start_time,
        			completion_time,
        			nas_config) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)', arguments do not match: argument 9 expected [string - 2016-12-31 20:02:05.123456] does not match actual [string - 2019-05-15 03:38:06.771618]
--- FAIL: TestGetExperiment (0.00s)
    interface_test.go:167: GetExperiment failed call to Query 'SELECT * FROM experiments WHERE name = ?' with args [{Name: Ordinal:1 Value:test1}], was not expected, next expectation is: ExpectedExec => expecting Exec or ExecContext which:
          - matches sql: 'INSERT INTO experiments \(
        			name, 
        			parameters, 
        			objective, 
        			algorithm, 
        			trial_template,
        			metrics_collector_spec,
        			parallel_trial_count, 
        			max_trial_count,
        			status,
        			start_time,
        			completion_time,
        			nas_config\)'
          - is with arguments:
            0 - test1
            1 - {"parameters":[]}
            2 - {"goal":0.99,"objectiveMetricName":"f1_score","additionalMetricNames":["loss","precision","recall"]}
            3 - {}
            4 - 
            5 - 
            6 - 10
            7 - 100
            8 - CREATED
            9 - 2016-12-31 20:02:05.123456
            10 - 
            11 - 
          - should return Result having:
              LastInsertId: 1
              RowsAffected: 1
--- FAIL: TestGetExperimentList (0.00s)
    interface_test.go:189: GetExperimentList failed call to Query 'SELECT name, status, start_time, completion_time FROM experiments' with args [], was not expected, next expectation is: ExpectedExec => expecting Exec or ExecContext which:
          - matches sql: 'INSERT INTO experiments \(
        			name, 
        			parameters, 
        			objective, 
        			algorithm, 
        			trial_template,
        			metrics_collector_spec,
        			parallel_trial_count, 
        			max_trial_count,
        			status,
        			start_time,
        			completion_time,
        			nas_config\)'
          - is with arguments:
            0 - test1
            1 - {"parameters":[]}
            2 - {"goal":0.99,"objectiveMetricName":"f1_score","additionalMetricNames":["loss","precision","recall"]}
            3 - {}
            4 - 
            5 - 
            6 - 10
            7 - 100
            8 - CREATED
            9 - 2016-12-31 20:02:05.123456
            10 - 
            11 - 
          - should return Result having:
              LastInsertId: 1
              RowsAffected: 1
--- FAIL: TestUpdateExperimentStatus (0.00s)
    interface_test.go:216: UpdateExperiment failed ExecQuery: could not match actual sql: "UPDATE experiments SET status = ?, start_time = ?, completion_time = ? WHERE name = ?" with expected regexp "INSERT INTO experiments \( name, parameters, objective, algorithm, trial_template, metrics_collector_spec, parallel_trial_count, max_trial_count, status, start_time, completion_time, nas_config\)"
2019/05/15 11:38:06 Failed to get current state call to Query 'SELECT setting_name, value FROM extra_algorithm_settings WHERE experiment_name = ?' with args [{Name: Ordinal:1 Value:test1}], was not expected, next expectation is: ExpectedExec => expecting Exec or ExecContext which:
  - matches sql: 'INSERT INTO experiments \(
			name, 
			parameters, 
			objective, 
			algorithm, 
			trial_template,
			metrics_collector_spec,
			parallel_trial_count, 
			max_trial_count,
			status,
			start_time,
			completion_time,
			nas_config\)'
  - is with arguments:
    0 - test1
    1 - {"parameters":[]}
    2 - {"goal":0.99,"objectiveMetricName":"f1_score","additionalMetricNames":["loss","precision","recall"]}
    3 - {}
    4 - 
    5 - 
    6 - 10
    7 - 100
    8 - CREATED
    9 - 2016-12-31 20:02:05.123456
    10 - 
    11 - 
  - should return Result having:
      LastInsertId: 1
      RowsAffected: 1
--- FAIL: TestUpdateAlgorithmExtraSettings (0.00s)
    interface_test.go:252: UpdateAlgorithmExtraSettings failed call to Query 'SELECT setting_name, value FROM extra_algorithm_settings WHERE experiment_name = ?' with args [{Name: Ordinal:1 Value:test1}], was not expected, next expectation is: ExpectedExec => expecting Exec or ExecContext which:
          - matches sql: 'INSERT INTO experiments \(
        			name, 
        			parameters, 
        			objective, 
        			algorithm, 
        			trial_template,
        			metrics_collector_spec,
        			parallel_trial_count, 
        			max_trial_count,
        			status,
        			start_time,
        			completion_time,
        			nas_config\)'
          - is with arguments:
            0 - test1
            1 - {"parameters":[]}
            2 - {"goal":0.99,"objectiveMetricName":"f1_score","additionalMetricNames":["loss","precision","recall"]}
            3 - {}
            4 - 
            5 - 
            6 - 10
            7 - 100
            8 - CREATED
            9 - 2016-12-31 20:02:05.123456
            10 - 
            11 - 
          - should return Result having:
              LastInsertId: 1
              RowsAffected: 1
--- FAIL: TestRegisterTrial (0.00s)
    interface_test.go:334: RegisterTrial failed: ExecQuery: could not match actual sql: "INSERT INTO trials ( name, experiment_name, objective, parameter_assignments, run_spec, metrics_collector_spec, observation, status, start_time, completion_time) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)" with expected regexp "INSERT INTO experiments \( name, parameters, objective, algorithm, trial_template, metrics_collector_spec, parallel_trial_count, max_trial_count, status, start_time, completion_time, nas_config\)"
--- FAIL: TestGetTrialList (0.00s)
    interface_test.go:369: GetTrialList failed call to Query 'SELECT * FROM trials WHERE experiment_name = ? AND name LIKE '%?%'' with args [{Name: Ordinal:1 Value:test1} {Name: Ordinal:2 Value:trial}], was not expected, next expectation is: ExpectedExec => expecting Exec or ExecContext which:
          - matches sql: 'INSERT INTO experiments \(
        			name, 
        			parameters, 
        			objective, 
        			algorithm, 
        			trial_template,
        			metrics_collector_spec,
        			parallel_trial_count, 
        			max_trial_count,
        			status,
        			start_time,
        			completion_time,
        			nas_config\)'
          - is with arguments:
            0 - test1
            1 - {"parameters":[]}
            2 - {"goal":0.99,"objectiveMetricName":"f1_score","additionalMetricNames":["loss","precision","recall"]}
            3 - {}
            4 - 
            5 - 
            6 - 10
            7 - 100
            8 - CREATED
            9 - 2016-12-31 20:02:05.123456
            10 - 
            11 - 
          - should return Result having:
              LastInsertId: 1
              RowsAffected: 1
--- FAIL: TestGetTrial (0.00s)
    interface_test.go:397: GetTrial incorrect return spec:<> status:<> 
--- FAIL: TestUpdateTrialStatus (0.00s)
    interface_test.go:447: UpdateTrial failed ExecQuery: could not match actual sql: "UPDATE trials SET status = ?, start_time = ?, completion_time = ?, observation = ? WHERE name = ?" with expected regexp "INSERT INTO experiments \( name, parameters, objective, algorithm, trial_template, metrics_collector_spec, parallel_trial_count, max_trial_count, status, start_time, completion_time, nas_config\)"
--- FAIL: TestRegisterObservationLog (0.00s)
    interface_test.go:501: RegisterExperiment failed: ExecQuery: could not match actual sql: "INSERT INTO observation_logs ( trial_name, time, metric_name, value ) VALUES (?, ?, ?, ?)" with expected regexp "INSERT INTO experiments \( name, parameters, objective, algorithm, trial_template, metrics_collector_spec, parallel_trial_count, max_trial_count, status, start_time, completion_time, nas_config\)"
2019/05/15 11:38:06 Failed to get ObservationLogs call to Query 'SELECT time, metric_name, value FROM observation_logs WHERE trial_name = ? AND time >= ? ORDER BY time' with args [{Name: Ordinal:1 Value:test1_trial1} {Name: Ordinal:2 Value:2016-12-31 21:01:05.123456}], was not expected, next expectation is: ExpectedExec => expecting Exec or ExecContext which:
  - matches sql: 'INSERT INTO experiments \(
			name, 
			parameters, 
			objective, 
			algorithm, 
			trial_template,
			metrics_collector_spec,
			parallel_trial_count, 
			max_trial_count,
			status,
			start_time,
			completion_time,
			nas_config\)'
  - is with arguments:
    0 - test1
    1 - {"parameters":[]}
    2 - {"goal":0.99,"objectiveMetricName":"f1_score","additionalMetricNames":["loss","precision","recall"]}
    3 - {}
    4 - 
    5 - 
    6 - 10
    7 - 100
    8 - CREATED
    9 - 2016-12-31 20:02:05.123456
    10 - 
    11 - 
  - should return Result having:
      LastInsertId: 1
      RowsAffected: 1
--- FAIL: TestGetObservationLog (0.00s)
    interface_test.go:524: GetObservationLog failed call to Query 'SELECT time, metric_name, value FROM observation_logs WHERE trial_name = ? AND time >= ? ORDER BY time' with args [{Name: Ordinal:1 Value:test1_trial1} {Name: Ordinal:2 Value:2016-12-31 21:01:05.123456}], was not expected, next expectation is: ExpectedExec => expecting Exec or ExecContext which:
          - matches sql: 'INSERT INTO experiments \(
        			name, 
        			parameters, 
        			objective, 
        			algorithm, 
        			trial_template,
        			metrics_collector_spec,
        			parallel_trial_count, 
        			max_trial_count,
        			status,
        			start_time,
        			completion_time,
        			nas_config\)'
          - is with arguments:
            0 - test1
            1 - {"parameters":[]}
            2 - {"goal":0.99,"objectiveMetricName":"f1_score","additionalMetricNames":["loss","precision","recall"]}
            3 - {}
            4 - 
            5 - 
            6 - 10
            7 - 100
            8 - CREATED
            9 - 2016-12-31 20:02:05.123456
            10 - 
            11 - 
          - should return Result having:
              LastInsertId: 1
              RowsAffected: 1
FAIL
coverage: 43.9% of statements

What did you expect to happen:

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

  • Kubeflow version:
  • Minikube version:
  • Kubernetes version: (use kubectl version):
  • OS (e.g. from /etc/os-release):
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants