Katib v0.15.0 Roadmap

[johnugeorge]

Major features tracked for next Katib release. Few might be a stretch but still tracked for next release.

Kubeflow Pipelines

• Better data passing for Katib component - (#1846)
• Use Kubeflow Pipelines as Trial target - (#1914)

MLFlow Integration

• Support Push based metric collection(#577)

UI enhancements

• Metrics out of view for experiment metrics (Metrics can go out of view when visualizing experiment metrics #1923)
• 500 error code after refresh Katib experiment detail page 500 error after refresh #1967
• Expose trial logs(Show trial logs on the interface #971 )
• Sorting trial list([UI Feature] Sorting for the Trial list table #1441 )
• Expose more status fields ( [rev] Extend new Katib UI with more Status fields #1434 )

Misc

• Narrow RBAC rules for Katib deployment(#1639)
• Customize Suggestion deployment(#1737 #1976)
• Support for Kubernetes 1.25 (Support Kubernetes v1.25 #1957)
• Support Log uniform scale ([feature] Support log-uniform scale in search space definition #1207 )
• Implement Conformance tests (Conformance tests for Katib #2044)

/cc @tenzen-y @andreyvelich @gaocegege

[tenzen-y]

/cc @anencore94

[johnugeorge]

@tenzen-y @anencore94 @andreyvelich Please add to the list if I missed out any.

[tenzen-y]

@johnugeorge Thanks for creating this!
I would like to add #1207 to the next katib release. WDYT?

[tenzen-y]

Also, I think that we need to add postponed #1728 (comment).

[johnugeorge]

@johnugeorge Thanks for creating this!
I would like to add #1207 to the next katib release. WDYT?

Added.

[anencore94]

we aim to this features are supposed to be implemented before Wednesday, Jan 25th 2023: Week 18 - Feature Freeze, at kubeflow v1.7 right ?

[johnugeorge]

Yes.

[andreyvelich]

I have the following thoughts:

• We need to promote tune API: Create Tune API in the Katib SDK #1951 in the new release to simplify Katib usage and attract more users.
• @johnugeorge @kimwnasptd Are we targeting the above UI changes before Jan 25th ?
• @tenzen-y @johnugeorge I want to have broader discussion around log-uniform and other HPs distributions before starting the API changes: [feature] Support log-uniform scale in search space definition #1207. It would be great if we could discuss it on the upcoming AutoML meeting (Nov 16th).

[tenzen-y]

@andreyvelich

I want to have broader discussion around log-uniform and other HPs distributions before starting the API changes

Sure. I think so too.

It would be great if we could discuss it on the upcoming AutoML meeting (Nov 16th).

Sorry, I'm not likely to be able to attend the community meeting for a while. So, can you share with me the results of the discussion at the meeting, later?

Also, I think that we might be good to extend API based on #1207 (comment).

[andreyvelich]

@tenzen-y Sure, we are going to share the updates.

Also, I think that we might be good to extend API based on #1207 (comment).

If we are going to add distribution to FeasibleSpace, maybe we should deprecate ParameterType in our APIs, since it's redundant.

e.g. Hyperopt directly set uniform, loguniform, etc. parameter with min and max value: http://hyperopt.github.io/hyperopt/getting-started/search_spaces/

[tenzen-y]

If we are going to add distribution to FeasibleSpace, maybe we should deprecate ParameterType in our APIs, since it's redundant.

sgtm
This is just a question, in that case, do we handle with categorical and discrete as distribution?

[andreyvelich]

This is just a question, in that case, do we handle with categorical and discrete as distribution?

I think, we don't need discrete at all, since for the suggestion services (at least of one that we used) categorical and discrete is the same parameter (some frameworks call it choice).

[tenzen-y]

This is just a question, in that case, do we handle with categorical and discrete as distribution?

I think, we don't need discrete at all, since for the suggestion services (at least of one that we used) categorical and discrete is the same parameter (some frameworks call it choice).

@andreyvelich Thanks for clarifying. I agree with your idea.

[tenzen-y]

Around next month, I will propose the API design for #1207.

[andreyvelich]

@tenzen-y If you want, you can create small proposal under docs: https://github.com/kubeflow/katib/tree/master/docs/proposals

[tenzen-y]

@tenzen-y If you want, you can create small proposal under docs: https://github.com/kubeflow/katib/tree/master/docs/proposals

Thanks for letting me know. Looks great!
I will submit a small enhancement proposal!

[tenzen-y]

#2055

[andreyvelich]

The Katib 0.15 has been officially released: https://github.com/kubeflow/katib/releases/tag/v0.15.0

Thank you very much for all of your help!