Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rename trusty flag to disableTrustyBiasMetrics #3340

Merged
merged 1 commit into from
Oct 18, 2024
Merged

rename trusty flag to disableTrustyBiasMetrics #3340

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion backend/src/types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ export type DashboardConfig = K8sResourceCommon & {
disableProjectSharing: boolean;
disableCustomServingRuntimes: boolean;
disablePipelines: boolean;
disableBiasMetrics: boolean;
disableTrustyBiasMetrics: boolean;
disablePerformanceMetrics: boolean;
disableKServe: boolean;
disableKServeAuth: boolean;
Expand Down
2 changes: 1 addition & 1 deletion backend/src/utils/constants.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ export const blankDashboardCR: DashboardConfig = {
disableModelServing: false,
disableProjectSharing: false,
disableCustomServingRuntimes: false,
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
disablePipelines: false,
disableKServe: false,
Expand Down
46 changes: 1 addition & 45 deletions backend/src/utils/resourceUtils.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,9 +76,7 @@ const DASHBOARD_CONFIG = {
};

const fetchDashboardCR = async (fastify: KubeFastifyInstance): Promise<DashboardConfig[]> => {
return fetchOrCreateDashboardCR(fastify)
.then(softDisableBiasMetrics(fastify))
.then((dashboardCR) => [dashboardCR]);
return fetchOrCreateDashboardCR(fastify).then((dashboardCR) => [dashboardCR]);
};

const fetchWatchedClusterStatus = async (
Expand All @@ -87,48 +85,6 @@ const fetchWatchedClusterStatus = async (
return fetchClusterStatus(fastify).then((clusterStatus) => [clusterStatus]);
};

/**
* TODO: Support Bias Metrics https://issues.redhat.com/browse/RHOAIENG-13084
* For RHOAI Only
* Always disable bias metrics until we can properly support the new UI flow.
* Note: This does no changes to the on-cluster value -- there can be a de-sync
*/
const softDisableBiasMetrics =
(fastify: KubeFastifyInstance) =>
(dashboardConfig: DashboardConfig): DashboardConfig & { status?: object } => {
if (!isRHOAI(fastify)) {
return dashboardConfig;
}

fastify.log.info(
'Trusty Bias Metrics are explicitly disabled in the cached odh-dashboard-config',
);
return {
...dashboardConfig,
spec: {
...dashboardConfig.spec,
dashboardConfig: {
...dashboardConfig.spec.dashboardConfig,
disableBiasMetrics: true,
},
},
// NOTE: This is a fake status to help show in the UI that we are overriding the value
// The CRD does not support a status property today and should not have existing statuses here
// No code should use this -- this is purely for debugging in the UI
// To be removed as soon as we can support trusty properly
status: {
conditions: [
{
message: 'This feature flag state is being ignored',
reason: 'IgnoredFeatureFlag',
status: 'False',
type: 'disableBiasMetricsAvailable',
},
],
},
};
};

const fetchOrCreateDashboardCR = async (fastify: KubeFastifyInstance): Promise<DashboardConfig> => {
return fastify.kube.customObjectsApi
.getNamespacedCustomObject(
Expand Down
6 changes: 3 additions & 3 deletions docs/dashboard-config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ The following are a list of features that are supported, along with there defaul
| disableKServeMetrics | false | Disables the ability to see KServe Metrics. |
| disableModelMesh | false | Disables the ability to select ModelMesh as a Serving Platform. |
| disableAcceleratorProfiles | false | Disables Accelerator profiles from the Admin Panel. |
| disableBiasMetrics | false | Disables Model Bias tab from Model Serving metrics. |
| disableTrustyBiasMetrics | false | Disables Model Bias tab from Model Serving metrics. |
| disablePerformanceMetrics | false | Disables Endpoint Performance tab from Model Serving metrics. |
| disableDistributedWorkloads | false | Disables Distributed Workload Metrics from the dashboard. |
| disableModelRegistry | false | Disables Model Registry from the dashboard. |
Expand Down Expand Up @@ -62,7 +62,7 @@ spec:
disableCustomServingRuntimes: false
disableAcceleratorProfiles: false
disableKServeMetrics: false
disableBiasMetrics: false
disableTrustyBiasMetrics: false
disablePerformanceMetrics: false
disableDistributedWorkloads: false
disableConnectionTypes: false
Expand Down Expand Up @@ -159,7 +159,7 @@ spec:
disableCustomServingRuntimes: false
disableAcceleratorProfiles: true
disableKServeMetrics: true
disableBiasMetrics: false
disableTrustyBiasMetrics: false
disablePerformanceMetrics: false
disableNIMModelServing: true
notebookController:
Expand Down
6 changes: 3 additions & 3 deletions frontend/src/__mocks__/mockDashboardConfig.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ type MockDashboardConfigType = {
disableModelMesh?: boolean;
disableAcceleratorProfiles?: boolean;
disablePerformanceMetrics?: boolean;
disableBiasMetrics?: boolean;
disableTrustyBiasMetrics?: boolean;
disableDistributedWorkloads?: boolean;
disableModelRegistry?: boolean;
disableConnectionTypes?: boolean;
Expand Down Expand Up @@ -51,7 +51,7 @@ export const mockDashboardConfig = ({
disableModelMesh = false,
disableAcceleratorProfiles = false,
disablePerformanceMetrics = false,
disableBiasMetrics = false,
disableTrustyBiasMetrics = false,
disableDistributedWorkloads = false,
disableModelRegistry = false,
disableConnectionTypes = true,
Expand Down Expand Up @@ -152,7 +152,7 @@ export const mockDashboardConfig = ({
disableCustomServingRuntimes,
disablePipelines,
disableProjectSharing: false,
disableBiasMetrics,
disableTrustyBiasMetrics,
disablePerformanceMetrics,
disableKServe,
disableKServeAuth,
Expand Down
52 changes: 26 additions & 26 deletions frontend/src/__tests__/cypress/cypress/tests/mocked/modelServing/modelMetrics.cy.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ import {

type HandlersProps = {
disablePerformanceMetrics?: boolean;
disableBiasMetrics?: boolean;
disableTrustyBiasMetrics?: boolean;
disableKServeMetrics?: boolean;
servingRuntimes?: ServingRuntimeKind[];
inferenceServices?: InferenceServiceKind[];
Expand Down Expand Up @@ -81,7 +81,7 @@ const mockTrustyDBSecret = (): SecretKind =>

const initIntercepts = ({
disablePerformanceMetrics,
disableBiasMetrics,
disableTrustyBiasMetrics,
disableKServeMetrics,
servingRuntimes = [mockServingRuntimeK8sResource({})],
inferenceServices = [mockInferenceServiceK8sResource({ isModelMesh: true })],
Expand All @@ -100,7 +100,7 @@ const initIntercepts = ({
cy.interceptOdh(
'GET /api/config',
mockDashboardConfig({
disableBiasMetrics,
disableTrustyBiasMetrics,
disablePerformanceMetrics,
disableKServeMetrics,
}),
Expand Down Expand Up @@ -210,7 +210,7 @@ const initIntercepts = ({
describe('Model Metrics', () => {
it('Empty State No Serving Data Available', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
hasServingData: false,
hasBiasData: false,
Expand All @@ -222,7 +222,7 @@ describe('Model Metrics', () => {

it('Serving Chart Shows Data', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
hasServingData: true,
hasBiasData: false,
Expand All @@ -234,7 +234,7 @@ describe('Model Metrics', () => {

it('Empty State No Bias Data Available', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
hasServingData: false,
hasBiasData: false,
Expand Down Expand Up @@ -264,7 +264,7 @@ describe('Model Metrics', () => {

it('Bias Charts Show Data', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
hasServingData: false,
hasBiasData: true,
Expand Down Expand Up @@ -294,7 +294,7 @@ describe('Model Metrics', () => {

it('Server metrics show no data available', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
hasServingData: false,
hasBiasData: false,
Expand All @@ -309,7 +309,7 @@ describe('Model Metrics', () => {

it('Server metrics show data', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
hasServingData: true,
hasBiasData: false,
Expand All @@ -324,7 +324,7 @@ describe('Model Metrics', () => {

it('Bias metrics is not configured', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
hasServingData: false,
hasBiasData: false,
Expand All @@ -339,7 +339,7 @@ describe('Model Metrics', () => {

it('Performance Metrics Tab Hidden', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: true,
hasServingData: false,
hasBiasData: false,
Expand All @@ -351,7 +351,7 @@ describe('Model Metrics', () => {

it('Bias Metrics Tab Hidden', () => {
initIntercepts({
disableBiasMetrics: true,
disableTrustyBiasMetrics: true,
disablePerformanceMetrics: false,
hasServingData: false,
hasBiasData: false,
Expand All @@ -363,7 +363,7 @@ describe('Model Metrics', () => {

it('Disable Trusty AI', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
hasServingData: false,
hasBiasData: false,
Expand Down Expand Up @@ -393,7 +393,7 @@ describe('Model Metrics', () => {

it('Enable Trusty AI', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
hasServingData: false,
hasBiasData: false,
Expand Down Expand Up @@ -461,7 +461,7 @@ describe('Model Metrics', () => {

it('Trusty AI enable service error', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
hasServingData: false,
hasBiasData: false,
Expand Down Expand Up @@ -510,7 +510,7 @@ describe('Model Metrics', () => {

it('Bias Metrics Show In Table', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
hasServingData: false,
hasBiasData: true,
Expand All @@ -537,7 +537,7 @@ describe('Model Metrics', () => {

it('Configure Bias Metric', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
hasServingData: false,
hasBiasData: true,
Expand Down Expand Up @@ -603,7 +603,7 @@ describe('Model Metrics', () => {
describe('KServe performance metrics', () => {
it('should inform user when area disabled', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
disableKServeMetrics: true,
hasServingData: false,
Expand All @@ -616,7 +616,7 @@ describe('KServe performance metrics', () => {

it('should show error when ConfigMap is missing', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
disableKServeMetrics: false,
hasServingData: true,
Expand All @@ -639,7 +639,7 @@ describe('KServe performance metrics', () => {

it('should inform user when serving runtime is unsupported', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
disableKServeMetrics: false,
hasServingData: true,
Expand All @@ -655,7 +655,7 @@ describe('KServe performance metrics', () => {

it('should handle a malformed graph definition gracefully', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
disableKServeMetrics: false,
hasServingData: true,
Expand All @@ -674,7 +674,7 @@ describe('KServe performance metrics', () => {

it('should display only 2 graphs, when the config specifies', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
disableKServeMetrics: false,
hasServingData: true,
Expand All @@ -695,7 +695,7 @@ describe('KServe performance metrics', () => {

it('charts should not error out if a query is missing and there is other data', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
disableKServeMetrics: false,
hasServingData: true,
Expand All @@ -718,7 +718,7 @@ describe('KServe performance metrics', () => {

it('charts should not error out if a query is missing and there is no data', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
disableKServeMetrics: false,
hasServingData: false,
Expand All @@ -741,7 +741,7 @@ describe('KServe performance metrics', () => {

it('charts should show data when serving data is available', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
disableKServeMetrics: false,
hasServingData: true,
Expand All @@ -761,7 +761,7 @@ describe('KServe performance metrics', () => {

it('charts should show empty state when no serving data is available', () => {
initIntercepts({
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
disableKServeMetrics: false,
hasServingData: false,
Expand Down
4 changes: 2 additions & 2 deletions frontend/src/concepts/areas/const.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ export const allFeatureFlags: string[] = Object.keys({
disableProjectSharing: false,
disableCustomServingRuntimes: false,
disablePipelines: false,
disableBiasMetrics: false,
disableTrustyBiasMetrics: false,
disablePerformanceMetrics: false,
disableKServe: false,
disableKServeAuth: false,
Expand Down Expand Up @@ -94,7 +94,7 @@ export const SupportedAreasStateMap: SupportedAreasState = {
reliantAreas: [SupportedArea.DS_PROJECTS_VIEW],
},
[SupportedArea.BIAS_METRICS]: {
featureFlags: ['disableBiasMetrics'],
featureFlags: ['disableTrustyBiasMetrics'],
requiredComponents: [StackComponent.TRUSTY_AI],
reliantAreas: [SupportedArea.MODEL_SERVING],
},
Expand Down
2 changes: 1 addition & 1 deletion frontend/src/k8sTypes.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1316,7 +1316,7 @@ export type DashboardCommonConfig = {
disableProjectSharing: boolean;
disableCustomServingRuntimes: boolean;
disablePipelines: boolean;
disableBiasMetrics: boolean;
disableTrustyBiasMetrics: boolean;
disablePerformanceMetrics: boolean;
disableKServe: boolean;
disableKServeAuth: boolean;
Expand Down
Loading
Loading