Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support for caliper graded profile #759

Merged
merged 24 commits into from
Apr 24, 2023
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ import { ExperimentUser } from '../models/ExperimentUser';
import { ExperimentUserService } from '../services/ExperimentUserService';
import { UpdateWorkingGroupValidator } from './validators/UpdateWorkingGroupValidator';
import { MonitoredDecisionPoint } from '../models/MonitoredDecisionPoint';
import { ISingleMetric, IGroupMetric, SERVER_ERROR } from 'upgrade_types';
import { ISingleMetric, IGroupMetric, SERVER_ERROR, ILogInput } from 'upgrade_types';
import { parse, toSeconds } from 'iso8601-duration';
import { FailedParamsValidator } from './validators/FailedParamsValidator';
import { ExperimentError } from '../models/ExperimentError';
import { FeatureFlag } from '../models/FeatureFlag';
Expand All @@ -31,6 +32,7 @@ import { Metric } from '../models/Metric';
import * as express from 'express';
import { AppRequest } from '../../types';
import { env } from '../../env';
import { CaliperLogEnvelope } from './validators/CaliperLogEnvelope';

interface IExperimentAssignment {
expId: string;
Expand Down Expand Up @@ -650,6 +652,70 @@ export class ExperimentClientController {
});
}


/**
* @swagger
* /log/caliper:
* post:
* description: Post Caliper format log data
* consumes:
* - application/json
* parameters:
* - in: body
* name: data
* required: true
* description: User Document
* tags:
* - Client Side SDK
* produces:
* - application/json
* responses:
* '200':
* description: Log data
* '500':
* description: null value in column "id\" of relation \"experiment_user\" violates not-null constraint
*/
@Post('log/caliper')
public async caliperLog(
@Body({ validate: { validationError: { target: false, value: false } } })
@Req()
request: AppRequest,
envelope: CaliperLogEnvelope
): Promise<Log[]> {
let logResponse: Log[] = [];
envelope.data.forEach(async log => {
request.logger.info({ message: 'Starting the log call for user' });
const userId = log.object.assignee.id
// getOriginalUserDoc call for alias
const experimentUserDoc = await this.getUserDoc(userId, request.logger);
if (experimentUserDoc) {
// append userDoc in logger
request.logger.child({ userDoc: experimentUserDoc });
request.logger.info({ message: 'Got the original user doc' });
}
const logs: ILogInput = {
"metrics": {
"attributes": log.generated.attempt.extensions.metrics.attributes || {},
"groupedMetrics": log.generated.attempt.extensions.metrics.groupedMetrics || [],
},
timestamp: log.eventTime
};

logs.metrics.attributes['duration'] = toSeconds(parse(log.generated.attempt.duration));
logs.metrics.attributes['scoreGiven'] = log.generated.scoreGiven;

let res = await this.experimentAssignmentService.dataLog(userId, [logs], {
logger: request.logger,
userDoc: experimentUserDoc,
})

logResponse.concat(res);

});
return logResponse;

}

/**
* @swagger
* /bloblog:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
import { IsNotEmpty, IsDefined, IsString, IsJSON, IsObject } from 'class-validator';
import { Attempt, CaliperActor, ScoreObject } from 'upgrade_types';

export class CaliperLogData {

@IsDefined()
@IsNotEmpty()
@IsString()
public profile: string;

@IsDefined()
@IsNotEmpty()
@IsJSON()
public actor: CaliperActor;

@IsDefined()
@IsNotEmpty()
@IsString()
public action: string;

@IsDefined()
@IsNotEmpty()
@IsString()
public eventTime: string;

@IsDefined()
@IsNotEmpty()
@IsJSON()
public object: Attempt;

@IsObject()
public extensions: object;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

object and Object Typescript types are less useful than they appear, as it will allow anything that is an object at all (arrays are objects!) and if you were to do write extensions.something it will complain that something is an unknown property of extensions.

There are a couple of alternatives that I see floated around to avoid object type where all you really know is that you could get an object with random stuff, which is either:

{ [key: string] : any } or Record<string, any>

I also see Record<string, unknown>, but I'm not 100% the benefit of unknown vs any. Not sure how you'd get around any or unknown though

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The end goal is to use the attempt extensions for additional metrics, so I could make it a ILogInput type? Is it too restricting to say that you can only include our metric format as additional data?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Record<string, unknown> is what we want if we are just throwing out this data and/or not doing any processing of those vars to try to access data on them or transform before writing to db.

Based on my thorough stack overflow skimming, any is like turning off type-checking entirely, where unknown means allow anything, but yell at me if I actually try to use it for anything.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or maybe the type can be ILogInput | Record<string, unknown>?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or maybe not; I think you'd just want Record<string, unknown> if we don't want to force the format... I think the use case would be to flexibly allow any unknown object, but yell at anyone who tries to access properties on extensions to remind them that nothing is guaranteed about these objects.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like Record<string, unknown> to just pass it on and save the metrics. If I do the same on the client lib side, does unknown make it hard to create the object?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good question... I guess it could. I think it's probably ok to say Record<string, any> if we're going to allow any objects, as they'd run into annoying issues if they were creating the object in steps:

const extensions: Record<string, unknown> = {};

extensions.prop.nestedProp = 4; // it would complain about nested prop being unknown

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how we're providing proper types actually though, or if anyone is using types for interacting with the library currently. If it's just being used as javascript, or if we aren't providing these types, none of it really matters in the client side.


@IsObject()
@IsNotEmpty()
@IsJSON()
public generated: ScoreObject;

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
import { IsNotEmpty, IsDefined, IsString } from 'class-validator';
import { CaliperLogData } from './CaliperLogData';


export class CaliperLogEnvelope {
@IsDefined()
@IsNotEmpty()
@IsString()
public sensor: string;

@IsDefined()
@IsNotEmpty()
@IsString()
public sendTime: string;

@IsDefined()
@IsNotEmpty()
@IsString()
public dataVersion: string;

public data: CaliperLogData[];
}
9 changes: 9 additions & 0 deletions clientlibs/js/src/UpgradeClient.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,13 @@ import getExperimentCondition from './functions/getExperimentCondition';
import markExperimentPoint from './functions/markExperimentPoint';
import getAllFeatureFlags from './functions/getAllfeatureFlags';
import log from './functions/log';
import logCaliper from './functions/logCaliper';
import setAltUserIds from './functions/setAltUserIds';
import addMetrics from './functions/addMetrics';
import getFeatureFlag from './functions/getFeatureFlag';
import init from './functions/init';
import * as uuid from 'uuid';
import { CaliperEnvelope } from '../../../types/src/Experiment/interfaces';

export default class UpgradeClient {
// Endpoints URLs
Expand All @@ -31,6 +33,7 @@ export default class UpgradeClient {
failedExperimentPoint: null,
getAllFeatureFlag: null,
log: null,
logCaliper: null,
altUserIds: null,
addMetrics: null,
};
Expand Down Expand Up @@ -59,6 +62,7 @@ export default class UpgradeClient {
failedExperimentPoint: `${hostUrl}/api/v1/failed`,
getAllFeatureFlag: `${hostUrl}/api/v1/featureflag`,
log: `${hostUrl}/api/v1/log`,
logCaliper: `${hostUrl}/api/v1/logCaliper`,
altUserIds: `${hostUrl}/api/v1/useraliases`,
addMetrics: `${hostUrl}/api/v1/metric`,
};
Expand Down Expand Up @@ -179,6 +183,11 @@ export default class UpgradeClient {
return await log(this.api.log, this.userId, this.token, this.clientSessionId, value, sendAsAnalytics);
}

async logCaliper(value: CaliperEnvelope, sendAsAnalytics = false): Promise<Interfaces.ILog[]> {
this.validateClient();
return await logCaliper(this.api.logCaliper, this.userId, this.token, this.clientSessionId, value, sendAsAnalytics);
}

async setAltUserIds(altUserIds: string[]): Promise<Interfaces.IExperimentUserAliases> {
this.validateClient();
return await setAltUserIds(this.api.altUserIds, this.userId, this.token, this.clientSessionId, altUserIds);
Expand Down
30 changes: 30 additions & 0 deletions clientlibs/js/src/functions/logCaliper.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
import { Types, Interfaces } from '../identifiers';
import fetchDataService from '../common/fetchDataService';
import { CaliperEnvelope } from '../../../../types/src/Experiment/interfaces';

export default async function logCaliper(
url: string,
userId: string,
token: string,
clientSessionId: string,
value: CaliperEnvelope,
sendAsAnalytics = false
): Promise<Interfaces.ILog[]> {
try {
const logResponse = await fetchDataService(
url,
token,
clientSessionId,
value,
Types.REQUEST_TYPES.POST,
sendAsAnalytics
);
if (logResponse.status) {
return logResponse.data;
} else {
throw new Error(JSON.stringify(logResponse.message));
}
} catch (error) {
throw new Error(error.message);
}
}
Loading