Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenAIFunction parameter validation #59

Open
remyma opened this issue Aug 21, 2024 · 2 comments
Open

OpenAIFunction parameter validation #59

remyma opened this issue Aug 21, 2024 · 2 comments

Comments

@remyma
Copy link

remyma commented Aug 21, 2024

Hello,

I think that JsonSchema interface is not complete.

https://github.com/VapiAI/web/blob/cb7f315e49889c84e1b03bfaa1664d14d6b62608/api.ts#L454

It's missing some properties useful for validation, for instance minLength, maxLength and format for string type:
https://json-schema.org/understanding-json-schema/reference/string#format

That would help on the tool specification so that LLM is aware of expected format.

For instance, I have a tool function, with startDate and endDate parameters.
This parameters should be date-time (type: string, format: date-time)

export const myFunction: OpenAIFunction = {
  name: 'myfunction',
  description: "...",
  parameters: {
    type: "object",
    properties: {
      startDate: {
        type: "string",
        description: "Start date.",
      },
      endDate: {
        type: "string",
        description: "End date.",
      },
    },
    required: ["startDate", "endDate"],
  },
};

Most of the time LLM call this function with correct format (2018-11-13T20:20:39+00:00) for parameters.
But it already happened that LLM called my function with a wrong format (2018-11-13) instead of expected one.

If I could specify the format, that would fix the issue I think.

For now, I will specify it in the parameter description.
But I think that would be great if the JsonSchema definition in Vapi could be fully compliant with JsonSchema.

Also in openAI doc there is an example with enums to enforce parameter validation:
https://platform.openai.com/docs/guides/function-calling/use-enums-for-function-arguments-when-possible

I don't think we can have as much control on validation on Vapi.

What do you think?

@nikhilro
Copy link
Contributor

We don't expose the full schema in types to not overwhelm people but entirety of it is supported it in the api. To unblock yourself, just use as any

@remyma
Copy link
Author

remyma commented Aug 22, 2024

ok, I'll do that. I understand your point about not overwhelming people, although I think it would be clearer to stick to the standard definition. It helps suggest to people that they may benefit from better validation of the parameters

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants