New Major Release 3.0.0

[sashirestela]

What's new?

• Full Support for Assistants API v2 🔥🔥
  • Assistants, Threads, Messages, Runs, Run Steps. ✅
  • File Search Tool, Vector Stores, Vector Stores Batches. ✅
  • Full Streaming, Pooling Helpers if not using streaming. ✅
  • Demo code using Functions, File Search and Streaming. ✅
• Functions & Tools code was revamped to be shared between
  Assistants and Chat Completions. 🛠
• Code for Assistants API v1 was entirely removed. 🛠

Notes

• Code running on previous releases won't be compatible with this release.
• It is highly recommended you migrate your code to use this new release.
• If you need to work with Assistants v1, you will have to work with the previous release.

See more details here

[the-gigi]

@sashirestela some useful functionality seems to be removed in 3.0.0 and I don't see a replacement. is that correct?

    public boolean isActionRequired() {
        return (requiredAction != null);
    }

    public List<ToolCall> getRequiredToolCalls() {
        if (requiredAction != null
                && requiredAction.getSubmitToolOutputs() != null
                && requiredAction.getSubmitToolOutputs().getToolCalls() != null) {
            return requiredAction.getSubmitToolOutputs().getToolCalls();
        } else {
            return List.of();
        }
    }

In addition, the revamping functions and tool code resulted in FunctionDef exposing getFunctionalClass(), but ToolFunctionDef doesn't. In our code we annotate tool function objects and the inability to get the class is a problem.

Does it make sense to restore these capabilities? The removed functions are not a big deal. We can just implement them ourselves (although if they are useful to us maybe they're useful to other users). But, not having getFunctionalClass() on ToolFunctionDef is not simple to handle.

[sashirestela]

You don't need to use ToolFunctionDef directly, it exists just for deserialization reasons.

After the change, you will use FunctionDef for whatever: Chat Completions or Assistants V2. Check the ConversationV2Demo code.

I don't remember where are those functions that I removed?

[the-gigi]

I'll see about using FunctionDef. The functions are from the ThreadRun class.

[the-gigi]

hmmm... it's not so simple. we're passing List<Tool> around and then want to check for annotations on the tool. Here is a sample code:

    String getExternalActionableToolName(List<Tool> chatTools) {
        return chatTools.stream()
            .filter(
                f ->
                    f.getFunction()
                        .getFunctionalClass()
                        .isAnnotationPresent(ExternalActionableFunction.class))
            .findFirst()
            .map(f -> f.getFunction().getName())
            .orElse(null);
    }

The f.getFunction() returns ToolFunctionDef and then the following getFunctionClass() call fails to compile.

We construct a FunctionExecutioner early and later call getFunctionTools() which returns the List<Tool> and then we lose the ability to check the class of the tool for annotations.

[sashirestela]

Just to clarify, when you say:

Does it make sense to restore these capabilities?

I suppose you are referring to helper functions in ThreadRun. If that is the case, no problem, in the next release I can restore them.

But, everything around ToolFunctionDef is new, so nothing has been removed, even more, ToolFunctionDef is kind of a replacement for the old AssistantFunction, which didn't have any functionalClass field.

I suggest keeping your functions definitions on a separate list and use them to feed your request, then refactor your getExternalActionableToolName method to use that list of FunctionDef instead of Tool.

[the-gigi]

Just to clarify, when you say:

Does it make sense to restore these capabilities?

I suppose you are referring to helper functions in ThreadRun. If that is the case, no problem, in the next release I can restore them.

Yes. that's what I mean. But, those are really syntactic sugar. If you prefer to keep them out to keep the public API lean that's very valid.

But, everything around ToolFunctionDef is new, so nothing has been removed, even more, ToolFunctionDef is kind of a replacement for the old AssistantFunction, which didn't have any functionalClass field.

You're right. It seems that by consolidating the ChatFunction and AssistantFunction into common we lost the ability to access the functionalClass of chat functions tools when their internal representation is stored as ToolFunctionDef. btw, why was there asymmetry originally in 2.x between ChatFunction and Assistant function? it would be helpful to have the ability to attach metadata to all tools/functions in the form of annotations on their class.

I suggest keeping your functions definitions on a separate list and use them to feed your request, then refactor your getExternalActionableToolName method to use that list of FunctionDef instead of Tool.

Yes. if you disagree that exposing the functional class is a useful feature then we will have to do that and maintain a separate mapping between tools and their class and make that mapping available to the code that requires it. Hopefully, that's a service that simple-openai can provide.

[sashirestela]

If there was any asymmetry in the past, it has been solved in this 3.x release. Now, ChatRequest as well as AssistantRequest both have a field: List<Tool> tools.

Why are we using ToolFunctionDef with the field JsonNode parameters?
This is because the OpenAI API only understands Json representation of functions for both requests and responses. JsonNode is a bidirectional field, but Class (the type of functionalClass) is unidirectional, so we cannot use it to receive the response from OpenAI for the Assistant API. Technically speaking, it is hard to try to deserialize from Json to Class.

The class FunctionDef is just a kind of helper to ease the definition of functions, instead of being a structural class.

[the-gigi]

OK. We will manage the mapping Tool -> Class in our code. It's unfortunate that we relied on this capability, but I agree that it's not the job of simple-openai to support attaching metadata to classes by exposing the original functional class.

I think the core issue is that in 2.x ChatFunction had a global @Getter

@Getter
@Builder
public class ChatFunction {

    @Required
    private String name;

    @JsonInclude(Include.NON_NULL)
    private String description;

    @Required
    @JsonSerialize(using = ParametersSerializer.class)
    @JsonProperty("parameters")
    private Class<? extends Functional> functionalClass;
}

the functionalClass was conceptually a write-only property. The global @Getter made it readable and then we utilized it to pass metadata in the form of annotations on the class.

[sashirestela]

We construct a FunctionExecutioner early

Are you replicating the behavior of the existing class FunctionExecutor?

Maybe you could call any validation logic in the beginning of your overridable execute() method of each function definition class?

[the-gigi]

no. we use the simple-openai FunctionExecutor. We have a complex machinery with generics, etc around the functionals themselves.

[the-gigi]

@sashirestela One more issue came up as a result of the refactoring. There is inconsistency in the use of Tool and AssistantTool.
For example, AssistantRequest requires Tool:

public class AssistantRequest {
    ...
    private List<Tool> tools;
    ...
}

While AssistantModifyRequest requires AssistantTool

public class AssistantModifyRequest {
    ...
    private List<AssistantTool> tools;
    ...
}

AssistantTool extends Tool. Looking at the code it seems that all AssistantTool adds is the pre-defined tools CODE_INTERPRETER and FILE_SEARCH. Is it necessary to actually instantiate AssistantTool objects?

The idea of the refactor to a common tool is that the Tool functionality is the same between chat and assistants. The fact that assistants have pre-defined tool doesn't require a new type of AssstantTool.

Consider replacing all references to AssistantTool with just Tool. The AssistantTool class itself will still provide the CODE_INTEPRETER and FILE_SEARCH tools, but it will provide them as Tool instances.

This will simplify the code, make it consistent and utilize just the common Tool for all use cases of chat and assistant APIs.

I tried it locally and it and it works just fine (all the tests pass and all the demos).

[sashirestela]

Let me check tomorrow