Azure Endpoint not working

[nubgamerz]

openai.error.InvalidRequestError: Must provide an 'engine' or 'deployment_id' parameter to create a <class 'openai.api_resources.chat_completion.ChatCompletion'>

It seems some of azure's required parameters are missing still.

Here is the error in full:

File "G:\Code\memgpt\MemGPT\main.py", line 336, in <module> app.run(run) File "G:\Code\memgpt\MemGPT\.venv\Lib\site-packages\absl\app.py", line 308, in run _run_main(main, args) File "G:\Code\memgpt\MemGPT\.venv\Lib\site-packages\absl\app.py", line 254, in _run_main sys.exit(main(argv)) ^^^^^^^^^^ File "G:\Code\memgpt\MemGPT\main.py", line 334, in run loop.run_until_complete(main()) File "C:\Python311\Lib\asyncio\base_events.py", line 650, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "G:\Code\memgpt\MemGPT\main.py", line 312, in main new_messages, heartbeat_request, function_failed, token_warning = await memgpt_agent.step(user_message, first_message=False, skip_verify=FLAGS.no_verify) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "G:\Code\memgpt\MemGPT\memgpt\agent.py", line 545, in step raise e File "G:\Code\memgpt\MemGPT\memgpt\agent.py", line 483, in step response = await get_ai_reply_async(model=self.model, message_sequence=input_message_sequence, functions=self.functions) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "G:\Code\memgpt\MemGPT\memgpt\agent.py", line 115, in get_ai_reply_async raise e File "G:\Code\memgpt\MemGPT\memgpt\agent.py", line 96, in get_ai_reply_async response = await acreate( ^^^^^^^^^^^^^^ File "G:\Code\memgpt\MemGPT\memgpt\openai_tools.py", line 98, in wrapper raise e File "G:\Code\memgpt\MemGPT\memgpt\openai_tools.py", line 76, in wrapper return await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "G:\Code\memgpt\MemGPT\memgpt\openai_tools.py", line 108, in acompletions_with_backoff return await openai.ChatCompletion.acreate(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "G:\Code\memgpt\MemGPT\.venv\Lib\site-packages\openai\api_resources\chat_completion.py", line 45, in acreate return await super().acreate(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "G:\Code\memgpt\MemGPT\.venv\Lib\site-packages\openai\api_resources\abstract\engine_api_resource.py", line 216, in acreate ) = cls.__prepare_create_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "G:\Code\memgpt\MemGPT\.venv\Lib\site-packages\openai\api_resources\abstract\engine_api_resource.py", line 85, in __prepare_create_request raise error.InvalidRequestError( openai.error.InvalidRequestError: Must provide an 'engine' or 'deployment_id' parameter to create a <class 'openai.api_resources.chat_completion.ChatCompletion'>

[streichsbaer]

It works for me on a mac.
Are you starting it with python3 main.py --use_azure_openai and are exporting the right environment variables?

AZURE_OPENAI_KEY=<api_key>
AZURE_OPENAI_ENDPOINT=https://xxx.openai.azure.com/
AZURE_OPENAI_VERSION=<api_version, e.g. 2023-07-01-preview>
AZURE_OPENAI_DEPLOYMENT=<engine>

[nubgamerz]

yes, I'm using Windows.

If you don't set the environment vars, it will say "you need to provide openai key" before getting this far.

I tried with setting environment variables (get the above error), also tried hardcoding the variables in the code, and replacing os.getenv with just the names of the appropriate vars.... same error as above.

[nubgamerz]

Not to worry, got it working, there was a spelling error in the deployment ID.

But when trying to run with docs:

python .\main.py --archival_storage_files="memgpt/personas/examples/preload_archival/*.txt" --persona=memgpt_doc --human=basic --use_azure_openai

It returns:

File "G:\Code\memgpt\MemGPT\memgpt\agent.py", line 489, in step
raise Exception(f'Hit first message retry limit ({first_message_retry_limit})')
Exception: Hit first message retry limit (10)
Windows fatal exception: access violation

I'm assuming this is a windows limitation or something.

[streichsbaer]

Not sure if it's a Windows limitation.

I can confirm that your command is working on my Mac.
Try adding the --no-verify=true flag which seems to bypass the logic that errors out for you.

python .\main.py --archival_storage_files="memgpt/personas/examples/preload_archival/*.txt" --persona=memgpt_doc --human=basic --use_azure_openai --no_verify=true

[nubgamerz]

Unfortunately that didn't work either.

(base) PS G:\Code\memgpt\MemGPT> python .\main.py --archival_storage_files="memgpt/personas/examples/preload_archival/*.txt" --persona=memgpt_doc --human=basic --use_azure_openai --no_verify=true

G:\Code\memgpt\MemGPT\main.py:496: DeprecationWarning: There is no current event loop
  loop = asyncio.get_event_loop()
⚙️ Using legacy command line arguments.
Running... [exit by typing '/exit']
Preloaded 12 chunks into archival memory.
Initializing InMemoryStateManager with agent object
InMemoryStateManager.all_messages.len = 4
InMemoryStateManager.messages.len = 4
💭 Bootup sequence complete. Persona activated. Testing messaging functionality.
Hit enter to begin (will request first MemGPT message)
[A[K
Traceback (most recent call last):                                                                                                                                                        
  File "G:\Code\memgpt\MemGPT\.venv\Lib\site-packages\openai\openai_object.py", line 59, in __getattr__
    return self[k]
           ~~~~^^^
KeyError: 'content'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "G:\Code\memgpt\MemGPT\main.py", line 499, in <module>
    app.run(run)
  File "G:\Code\memgpt\MemGPT\.venv\Lib\site-packages\absl\app.py", line 308, in run
    _run_main(main, args)
  File "G:\Code\memgpt\MemGPT\.venv\Lib\site-packages\absl\app.py", line 254, in _run_main
    sys.exit(main(argv))
             ^^^^^^^^^^
  File "G:\Code\memgpt\MemGPT\main.py", line 497, in run
    loop.run_until_complete(main())
  File "C:\Python311\Lib\asyncio\base_events.py", line 650, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "G:\Code\memgpt\MemGPT\main.py", line 471, in main
    ) = await memgpt_agent.step(
        ^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Code\memgpt\MemGPT\memgpt\agent.py", line 545, in step
    raise e
  File "G:\Code\memgpt\MemGPT\memgpt\agent.py", line 499, in step
    all_response_messages, heartbeat_request, function_failed = await self.handle_ai_response(response_message)
                                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Code\memgpt\MemGPT\memgpt\agent.py", line 322, in handle_ai_response
    await self.interface.internal_monologue(response_message.content)
                                            ^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Code\memgpt\MemGPT\.venv\Lib\site-packages\openai\openai_object.py", line 61, in __getattr__
    raise AttributeError(*err.args)
AttributeError: content
Windows fatal exception: access violation

Current thread 0x00004d90 (most recent call first):
  Garbage-collecting
  File "C:\Python311\Lib\asyncio\proactor_events.py", line 81 in __repr__
  File "C:\Python311\Lib\asyncio\proactor_events.py", line 115 in __del__

[nubgamerz]

To add to this, it's the exact same error just running in normal chat mode:

python .\main.py --use_azure_openai --no_verify=true

I have to add the no verify to get passed a different error:

Exception: Hit first message retry limit (10)

[streichsbaer]

There may be some bugs due to how Windows works as compared to Mac that result in this issue.
I'd suggest running all of this in a docker container first, to make sure it actually works.
Otherwise, maybe we have Windows users here that can shed some light as well.

[nubgamerz]

Yeah, so I got my macbook out and it's working there. So this is clearly a windows related issue.

I will say though, that even on a mac, creating embeddings doesn't work with azure

main.py --archival_storage_files_compute_embeddings="whatever/*.ext" --persona=memgpt_doc --human=basic --use_azure_openai

It seems this embeddings function is not checking for azure open ai usage, and it's trying to look for openAI key.

Azure OpenAI models do support embeddings.

https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/embeddings?tabs=console

[cpacker]

Should be fixed by #140