CRITICAL BUG: images.generate does not work on Azure OpenAI #692

nicolaschaillan · 2023-11-06T23:31:19Z

        openai_api_version = "2023-09-01-preview"
        client = AzureOpenAI(azure_endpoint = openai_api_base, api_key=openai_api_key, api_version=openai_api_version)

        response = client.images.generate(
            prompt=prompt,
            size=size,
            n=n
        )

Fails with:

{'error': {'code': '404', 'message': 'Resource not found'}}

The text was updated successfully, but these errors were encountered:

kristapratico · 2023-11-07T00:04:53Z

Hi @nicolaschaillan, Azure OpenAI Dall-E is not supported in the v1 SDK yet, but support will be added soon.

https://learn.microsoft.com/en-us/azure/ai-services/openai/dall-e-quickstart?tabs=command-line&pivots=programming-language-python#install-the-python-sdk

nicolaschaillan · 2023-11-07T00:06:20Z

This should be documented. Once folks migrate it's impossible to go back with all those changes and now I'm left with no option for our DALL E deployment... Not good! Any ETA for the support?

We have hundreds of users on Azure OpenAI using it.

kristapratico · 2023-11-07T02:16:42Z

@nicolaschaillan I will share as soon as I have an ETA. As a stop-gap, it is possible to access Azure OpenAI DALL-E by passing in a httpx custom transport:

import time
import json
import httpx
import openai


class CustomHTTPTransport(httpx.HTTPTransport):
    def handle_request(
        self,
        request: httpx.Request,
    ) -> httpx.Response:
        if "images/generations" in request.url.path and request.url.params[
            "api-version"
        ] in [
            "2023-06-01-preview",
            "2023-07-01-preview",
            "2023-08-01-preview",
            "2023-09-01-preview",
            "2023-10-01-preview",
        ]:
            request.url = request.url.copy_with(path="/openai/images/generations:submit")
            response = super().handle_request(request)
            operation_location_url = response.headers["operation-location"]
            request.url = httpx.URL(operation_location_url)
            request.method = "GET"
            response = super().handle_request(request)
            response.read()

            timeout_secs: int = 120
            start_time = time.time()
            while response.json()["status"] not in ["succeeded", "failed"]:
                if time.time() - start_time > timeout_secs:
                    timeout = {"error": {"code": "Timeout", "message": "Operation polling timed out."}}
                    return httpx.Response(
                        status_code=400,
                        headers=response.headers,
                        content=json.dumps(timeout).encode("utf-8"),
                        request=request,
                    )

                time.sleep(int(response.headers.get("retry-after", 10)))
                response = super().handle_request(request)
                response.read()

            if response.json()["status"] == "failed":
                error_data = response.json()
                return httpx.Response(
                    status_code=400,
                    headers=response.headers,
                    content=json.dumps(error_data).encode("utf-8"),
                    request=request,
                )

            result = response.json()["result"]
            return httpx.Response(
                status_code=200,
                headers=response.headers,
                content=json.dumps(result).encode("utf-8"),
                request=request,
            )
        return super().handle_request(request)


client = openai.AzureOpenAI(
    azure_endpoint="<azure_endpoint>",
    api_key="<api_key>",
    api_version="<api_version>",
    http_client=httpx.Client(
        transport=CustomHTTPTransport(),
    ),
)
image = client.images.generate(prompt="a cute baby seal")

print(image.data[0].url)

Once support is officially added, you can remove the http_client keyword argument and the custom transport class from your code and all should work as expected. Let me know if that helps.

nicolaschaillan · 2023-11-07T02:38:45Z

Thanks! Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Krista Pratico ***@***.***> Sent: Monday, November 6, 2023 9:16:52 PM To: openai/openai-python ***@***.***> Cc: Nicolas Chaillan ***@***.***>; Mention ***@***.***> Subject: Re: [openai/openai-python] CRITICAL BUG: images.generate does not work on Azure OpenAI (Issue #692) @nicolaschaillan<https://github.com/nicolaschaillan> I will share as soon as I have an ETA. As a stop-gap, it is possible to access Azure OpenAI DALL-E by passing in a httpx custom transport: import time import json import httpx import openai class CustomHTTPTransport(httpx.HTTPTransport): def handle_request( self, request: httpx.Request, ) -> httpx.Response: if "images/generations" in request.url.path and request.url.params[ "api-version" ] in [ "2023-06-01-preview", "2023-07-01-preview", "2023-08-01-preview", "2023-09-01-preview", "2023-10-01-preview", ]: request.url = request.url.copy_with(path="/openai/images/generations:submit") response = super().handle_request(request) operation_location_url = response.headers["operation-location"] request.url = httpx.URL(operation_location_url) request.method = "GET" response = super().handle_request(request) response.read() timeout_secs: int = 120 start_time = time.time() while response.json()["status"] not in ["succeeded", "failed"]: if time.time() - start_time > timeout_secs: raise openai.OpenAIError("Operation polling timed out.") time.sleep(int(response.headers.get("retry-after")) or 10) response = super().handle_request(request) response.read() if response.json()["status"] == "failed": error_data = response.json()["error"] message = error_data.get("message", "Operation failed") code = error_data.get("code") raise openai.OpenAIError(message, code) result = response.json()["result"] return httpx.Response( status_code=200, headers=response.headers, content=json.dumps(result).encode("utf-8"), request=request, ) client = openai.AzureOpenAI( azure_endpoint="<azure_endpoint>", api_key="<api_key>", api_version="<api_version>", http_client=httpx.Client( transport=CustomHTTPTransport(), ), ) image = client.images.generate(prompt="a cute baby seal") print(image.data[0].url) Once support is officially added, you can remove the http_client keyword argument and the custom transport class from your code and all should work as expected. Let me know if that helps. — Reply to this email directly, view it on GitHub<#692 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABK2VBVJHUV4EEZ35TIUMQ3YDGKZJAVCNFSM6AAAAAA7AJI3A6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJXGE4DCNZUGM>. You are receiving this because you were mentioned.Message ID: ***@***.***>

ashjeanbird · 2023-11-10T01:53:48Z

Hey @kristapratico, I made an update to the docs per @nicolaschaillan suggestion. Can you please take a look to see if it works and if there is anything else that I should add? Happy to add any additional details.

StephenHodgson · 2023-11-10T15:06:44Z

It is because msft adds an additional :submit part to the end of the uri they specify for generation.
You can easily append it to your endpoint request as needed.

I also added feedback to msft to remove this uri part:

https://feedback.azure.com/d365community/idea/f2204961-d97f-ee11-a81c-6045bdb7ea56

kristapratico · 2023-11-10T18:05:19Z

@ashjeanbird I'll take a look.

@StephenHodgson yes, the Azure Dall-E endpoint contains a :submit for the initial call. However, it's not enough to just append this to the endpoint. The initial call returns a status monitor which has to be checked until the final result is ready. This code shows how to workaround this for now: #692 (comment)

StephenHodgson · 2023-11-10T18:58:24Z

Yeah I don't understand why Microsoft went so far different with this endpoint implementation. Super frustrating.

ashjeanbird · 2023-11-10T19:01:28Z

@kristapratico thank you for checking out my PR #766. I have made a change to reflect the known issues verbiage link and the associated workaround.

kristapratico · 2023-11-15T17:46:45Z

@nicolaschaillan @StephenHodgson @ashjeanbird Azure OpenAI now supports Dall-E 3. See this Quickstart for details. It is supported on version 1.2.4 or greater of the openai library.

Please note that if you wish to continue to use Dall-E 2 you must either use the 0.28.1 version of the library or use 1.X version with the workaround.

nicolaschaillan · 2023-11-15T22:23:16Z

@nicolaschaillan @StephenHodgson @ashjeanbird Azure OpenAI now supports Dall-E 3. See this Quickstart for details. It is supported on version 1.2.4 or greater of the openai library.

Please note that if you wish to continue to use Dall-E 2 you must either use the 0.28.1 version of the library or use 1.X version with the workaround.

This doesn't work whatsoever on my East US API...

{'error': {'code': 'DeploymentNotFound', 'message': 'The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.'}}

Any clue why?

kristapratico · 2023-11-15T22:34:45Z

Are you using Dall-E 2 or 3?

Dall-E 3 is currently only available on SwedenCentral: https://learn.microsoft.com/en-us/azure/ai-services/openai/dall-e-quickstart?tabs=dalle3%2Ccommand-line&pivots=programming-language-python

You'll need to deploy the dall-e-3 model (you can do this from the Azure OpenAI Studio) and then target the deployment name in the method call:

result = client.images.generate(
    model="dalle3", # the name of your DALL-E 3 deployment
    prompt="a close-up of a bear walking through the forest",
)

nicolaschaillan · 2023-11-15T22:47:08Z

I tried both. Both don't work.

I use openai_api_version = "2023-12-01-preview"

model = "dalle2"

Fails.

We can't use Sweden for the U.S government work obviously, this should be better documented... list of supported models should be in a single location and easy to find. This is just a plain mess/nightmare.

nicolaschaillan · 2023-11-15T22:49:12Z

Btw on East, I don't see a way to add DALL-E but yet the example code is there and I can generate images in the API with the old code (without the new client)

kristapratico · 2023-11-16T00:22:05Z

I'm sorry for the confusion. I'm relaying your feedback to the team.

For clarity,

Azure OpenAI supports two models for image generation: Dall-E 2 and Dall-E 3

Dall-E 2 is available in East US. You can use version 0.28.1 of the openai library to access it. If you want to use this model version with v1.X of the openai library, then you can copy the code from the workaround.

Dall-E 3 is currently only available in SwedenCentral. You can use v1.2.4+ of the openai library to access it. Note that you must first deploy the dall-e-3 model from the Azure OpenAI studio and pass your deployment name in the model parameter when calling the API.

I am reaching out, but not sure yet when Dall-E 3 will be supported in US regions. The model documentation will be updated when it becomes available in more regions: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models#dall-e-models-preview

nicolaschaillan · 2023-11-16T00:37:14Z

Thanks, all these breaking changes because of MSFT are quite annoying, not going to lie. Why can't I use the new 1.2.4+ on DALL-E in East region?! Seems like it should be quite simple?

StephenHodgson · 2023-11-16T00:40:57Z

I'm guessing whenever this goes in Azure/azure-rest-api-specs#26582

brunobraga · 2023-12-01T10:18:44Z

I am having this issue as well... If helps anyone, solution below.

Code that I expected to work (Dall-E 2 - US East):

client = AzureOpenAI(
    api_key=os.getenv("AZURE_OPENAI_API_KEY"),
    azure_endpoint=f'https://{os.getenv("AZURE_OPENAI_API_INSTANCE_NAME")}.openai.azure.com',
    api_version=os.getenv('AZURE_OPENAI_API_VERSION'),
)
response = client.images.generate(
    prompt='dancing cat',
    size='512x512',
    n=1
)
image_url = response.data[0].url

Alternative code that works fine (Dall-E 2):

url = "{}/openai/images/generations:submit?api-version={}".format(
    f'https://{os.getenv("AZURE_OPENAI_API_INSTANCE_NAME")}.openai.azure.com',
    os.getenv('AZURE_OPENAI_API_VERSION')
)
headers= { "api-key": os.getenv("AZURE_OPENAI_API_KEY"), "Content-Type": "application/json" }
body = {
    "prompt": "dancing cat",
    "n": 1,
    "size": '512x512'
}
submission = requests.post(url, headers=headers, json=body)
operation_location = submission.headers['Operation-Location']
status = ""
while (status != "succeeded"):
    time.sleep(3) # workaround rate limit
    response = requests.get(operation_location, headers=headers)
    status = response.json()['status']
image_url = response.json()['result']['data'][0]['url']

Code currently working for Dall-E 3 (Sweden Central):

client = AzureOpenAI(
    api_key=os.getenv("AZURE_OPENAI_API_KEY"),
    azure_endpoint=f'https://{os.getenv("AZURE_OPENAI_API_INSTANCE_NAME")}.openai.azure.com',
    api_version=os.getenv('AZURE_OPENAI_API_VERSION'),
)
response = client.images.generate(
    prompt='dancing cat',
    model='dall-e-3',
    n=1
)
image_url = response.data[0].url

cohml · 2024-01-05T16:58:24Z

@kristapratico Is this workaround (i.e., implementing CustomHTTPTransport ) still required for accessing DALL-E via Azure OpenAI?

kristapratico · 2024-01-05T17:18:08Z

@kristapratico Is this workaround (i.e., implementing CustomHTTPTransport ) still required for accessing DALL-E via Azure OpenAI?

The workaround is only necessary if you're using Azure with DALL-E 2. Azure DALL-E 3 is supported by the library.

cohml · 2024-01-05T17:28:30Z

The workaround is only necessary if you're using Azure with DALL-E 2. Azure DALL-E 3 is supported by the library.

Thanks for the quick reply.

Do you know if that is set to change at some point? Or if you're forever stuck with DALL-E 2, are you forever stuck with CustomHTTPTransport?

kristapratico · 2024-01-05T17:41:34Z

The workaround is only necessary if you're using Azure with DALL-E 2. Azure DALL-E 3 is supported by the library.

Thanks for the quick reply.

Do you know if that is set to change at some point? Or if you're forever stuck with DALL-E 2, are you forever stuck with CustomHTTPTransport?

As far as I know, there won't be updates to Azure DALL-E 2 in the future, meaning the workaround will be necessary to use it with this library. Are you able to migrate to DALL-E 3?

StephenHodgson · 2024-01-05T19:41:06Z

Do you know if that is set to change at some point? Or if you're forever stuck with DALL-E 2, are you forever stuck with CustomHTTPTransport?

As far as I know, there won't be updates to Azure DALL-E 2 in the future, meaning the workaround will be necessary to use it with this library. Are you able to migrate to DALL-E 3?

Good to know so I can document this in my client libraries.

ronaldchoi · 2024-02-08T08:12:57Z

@nicolaschaillan I will share as soon as I have an ETA. As a stop-gap, it is possible to access Azure OpenAI DALL-E by passing in a httpx custom transport:
import time
import json
import httpx
import openai


class CustomHTTPTransport(httpx.HTTPTransport):
    def handle_request(
        self,
        request: httpx.Request,
    ) -> httpx.Response:
        if "images/generations" in request.url.path and request.url.params[
           ...
Once support is officially added, you can remove the http_client keyword argument and the custom transport class from your code and all should work as expected. Let me know if that helps.

This workaround does not work with my Azure setup and always hits the "[429 Too Many Requests]" errors.

I was able to fix it by adding a default value for the "retry-after" header in case it doesn't exist:

time.sleep(int(response.headers.get("retry-after", 10)))

kristapratico · 2024-02-08T17:22:09Z

@ronaldchoi thanks for spotting the bug. I'll update the example in my comment.

ashjeanbird mentioned this issue Nov 10, 2023

Adds DALL-E note to Azure OpenAI in v1 SDK #766

Merged

1 task

StephenHodgson mentioned this issue Nov 15, 2023

Revert recent changes to image generation for azure RageAgainstThePixel/OpenAI-DotNet#170

Closed

StephenHodgson mentioned this issue Nov 16, 2023

Remove DALL-E note #830

Merged

1 task

rattrayalex closed this as completed Jan 10, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CRITICAL BUG: images.generate does not work on Azure OpenAI #692

CRITICAL BUG: images.generate does not work on Azure OpenAI #692

nicolaschaillan commented Nov 6, 2023

kristapratico commented Nov 7, 2023

nicolaschaillan commented Nov 7, 2023

kristapratico commented Nov 7, 2023 •

edited

Loading

nicolaschaillan commented Nov 7, 2023 via email

ashjeanbird commented Nov 10, 2023

StephenHodgson commented Nov 10, 2023

kristapratico commented Nov 10, 2023

StephenHodgson commented Nov 10, 2023

ashjeanbird commented Nov 10, 2023

kristapratico commented Nov 15, 2023

nicolaschaillan commented Nov 15, 2023

kristapratico commented Nov 15, 2023

nicolaschaillan commented Nov 15, 2023

nicolaschaillan commented Nov 15, 2023

kristapratico commented Nov 16, 2023

nicolaschaillan commented Nov 16, 2023

StephenHodgson commented Nov 16, 2023

brunobraga commented Dec 1, 2023 •

edited

Loading

cohml commented Jan 5, 2024 •

edited

Loading

kristapratico commented Jan 5, 2024

cohml commented Jan 5, 2024

kristapratico commented Jan 5, 2024

StephenHodgson commented Jan 5, 2024

ronaldchoi commented Feb 8, 2024 •

edited

Loading

kristapratico commented Feb 8, 2024

CRITICAL BUG: images.generate does not work on Azure OpenAI #692

CRITICAL BUG: images.generate does not work on Azure OpenAI #692

Comments

nicolaschaillan commented Nov 6, 2023

kristapratico commented Nov 7, 2023

nicolaschaillan commented Nov 7, 2023

kristapratico commented Nov 7, 2023 • edited Loading

nicolaschaillan commented Nov 7, 2023 via email

ashjeanbird commented Nov 10, 2023

StephenHodgson commented Nov 10, 2023

kristapratico commented Nov 10, 2023

StephenHodgson commented Nov 10, 2023

ashjeanbird commented Nov 10, 2023

kristapratico commented Nov 15, 2023

nicolaschaillan commented Nov 15, 2023

kristapratico commented Nov 15, 2023

nicolaschaillan commented Nov 15, 2023

nicolaschaillan commented Nov 15, 2023

kristapratico commented Nov 16, 2023

nicolaschaillan commented Nov 16, 2023

StephenHodgson commented Nov 16, 2023

brunobraga commented Dec 1, 2023 • edited Loading

cohml commented Jan 5, 2024 • edited Loading

kristapratico commented Jan 5, 2024

cohml commented Jan 5, 2024

kristapratico commented Jan 5, 2024

StephenHodgson commented Jan 5, 2024

ronaldchoi commented Feb 8, 2024 • edited Loading

kristapratico commented Feb 8, 2024

kristapratico commented Nov 7, 2023 •

edited

Loading

brunobraga commented Dec 1, 2023 •

edited

Loading

cohml commented Jan 5, 2024 •

edited

Loading

ronaldchoi commented Feb 8, 2024 •

edited

Loading