AI Featured Image: support backend prompts

[lhkowalski]

The frontend code change to implement backend prompts on the image generation. Test in pair with D150640-code.

Proposed changes:

• Change the image generation hook to export the generic image generation function
• Remove specific mentions to models on the AI Featured Image tool, so the model selection happens on the backend
• With this change, the backend will select the model for the generation; the D150640-code change will use Dall-e-3 for Jetpack sites and Stable Diffusion for Simple sites

Notes:

• the model detection for logging/tracking purposes is not perfect in this PR (it's using the cost of the request as a proxy, assuming each model has different costs). A better way would be to expose the "active model" or "default model" on the "ai-assistant-feature" endpoint, but this is a bigger change considering the timelime we have before the next release (need to add some code on the backend, as well as change the Jetpack here)
• I kept the original prompts (and methods) for the Dall-e and Stable Diffusion generation on the hook so we can keep referring to it; there is always time for some clean up in the future

Other information:

• Have you written new tests for your changes, if applicable?
• Have you checked the E2E test CI results, and verified that your changes do not break them?
• Have you tested your changes on WordPress.com, if applicable (if so, you'll see a generated comment below with a script to run)?

Jetpack product discussion

Does this pull request change what data or activity we track or use?

Testing instructions:

Testing on a Jetpack site:

• Apply D150640-code to your sandbox and sandbox the public-api
• Run this branch on your Jetpack testing site
• Open the browser console and enable debug with localStorage.setItem( 'debug', '*' )
• Go to the block editor and add some content
• Trigger the featured image generation
• On the console, confirm the event tracking shows the model as dall-e-3:

• Confirm the payload of the request does not mention the model of the generation, and contains the messages array:

{
    "feature": "featured-post-image",
    "size": "1792x1024",
    "responseFormat": "b64_json",
    "style": "photographic",
    "messages": [
        {
            "role": "jetpack-ai",
            "context": {
                "type": "featured-image-generation",
                "request": null,
                "content": "the content"
            }
        }
    ]
}

• Confirm the image was generated by Dall-e-3
  • the image will "looks like Dall-e"
  • if you inspect the image, you will see the dimensions are 1792×1024

Testing on a Simple site:

• Apply D150640-code to your sandbox
• After it, apply this branch on your sandbox using the jetpack-downloader command provided below
  • the tool will ask you to checkout trunk before applying it, type n to stay on the current branch
  • the tool will ask for another confirmation, to stay on the current branch, type y to stay on it
• Sandbox the public-api and also your testing simple site
• Open the browser console and enable debug with localStorage.setItem( 'debug', '*' )
• Go to the block editor and add some content
• Trigger the featured image generation
• On the console, confirm the event tracking shows the model as stable-diffusion

• Confirm the payload of the request does not mention the model of the generation, and contains the messages array:

{
    "feature": "featured-post-image",
    "size": "1792x1024",
    "responseFormat": "b64_json",
    "style": "photographic",
    "messages": [
        {
            "role": "jetpack-ai",
            "context": {
                "type": "featured-image-generation",
                "request": null,
                "content": "the content"
            }
        }
    ]
}

• Confirm the image was generated by Stable Diffusion
  • the image will "looks like Stable Diffusion"
  • if you inspect the image, you will see the dimensions are 1024x768

[github-actions]

Thank you for your PR!

When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:

• ✅ Include a description of your PR changes.
  
• ✅ Add a "[Status]" label (In Progress, Needs Team Review, ...).
  
• ✅ Add testing instructions.
  
• ✅ Specify whether this PR includes any changes to data or privacy.
  
• ✅ Add changelog entries to affected projects
  

This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation 🤖

---

The e2e test report can be found here. Please note that it can take a few minutes after the e2e tests checks are complete for the report to be available.

---

Once your PR is ready for review, check one last time that all required checks appearing at the bottom of this PR are passing or skipped.
Then, add the "[Status] Needs Team Review" label and ask someone from your team review the code. Once reviewed, it can then be merged.
If you need an extra review from someone familiar with the codebase, you can update the labels from "[Status] Needs Team Review" to "[Status] Needs Review", and in that case Jetpack Approvers will do a final review of your PR.

---

Jetpack plugin:

The Jetpack plugin has different release cadences depending on the platform:

• WordPress.com Simple releases happen daily.
• WoA releases happen weekly.
• Releases to self-hosted sites happen monthly. The next release is scheduled for July 2, 2024 (scheduled code freeze on June 24, 2024).

If you have any questions about the release process, please ask in the #jetpack-releases channel on Slack.

[github-actions]

Are you an Automattician? Please test your changes on all WordPress.com environments to help mitigate accidental explosions.

• To test on WoA, go to the Plugins menu on a WordPress.com Simple site. Click on the "Upload" button and follow the upgrade flow to be able to upload, install, and activate the Jetpack Beta plugin. Once the plugin is active, go to Jetpack > Jetpack Beta, select your plugin, and enable the update/ai-featured-image-support-backend-prompts branch.

• To test on Simple, run the following command on your sandbox:

  bin/jetpack-downloader test jetpack update/ai-featured-image-support-backend-prompts
  

Interested in more tips and information?

• In your local development environment, use the jetpack rsync command to sync your changes to a WoA dev blog.
• Read more about our development workflow here: PCYsg-eg0-p2
• Figure out when your changes will be shipped to customers here: PCYsg-eg5-p2

[vianasw]

Codewise it looks good. I tested it with a simple site and a JN site and I haven't seen any regression compared to having the prompts on the frontend.

[vianasw]

🚢