Client: Support endpoints that return layout components #4871

freddyaboulton · 2023-07-11T03:03:42Z

Description

Closes: #4836

If an endpoint returns a layout component, it will not be returned by the api but all other IOComponents will (excluding State). These components are excluded from the view_api page as well. But the endpoint is still considered valid for the client.

🎯 PRs Should Target Issues

Before your create a PR, please check to see if there is an existing issue for this change. If not, please create an issue before you create this PR, unless the fix is very small.

Not adhering to this guideline will result in the PR being closed.

Tests & Changelog

PRs will only be merged if tests pass on CI. To run the tests locally, please set up your Gradio environment locally and run the tests: bash scripts/run_all_tests.sh
You may need to run the linters: bash scripts/format_backend.sh and bash scripts/format_frontend.sh
Unless the pull request is labeled with the "no-changelog-update" label by a maintainer of the repo, all pull requests must update the changelog located in CHANGELOG.md:

Please add a brief summary of the change to the Upcoming Release section of the CHANGELOG.md file and include
a link to the PR (formatted in markdown) and a link to your github profile (if you like). For example, "* Added a cool new feature by [@myusername](link-to-your-github-profile) in [PR 11111](https://github.com/gradio-app/gradio/pull/11111)".

vercel · 2023-07-11T03:03:46Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
gradio	✅ Ready (Inspect)	Visit Preview	Jul 17, 2023 5:13pm

gradio-pr-bot · 2023-07-11T03:06:19Z

All the demos for this PR have been deployed at https://huggingface.co/spaces/gradio-pr-deploys/pr-4871-all-demos

You can install the changes in this PR by running:

pip install https://gradio-builds.s3.amazonaws.com/f2fd37ee59cd91467ad43942fe2a2730cbdfe0d9/gradio-3.36.1-py3-none-any.whl

gradio-pr-bot · 2023-07-11T03:06:39Z

🎉 Chromatic build completed!

There are 24 visual changes to review.
There are 0 failed tests to fix.

freddyaboulton · 2023-07-11T22:06:29Z

UI and gradio unit tests should pass when new version of the client is released so marking as ready for review now!

abidlabs · 2023-07-12T04:47:00Z

Instead of making "row", etc. all serializable components, wouldn't a simpler approach be to modify skip_components here:

gradio/gradio/blocks.py

Line 505 in 5dc445b

skip_components = ["state"]

to include "row", etc. Then, we could just remove this logic altogether:

gradio/gradio/blocks.py

Line 520 in 5dc445b

skip_endpoint = True # if component not serializable, skip endpoint

I think that's more appropriate since the layouts are not actually serializable and we shouldn't treat them as such.

akx · 2023-07-12T08:05:48Z

client/python/gradio_client/client.py

@@ -427,7 +427,7 @@ def view_api(

        # Versions of Gradio older than 3.29.0 returned format of the API info
        # from the /info endpoint
-        if version.parse(self.config.get("version", "2.0")) > version.Version("3.29.0"):
+        if version.parse(self.config.get("version", "2.0")) > version.Version("3.36.1"):


Is this > on purpose (c.f. >=)?

The comment above is also now outdated.

Yes. The next release of gradio will include the updated logic for fetching the api info in this PR and so the app's /info route can be used.

client/python/gradio_client/utils.py

freddyaboulton · 2023-07-12T17:37:49Z

@abidlabs The skip_components in blocks.py are already updated to include state and all the layout components. The reason I included them in the serializer map is to preserve the principle that each output component has a serializer. That way we treat the layout components the same way we do state. I think that it's conceptually sound (they are both returned by the server but we don't want to display to users) and it makes the footprint of this PR pretty small. Although the layout components are in the serializer map, they will not be displayed in the api info or returned to users and the serializing logic is a no-op.

abidlabs · 2023-07-14T19:53:51Z

The skip_components in blocks.py are already updated to include state and all the layout components

Nice, I missed this.

But if we are already skipping the layout components, why do we need to add serializers for them? Imo that doesn't make sense because the serializers should only exist for components that can hold values since they define how this value is serialized between different formats. gr.State can hold a value, but the layout components don't hold any values.

We should instead "skip" the serialization of these components. (Skipping could just mean the identity function). Maybe its just semantics so happy to sync if that's easier

freddyaboulton · 2023-07-14T20:32:20Z

We should instead "skip" the serialization of these components. (Skipping could just mean the identity function)

In COMPONENT_MAPPING, I map the layout components to the SimpleSerializable. That's the same as the identify function no?

I guess I I could remove the layout components from the COMPONENT_MAPPING and instead modify the following code to not trigger the assert for the layout components and manually add a SimpleSerializer for the layout components. That's equivalent to what the current code does though and feels harder to maintain.

                        assert (
                            component_name in serializing.COMPONENT_MAPPING
                        ), f"Unknown component: {component_name}, you may need to update your gradio_client version."
                        serializer = serializing.COMPONENT_MAPPING[component_name]
                    serializers.append(serializer())  # type: ignore

abidlabs · 2023-07-14T21:05:00Z

I think we should replace:

                    else:
                        assert (
                            component_name in serializing.COMPONENT_MAPPING
                        ), f"Unknown component: {component_name}, you may need to update your gradio_client version."
                        deserializer = serializing.COMPONENT_MAPPING[component_name]

with

                    elif component_name in utils.SKIP_COMPONENTS:
                         deserializer = serializing.SimpleSerializable
                    else:
                        assert (
                            component_name in serializing.COMPONENT_MAPPING
                        ), f"Unknown component: {component_name}, you may need to update your gradio_client version."
                        deserializer = serializing.COMPONENT_MAPPING[component_name]

And then just remove these from serializing.COMPONENT_MAPPING:

    "row": SimpleSerializable,
    "column": SimpleSerializable,
    "tabs": SimpleSerializable,
    "tab": SimpleSerializable,
    "box": SimpleSerializable,
    "form": SimpleSerializable,
    "accordion": SimpleSerializable,
    "group": SimpleSerializable,

That reduces the footprint of the PR and should make it even easier to maintain since we only need to add components in one place: SKIP_COMPONENTS.

freddyaboulton · 2023-07-14T22:45:42Z

Followed your suggestion @abidlabs !

abidlabs · 2023-07-14T23:14:40Z

client/python/gradio_client/utils.py

@@ -31,10 +31,22 @@
 CONFIG_URL = "config"
 API_INFO_URL = "info"
 RAW_API_INFO_URL = "info?serialize=False"
-SPACE_FETCHER_URL = "https://gradio-space-api-fetcher-v2.hf.space/api"
+SPACE_FETCHER_URL = "https://freddyaboulton-space-api-fetcher.hf.space/api"


We just need to remember to update this

abidlabs · 2023-07-14T23:16:37Z

client/python/test/test_serializing.py

@@ -22,7 +22,9 @@ def test_duplicate(serializer_class):
 def test_check_component_fallback_serializers():
    for component_name, class_type in COMPONENT_MAPPING.items():
        # skip components that cannot be instantiated without parameters
-        if component_name in ["dataset", "interpretation"]:
+        if component_name in ["dataset", "interpretation"] + list(
+            SKIP_COMPONENTS - {"state"}


Should we just add ["dataset", "interpretation"] to SKIP_COMPONENTS and remove them from COMPONENT_MAPPING?

abidlabs · 2023-07-14T23:18:59Z

Small nit above. Code looks good + tested. LGTM thanks @freddyaboulton

freddyaboulton · 2023-07-17T17:21:51Z

Thanks for the review @abidlabs !!

vercel bot deployed to Preview July 11, 2023 03:09 View deployment

vercel bot deployed to Preview July 11, 2023 03:25 View deployment

vercel bot deployed to Preview July 11, 2023 16:30 View deployment

freddyaboulton marked this pull request as ready for review July 11, 2023 22:04

freddyaboulton requested review from abidlabs, aliabid94, aliabd, dawoodkhan82, pngwn and hannahblair July 11, 2023 22:05

akx reviewed Jul 12, 2023

View reviewed changes

client/python/gradio_client/utils.py Outdated Show resolved Hide resolved

freddyaboulton force-pushed the 4836-endpoints-with-non-serializable-outputs branch from abca220 to aec3eda Compare July 12, 2023 17:20

vercel bot deployed to Preview July 12, 2023 17:21 View deployment

freddyaboulton force-pushed the 4836-endpoints-with-non-serializable-outputs branch from aec3eda to f90ac58 Compare July 12, 2023 17:38

vercel bot deployed to Preview July 12, 2023 17:39 View deployment

vercel bot deployed to Preview July 14, 2023 19:39 View deployment

vercel bot deployed to Preview July 14, 2023 19:51 View deployment

vercel bot deployed to Preview July 14, 2023 21:58 View deployment

vercel bot deployed to Preview July 14, 2023 22:11 View deployment

vercel bot deployed to Preview July 14, 2023 22:17 View deployment

abidlabs reviewed Jul 14, 2023

View reviewed changes

abidlabs approved these changes Jul 14, 2023

View reviewed changes

freddyaboulton force-pushed the 4836-endpoints-with-non-serializable-outputs branch from 2f1d883 to 5022927 Compare July 17, 2023 17:04

vercel bot deployed to Preview July 17, 2023 17:04 View deployment

freddyaboulton added 8 commits July 17, 2023 13:06

Add code

e3474fe

CHANGELOG

aeb3ac5

Add code

180182b

Use set

ac55aa5

Modify _setup_serializers instead

5705f3f

Push up code

4510af0

Remove from serializing

7241498

Add gradio_client changes

62469d6

freddyaboulton force-pushed the 4836-endpoints-with-non-serializable-outputs branch from 5022927 to 62469d6 Compare July 17, 2023 17:06

vercel bot deployed to Preview July 17, 2023 17:08 View deployment

Update requirements.txt

54c6f35

vercel bot deployed to Preview July 17, 2023 17:13 View deployment

freddyaboulton merged commit 7b18891 into main Jul 17, 2023

freddyaboulton deleted the 4836-endpoints-with-non-serializable-outputs branch July 17, 2023 17:21

kznrluk mentioned this pull request Jul 24, 2023

Fix the version of gradio_client and maintain compatibility with gradio==3.35.2 haotian-liu/LLaVA#287

Merged

pngwn mentioned this pull request Jul 26, 2023

chore: update versions #4999

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Client: Support endpoints that return layout components #4871

Client: Support endpoints that return layout components #4871

freddyaboulton commented Jul 11, 2023 •

edited

Loading

vercel bot commented Jul 11, 2023 •

edited

Loading

gradio-pr-bot commented Jul 11, 2023 •

edited

Loading

gradio-pr-bot commented Jul 11, 2023 •

edited

Loading

freddyaboulton commented Jul 11, 2023 •

edited

Loading

abidlabs commented Jul 12, 2023

akx Jul 12, 2023 •

edited

Loading

freddyaboulton Jul 12, 2023

freddyaboulton commented Jul 12, 2023

abidlabs commented Jul 14, 2023 •

edited

Loading

freddyaboulton commented Jul 14, 2023

abidlabs commented Jul 14, 2023

freddyaboulton commented Jul 14, 2023

abidlabs Jul 14, 2023

abidlabs Jul 14, 2023

abidlabs commented Jul 14, 2023 •

edited

Loading

freddyaboulton commented Jul 17, 2023

Client: Support endpoints that return layout components #4871

Client: Support endpoints that return layout components #4871

Conversation

freddyaboulton commented Jul 11, 2023 • edited Loading

Description

🎯 PRs Should Target Issues

Tests & Changelog

vercel bot commented Jul 11, 2023 • edited Loading

gradio-pr-bot commented Jul 11, 2023 • edited Loading

gradio-pr-bot commented Jul 11, 2023 • edited Loading

freddyaboulton commented Jul 11, 2023 • edited Loading

abidlabs commented Jul 12, 2023

akx Jul 12, 2023 • edited Loading

Choose a reason for hiding this comment

freddyaboulton Jul 12, 2023

Choose a reason for hiding this comment

freddyaboulton commented Jul 12, 2023

abidlabs commented Jul 14, 2023 • edited Loading

freddyaboulton commented Jul 14, 2023

abidlabs commented Jul 14, 2023

freddyaboulton commented Jul 14, 2023

abidlabs Jul 14, 2023

Choose a reason for hiding this comment

abidlabs Jul 14, 2023

Choose a reason for hiding this comment

abidlabs commented Jul 14, 2023 • edited Loading

freddyaboulton commented Jul 17, 2023

freddyaboulton commented Jul 11, 2023 •

edited

Loading

vercel bot commented Jul 11, 2023 •

edited

Loading

gradio-pr-bot commented Jul 11, 2023 •

edited

Loading

gradio-pr-bot commented Jul 11, 2023 •

edited

Loading

freddyaboulton commented Jul 11, 2023 •

edited

Loading

akx Jul 12, 2023 •

edited

Loading

abidlabs commented Jul 14, 2023 •

edited

Loading

abidlabs commented Jul 14, 2023 •

edited

Loading