Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CONFIG] Resolve python requirements for python version and dependencies with server. #4935

Closed
frascuchon opened this issue Mar 22, 2024 · 9 comments · Fixed by #5666
Closed
Assignees
Labels
area: python sdk Indicates that an issue or pull request is related to the Python SDK area: server Indicates that an issue or pull request is related to the server
Milestone

Comments

@frascuchon
Copy link
Member

No description provided.

@burtenshaw burtenshaw changed the title [CONFIG] Setup python requirement to >=3.10 [CONFIG] Resolve python requirements for python version and dependencies with server. Apr 17, 2024
@frascuchon
Copy link
Member Author

  • Define python version 3.10?

-> >=3.9, <3.13

  • Define core dependencies i.e. pydantic

Already defined in the SDK

  • Be compatible with server

The server should be adapted to use pydantyc v2 which is not an evident task. Not sure if we can tackle this for v2.

@nataliaElv nataliaElv transferred this issue from argilla-io/argilla-python Jun 4, 2024
@nataliaElv nataliaElv added this to the v2.0.0 milestone Jun 4, 2024
@burtenshaw burtenshaw added the area: python sdk Indicates that an issue or pull request is related to the Python SDK label Jun 5, 2024
@burtenshaw
Copy link
Contributor

@jfcalvo Do you think you could tackle this one?

@jfcalvo
Copy link
Member

jfcalvo commented Jun 11, 2024

@jfcalvo Do you think you could tackle this one?

I can take a look for sure.

@jfcalvo
Copy link
Member

jfcalvo commented Jun 11, 2024

As @frascuchon commented we can change Python versions but we will still have problems with pydantic version on argilla-server.

I have used pip check to show the current problems we have trying to install argilla-sdk and argilla-server in the same Python environment.

From argilla-sdk:

argilla-sdk$ pip check
argilla-server 2.0.0.dev0 requires pydantic<2.0,>=1.10.7, but you have pydantic 2.7.3 which is incompatible.

From argilla-server:

argilla-server$ pip check
argilla-sdk 2.0.0a0.dev0 has requirement pydantic<3.0.0,>=2.6.0, but you have pydantic 1.10.15.

@burtenshaw
Copy link
Contributor

Thanks @jfcalvo. How feasible would it be to upgrade the server to pydantic v2?

Pydantic v1 features are exposed via the v2 api: https://docs.pydantic.dev/latest/migration/#continue-using-pydantic-v1-features

@frascuchon explored this before we deprecated the argilla api v1 endpoints. Maybe it's easier now...

@jfcalvo
Copy link
Member

jfcalvo commented Jun 11, 2024

@burtenshaw It's something we can explore but not sure if it will be possible inside the schedule with have for releasing v2.0.

@frascuchon
Copy link
Member Author

Thanks @jfcalvo. Using pydantic v1 from v2 wouldn't be a problem. The problem is using those with FastAPI which expects v2 models.

We need to migrate all the schemas at once. (it would be nice if we could migrate endpoints iteratively, but it's not possible). Several changes imply a deep endpoint revision:

  • The pattern regex is not supported anymore. We probably need to have custom validators for regex
  • The default values None for Optionals are not supported anymore. We need to explicitly set =None for those models where applies.

These changes and probably some more would be a potential nest of bugs if we don't take the time to validate everything.

@burtenshaw
Copy link
Contributor

Thanks @frascuchon. It's useful to have that transition written out.

What would you suggest for this issue then?

@burtenshaw
Copy link
Contributor

@jfcalvo @frascuchon As discussed, it's not feasible to solve this within the current time frame, and there's minimal value to the end user because they can run server and client in docker of spaces. Therefore, we should solve this outside of implementation by adding detailed docs with warnings that make it clear that the server and sdk cannot be run in the same python env due to a pydantic conflict.

We could support this further by checking the metrics on python server users, remove server dependence in argilla-sdk.

@nataliaElv We can focus on this v2.2.

@nataliaElv nataliaElv modified the milestones: v2.0.0, v2.2.0 Jun 12, 2024
@frascuchon frascuchon modified the milestones: v2.2.0, v2.3.0 Sep 17, 2024
@frascuchon frascuchon added the area: server Indicates that an issue or pull request is related to the server label Sep 24, 2024
@frascuchon frascuchon modified the milestones: v2.3.0, v2.4.0 Sep 27, 2024
@nataliaElv nataliaElv modified the milestones: v2.4.0, v2.5.0 Oct 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: python sdk Indicates that an issue or pull request is related to the Python SDK area: server Indicates that an issue or pull request is related to the server
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants