Field validators broken for nested models since v2.3.2 #331
Comments
|
Thanks @LachlanMarnham for reporting this. Actually, the example code shouldn't have worked before because By fixing the bug that we had before, you don't need a from pydantic import BaseModel
import os
from pydantic_settings import BaseSettings, SettingsConfigDict
class Child(BaseModel):
ATTRIBUTE: list[str]
class Parent(BaseSettings):
model_config = SettingsConfigDict(
env_prefix="PREFIX_",
env_nested_delimiter="__",
case_sensitive=False,
env_file_encoding="utf-8",
)
child: Child
os.environ["PREFIX_CHILD__ATTRIBUTE"] = '["a", "b", "c"]'
settings = Parent()
assert settings.child.ATTRIBUTE == ["a", "b", "c"] |
|
Ok thanks @hramezani. I think this is maybe a problem of documentation, with a conflict between
Which is fair enough. However,
It does feel a bit as if |
|
Good point @LachlanMarnham. The json parsing behavior was in |
Do you mean a flag in the Maybe the way I'm using Thanks |
|
Yes, I think the current behaviour is correct. what do you think? which part is wrong? |
I am "some people" here 😄 I would want to do custom parsing for only certain fields. How about some way to specify a list of fields for which to disable JSON parsing? Maybe a |
|
@fajpunk I updated my previous comment. the idea is to implement something like |
Was the behavior present in V1? I just updated a pre validator from v1 that always worked fine to a before validator and ran into this issue |
|
@Numerlor I am not sure. Could you give me the validator that works in V1 and doesn't work in V2. I can probably help you if there is a way |
The behaviors seem to differ between nested models and attributes directly on repro code v2Same with import json
import os
import typing
import pydantic
import pydantic_settings
from pydantic_settings import SettingsConfigDict
def parse_json_or_space_delimited_list(string: str) -> list[str]:
try:
return typing.cast(list[str], json.loads(string))
except json.JSONDecodeError:
return string.split(" ")
class Model(pydantic.BaseModel):
test: list[str] = []
test_validator = pydantic.field_validator("test", mode="before")(parse_json_or_space_delimited_list)
class Settings(pydantic_settings.BaseSettings):
test: Model
model_config = SettingsConfigDict(env_nested_delimiter="__")
os.environ["test__test"] = "a b c"
Settings()working v1import json
import os
import typing
import pydantic.v1
def parse_json_or_space_delimited_list(string: str) -> list[str]:
try:
return typing.cast(list[str], json.loads(string))
except json.JSONDecodeError:
return string.split(" ")
class Model(pydantic.v1.BaseModel):
test: list[str] = []
test_validator = pydantic.v1.validator("test", pre=True)(parse_json_or_space_delimited_list)
class Settings(pydantic.v1.BaseSettings):
test: Model
class Config:
env_nested_delimiter="__"
os.environ["test__test"] = "a b c"
Settings() |
|
@Numerlor, in V2, import os
from typing import Any, Tuple, Type
from pydantic import BaseModel
from pydantic.fields import FieldInfo
from pydantic_settings import BaseSettings, SettingsConfigDict, EnvSettingsSource, PydanticBaseSettingsSource
class MyEnvSettingsSource(EnvSettingsSource):
def decode_complex_value(self, field_name: str, field: FieldInfo, value: Any) -> Any:
return value.split(" ")
class NestedSettings(BaseModel):
test: list[str]
class Settings(BaseSettings):
test: NestedSettings
model_config = SettingsConfigDict(env_nested_delimiter="__")
@classmethod
def settings_customise_sources(
cls,
settings_cls: Type[BaseSettings],
init_settings: PydanticBaseSettingsSource,
env_settings: PydanticBaseSettingsSource,
dotenv_settings: PydanticBaseSettingsSource,
file_secret_settings: PydanticBaseSettingsSource,
) -> Tuple[PydanticBaseSettingsSource, ...]:
return init_settings, MyEnvSettingsSource(settings_cls), dotenv_settings, file_secret_settings
os.environ["test__test"] = "a b c"
print(Settings()) |
|
For what it's worth, I've also just run into this issue, trying to do something like the following: import os
from typing import Annotated
from pydantic import BeforeValidator
from pydantic_settings import BaseSettings
CommaSet = Annotated[set[str], BeforeValidator(lambda s: set(x for x in s.split(",") if x))]
class Example(BaseSettings):
EXAMPLE: CommaSet
os.environ["EXAMPLE"] = "asdf,fdsa"
print(Example())It really would be nice if there was a way to tell pydantic-settings what the expected 'input' type for the type is and base the 'is complex value' logic on that. There is the 'WithJsonSchema' annotator already in Pydantic for example, maybe somethinge like that. Or just something like a 'DontDeJSON' annotator that pydantic-settings respects. |
|
I created #492 to add |
|
Addressed in 3667aed |
pydantic version: 2.8.0
OS: confirmed broken on Windows and Linux, haven't tested others
pydantic-settings version: 2.3.2
Hello, I noticed that there is a breaking change in
pydantic-settings==2.3.2, which I think was introduced by PR: #309. Here is a minimal working example:This test passes for
pydantic-settingsup to and including 2.3.1, but has been broken since 2.3.2. The test doesn't simply fail: the instantiation ofParentraises with the following traceback:So it's trying to JSON-decode the string
"a|b|c"and failing. But myfield_validatoris running in"before"modeso this shouldn't happen. Indeed, if I simply comment-out the
field_validatorI see exactly the same error. So I believe that decorator is either being ignored altogether, or"before"mode has been broken.The text was updated successfully, but these errors were encountered: