OpenAPI Test Client (with an emphasis on "test" client) can generate/update API clients in Python from any OpenAPI 3.x specifications.
The goal of the project is to provide easy-to-use/maintain auto-generated API clients for QA engineers who need to thoroughly perform automated API testing against API-first applications in a fast-paced development environment.
During each software release cycle, QA will need to perform various testing to ensure that new APIs to be released work as expected. The testing tool for QA requires to have a lot of flexibility and intelligence to achieve this efficiently, and to fully automate everything which of course includes negative cases and some unusual scenarios. This API test client will make the entire testing process very easy and efficient by automatically handling a lot of boilerplate under the hood, while also providing you with full control over how and what you want to test, as well as how you want to extend or customize it for your own use case.
One important thing to clarify is that this is NOT an SDK client but a "test" client although both may share some common capabilities. An SDK client utilizes stable APIs under the hood for development purposes, while an API test client is designed for testing raw APIs that are currently in development and may still contain bugs. What this means is the test client will NOT automatically validate requests and responses for you. It is the QA's main responsibility to work on, and the test client will provide everything needed for testing APIs.
Tip
If you would like to expand the client's use case beyond testing, you can optionally enable "validation mode" to make
it partially behave like an SDK. When validation mode is enabled, your request payload will be validated in
Pydantic strict mode on the client side.
Please note that validation against response is not currently supported.
Important
Before moving forward, please make sure to create a new venv
(requires Python 3.11+) and start with a fresh
environment. We will use pip install
to simplify the setup process, but this project is not intended to be used as
a publicly available Python package. Since all dependencies are intentionally pinned strictly to exact versions,
dependency conflicts may occur if the installation is done within your existing project.
pip install git+https://github.com/yugokato/openapi-test-client.git
Or, if you want to try the demo client with the demo backend app:
pip install "openapi-test-client[app] @ git+https://github.com/yugokato/openapi-test-client.git"
Alternatively, you can clone the repo and run pip install -e .
(or pip install -e .[app]
) if you prefer to reuse
this project as your base project.
Once the setup is complete, the openapi-client
CLI command should be available for generating and updating an API
client from an OpenAPI spec file. It will take either generate
or update
as the first argument. Available command
options will change based on which action you choose. (see help of each action with -h
for more details)
$ openapi-client -h
usage: openapi-client [-h] {generate,update} ...
positional arguments:
{generate,update} Action to take
generate Generate a new API client from an OpenAPI spec URL
update Update an existing API client
options:
-h, --help show this help message and exit
You can generate a new API client from OpenAPI specs by running the generate
command:
openapi-client generate -u <url> -a <app_name> [--dir <directory>]
url
: The URL of the OpenAPI spec file (JSON/YAML, must be version 3.x) from which you want to generate the API client
app_name
: The name of the target application. The value will be used for generating the client name
directory
: A directory path to save the generated client modules. The specified directory will become the top-level
module for accessing all your clients. (This is optional if your project was cloned from the original repository)
To generate another client for a different app, simply run the same command with the same --dir
value, but with new
-u
and -a
values.
Note
The following manual step(s) will be needed after generating a client
- For each auto-generated API function, rename the function name to whatever value you like. All functions will be generated as "_unnamed_endpoint_{idx}" by default
- (If needed) Adjust the value of the API base URL in the
cfg/urls.json
, and thedoc
value in the generated API client's__init__()
code. These values will be automatically added during the client generation based on theurl
value you specify, but your application may expect to split the URL differently
You can update the existing client code by running the update
command to reflect the latest OpenAPI specs:
openapi-client update -c <client_name>
client_name
: The name of the client assigned at the client generation step
This update functionality primarily focuses on reflecting the latest OpenAPI specs. It will only touch the following areas:
- Existing API class tag
- Existing API class functions (signatures and docstrings)
- Existing API parameter models
- Currently missing API functions and parameter models (a new function will be added as "_unnamed_endpoint_{idx}")
- Currently missing API classes (this requires
-A
option)
Anything else, (eg. API class/function names, function body, decorators applied on API functions, etc.) will remain intact so that updated client code won't break your existing test code.
Tip
- You might want to run the command with
-d
/--dry-run
option first to see changes before actually updating code - You can limit the scope of the update (eg. specific tag, specific endpoint, etc.) by using various command options.
See more details with
-h
Your client should be ready to use by now. For demonstration purposes, the rest of the sections will be explained using
the DemoAppAPIClient that comes with this repository as an example client. This demo client was actually
generated by the following openapi-client generate
command against the Quart-based demo app
demo application:
openapi-client generate -u http://127.0.0.1:5000/openapi.json -a demo_app
It was also manually extended with a custom post-request hook for automatically managing auth tokens before login and after logout.
Note
If you would like to try this demo_app
API client by yourself, follow the following steps to start the demo application before moving forward:
- Clone the repo
- Set up with
app
server-side extra dependency (see the "Setup" section) - Start the application with
QUART_APP=demo_app.main:app quart run
command
$ QUART_APP=demo_app.main:app quart run
* Serving Quart app 'demo_app'
* Debug mode: False
* Please use an ASGI server (e.g. Hypercorn) directly in production
* Running on http://127.0.0.1:5000 (CTRL + C to quit)
[2024-01-01 00:00:00 -0000] [39691] [INFO] Running on http://127.0.0.1:5000 (CTRL + C to quit)
Once you have generated an API client, the client class will be importable as
from <your_module>.clients.<client_name> import <ClientName>APIClient
.
>>> from openapi_test_client.clients.demo_app import DemoAppAPIClient
>>> client = DemoAppAPIClient()
Make sure to replace the "openapi_test_client" part with your own module name when importing your own clients.
Alternatively, you can instantiate your client directly from the parent OpenAPIClient
class.
>>> from openapi_test_client.clients import OpenAPIClient
>>> client = OpenAPIClient.get_client("<client_name>")
To make an API request with your API client, call an API function as client.<API_TAG>.<api_function_name>()
.
The function will take all parameters documented in the OpenAPI specs as keyword arguments.
eg. To call the login API defined under the Auth tag:
>>> r = client.AUTH.login(username='foo', password='bar')
2024-01-01T00:00:00.863-0800 - request: POST http://127.0.0.1:5000/v1/auth/login
2024-01-01T00:00:00.877-0800 - response: 201 (Created)
- request_id: a2b20acf-22d5-4131-ac0d-6796bf19d2af
- request: POST http://127.0.0.1:5000/v1/auth/login
- payload: {"username": "foo", "password": "***"}
- status_code: 201 (Created)
- response: {
"token": "IjFlNTAxNmI2LTVlZjctNGQxYi1iMGJhLTYxY2M3ZWIzY2VmYSI.ZadDlA.tKS_La5uDuteXH7OMT_WZVK1o_hAnWZVn_J5rSsJQILHu1juXYg0EYLkgpH1LChzeOhN_YUUXaO37rlt_UV1Ag"
}
- response_time: 0.01344s
Note
- A new UUID will be generated at each request and will be set to the
X-Request-ID
header DemoAppAPIClient
will automatically set thetoken
value to the API session in the client's post-request logic mentioned earlier
The function call will return a RestResponse
object where the decoded response is accessible as response
. If you
need to access the raw request and response objects from the requests
library, you can do so through the request
and _response
attributes.
>>> r.status_code
201
>>> r.response
{'token': 'IjFlNTAxNmI2LTVlZjctNGQxYi1iMGJhLTYxY2M3ZWIzY2VmYSI.ZadDlA.tKS_La5uDuteXH7OMT_WZVK1o_hAnWZVn_J5rSsJQILHu1juXYg0EYLkgpH1LChzeOhN_YUUXaO37rlt_UV1Ag'}
>>> pprint(r)
RestResponse(_response=<Response [201],
request_id='a2b20acf-22d5-4131-ac0d-6796bf19d2af',
status_code=201,
response={'token': 'IjFlNTAxNmI2LTVlZjctNGQxYi1iMGJhLTYxY2M3ZWIzY2VmYSI.ZadDlA.tKS_La5uDuteXH7OMT_WZVK1o_hAnWZVn_J5rSsJQILHu1juXYg0EYLkgpH1LChzeOhN_YUUXaO37rlt_UV1Ag'},
response_time=0.01344,
request=<PreparedRequest [POST]>,
ok=True,
is_stream=False)
Note that we extend the requests
library's PreparedRequest
and Session
classes to add a few small
capabilities.
As an example, you can get the request start/end time (UTC) through request
as a datatime
object.
>>> r.request.start_time
datetime.datetime(2024, 1, 1, 0, 0, 0, 86309, tzinfo=datetime.timezone.utc)
>>> r.request.end_time
datetime.datetime(2024, 1, 1, 0, 0, 0, 87714, tzinfo=datetime.timezone.utc)
The test client provides a few ways to customize how an API call via an API function will be processed.
By default, each auto-generated API function will look like a stub function with a placeholder (...
). For example:
@endpoint.is_public
@endpoint.post("/v1/auth/login")
def login(self, *, username: str = Unset, password: str = Unset, **kwargs) -> RestResponse:
"""Login"""
...
Even without having actual function logic, API requests made through these functions will just work - The function call will automatically make an HTTP request to the associated endpoint with the provided parameters, parse the response, and return it as a RestResponse
object. This behavior is managed by the @endpoint.<method>(path>)
decorator.
In most cases, you won’t need to modify the generated function body at all.
However, if you need to implement custom logic for making an API request for specific API functions, you can replace the placeholder with your own code. Just ensure that the function returns a RestResponse
object, which is what the underlying Rest API client we use will return.
Note
The client will raise a RuntimeError
if anything other than a RestResponse
object or None
is returned.
Tip
If you only need to add extra behavior before or after the request, consider using decorators or request hooks instead
To extend the default API call logic, you can apply decorators on an API function. Note that each decorator MUST be decorated with @endpoint.decorator
for it to work properly.
For example:
- Define a decorator
from functools import wraps
from openapi_test_client.libraries.api import endpoint
@endpoint.decorator
def my_decorator(f):
@wraps(f)
def wrapper(*args, **kwargs):
# Do something before request
response = f(**args, **kwargs)
# Do something after request
return response
return wrapper
- Apply it
@my_decorator
@endpoint.is_public
@endpoint.post("/v1/auth/login")
def login(self, *, username: str = Unset, password: str = Unset, **kwargs) -> RestResponse:
"""Login"""
...
The test client supports three types of request hooks in API classes:
- Pre-request Hook: Called before an API request is made
- Post-request Hook: Called after an API request is completed
- Request wrapper: Wraps the original API request logic before any decorators are applied, allowing you to apply additional behavior before and after a request
Hooks work similarly to decorators but are more useful for applying general control that affects multiple or all endpoints at the central place.
For examples and usage, see demo_app client hooks
The following are some of the important building blocks of the API test client:
- API client
- API class and functions
- API endpoint function
- API endpoint model
- API parameter model
- Pydantic model (validation mode)
All API clients will inherit the base client OpenAPIClient
, which will handle parsing OpenAPI specs and resolving
$ref
. The generated API client will have a @cached_property
definition for each API tag (or set of tags when
managing APIs in multiple tags), where an API class instance will be returned.
# openapi_test_client/clients/demo_app/demo_app_client.py
from functools import cached_property
from openapi_test_client.clients.base import OpenAPIClient
from .api.auth import AuthAPI
from .api.users import UsersAPI
class DemoAppAPIClient(OpenAPIClient):
"""API client for demo_app"""
def __init__(self, env: str = "dev"):
super().__init__("demo_app", env=env, doc="openapi.json")
@cached_property
def AUTH(self):
return AuthAPI(self)
@cached_property
def USERS(self):
return UsersAPI(self)
All API endpoints for a tag (or set of tags) will be managed inside an "API class", where each function will map to a specific endpoint documented in the OpenAPI spec.
For example, if the OpenAPI spec defines Auth APIs like this:
"paths": {
"/v1/auth/login": {
"post": {
"parameters": [],
"responses": {},
"description": "",
"summary": "Login",
"tags": [
"Auth"
],
"security":[],
"requestBody": {
"content": {
"application/json": {
"schema": {
"properties": {
"username": {
"title": "Username",
"type": "string"
},
"password": {
"title": "Password",
"type": "string"
}
},
"required": [
"username",
"password"
],
"title": "LoginData",
"type": "object"
}
}
}
},
"operationId": "post_login"
}
},
"/v1/auth/logout": {
"get": {
"parameters": [],
"responses": {},
"description": "",
"summary": "Logout",
"tags": [
"Auth"
],
"security":[],
"operationId": "get_logout"
}
},
Then the API class and functions will be generated like this:
# openapi_test_client/clients/demo_app/api/auth.py
from common_libs.clients.rest_client import RestResponse
from openapi_test_client.clients.demo_app.api.base import DemoAppBaseAPI
from openapi_test_client.libraries.api.api_functions import endpoint
from openapi_test_client.libraries.api.types import Unset
class AuthAPI(DemoAppBaseAPI):
TAGs = ("Auth",)
@endpoint.is_public
@endpoint.post("/v1/auth/login")
def login(self, *, username: str = Unset, password: str = Unset, **kwargs) -> RestResponse:
"""Login"""
...
@endpoint.is_public
@endpoint.get("/v1/auth/logout")
def logout(self, **kwargs) -> RestResponse:
"""Logout"""
...
A function can take API path parameters and query/body parameters as arguments, where path parameters are always
defined as positional-only arguments, and other parameters are always defined as keyword-only arguments with a default
value of Unset
. Any parameters with this sentinel value will be excluded from the actual API call parameters.
Additionally, it will always have **kwargs
for supporting making a call with any undocumented
parameters and/or with raw requests
library options (eg. timeout
, headers
, etc) if you need to.
Note
Parameters defined as optional in the OpenAPI spec will be annotated with Optional[]
in the function signature, but
this does not mean the parameter is actually optional for the endpoint. It just means you can hit the endpoint
without specifying that parameter, whether it is actually required or not (for negative case testing)
Some attributes available from the API class:
>>> # Get tag data
>>> client.AUTH.TAGs
('Auth',)
>>> # Get available endpoints under this API class
>>> pprint(client.AUTH.endpoints)
[Endpoint(tags=('Auth',),
api_class=<class 'openapi_test_client.clients.demo_app.api.auth.AuthAPI'>,
method='post',
path='/v1/auth/login',
func_name='login',
model=<class 'types.AuthAPILoginEndpointModel'>,
url=None,
content_type=None,
is_public=True,
is_documented=True,
is_deprecated=False),
Endpoint(tags=('Auth',),
api_class=<class 'openapi_test_client.clients.demo_app.api.auth.AuthAPI'>,
method='get',
path='/v1/auth/logout',
func_name='logout',
model=<class 'types.AuthAPILogoutEndpointModel'>,
url=None,
content_type=None,
is_public=True,
is_documented=True,
is_deprecated=False)]
A list of defined API classes are available as API_CLASSES
.
>>> from itertools import chain
>>> from openapi_test_client.clients.demo_app.api import API_CLASSES
# list all andpoints
>>> all_endpoints = chain(*[x.endpoints for x in API_CLASSES])
>>> for endpoint in all_endpoints:
... print(endpint)
...
POST /v1/auth/login
GET /v1/auth/logout
POST /v1/users
GET /v1/users/{user_id}
GET /v1/users
POST /v1/users/images
DELETE /v1/users/{user_id}
Each API class function is decorated with a @endpoint.<method>(<path>)
endpoint decorator. This decorator converts the original function into an instance of the EndpointHandler
class at runtime. The EndpointHandler
object acts as a proxy to a per-endpoint EndpointFunc
object, which is also created at runtime and is responsible for handling the actual API calls via its base class's __call__()
method. Additionally, the EndpointFunc
object provides various capabilities and attributes related to the endpoint.
eg. The Login API is accessible via client.AUTH.login()
API function, which is actually an instance of
AuthAPILoginEndpointFunc
class returned by its associated EndpointHandler
obj.
>>> client.AUTH.login
<openapi_test_client.libraries.api.api_functions.endpoint.AuthAPILoginEndpointFunc object at 0x1074abf10>
(mapped to: <function AuthAPI.login at 0x10751c360>)
The endpoint function is also accessible directly from the API class:
>>> from openapi_test_client.clients.demo_app.api.auth import AuthAPI
>>> AuthAPI.login
<openapi_test_client.libraries.api.api_functions.endpoint.AuthAPILoginEndpointFunc object at 0x107650b50>
(mapped to: <function AuthAPI.login at 0x10751c360>)
Various endpoint data is available from the endpoint function via endpoint
property:
>>> print(client.AUTH.login.endpoint)
POST /v1/auth/login
>>> pprint(client.AUTH.login.endpoint)
Endpoint(tags=('Auth',),
api_class=<class 'openapi_test_client.clients.demo_app.api.auth.AuthAPI'>,
method='post',
path='/v1/auth/login',
func_name='login',
model=<class 'types.AuthAPILoginEndpointModel'>,
url='http://127.0.0.1:5000/v1/auth/login',
content_type=None,
is_public=True,
is_documented=True,
is_deprecated=False)
>>> client.AUTH.login.endpoint.method
'post'
>>> client.AUTH.login.endpoint.path
'/v1/auth/login'
>>> client.AUTH.login.endpoint.url
'http://127.0.0.1:5000/v1/auth/login'
Note that the same endpoint data is also available directly from the API class, except for url
will always be None
.
>>> from openapi_test_client.clients.demo_app.api.auth import AuthAPI
>>> client.AUTH.login.endpoint == AuthAPI.login.endpoint
True
>>> print(AuthAPI.login.endpoint)
POST /v1/auth/login
>>> pprint(AuthAPI.login.endpoint)
Endpoint(tags=('Auth',),
api_class=<class 'openapi_test_client.clients.demo_app.api.auth.AuthAPI'>,
method='post',
path='/v1/auth/login',
func_name='login',
model=<class 'types.AuthAPILoginEndpointModel'>,
url=None,
content_type=None,
is_public=True,
is_documented=True,
is_deprecated=False)
An example of the additional capability the EndpointFunc
obj provides - Automatic retry:
# Call the endpiont with the automatic retry (you can specify a retry condition if needed)
>>> r = client.AUTH.login.with_retry(username='foo', password='bar')
2024-01-01T00:00:00.153-0000 - request: POST http://127.0.0.1:5000/v1/auth/login
2024-01-01T00:00:00.158-0000 - response: 429 (Too Many Requests)
- request_id: 1b028ff7-0880-430c-b5a3-12aa057892cf
- request: POST http://127.0.0.1:5000/v1/auth/login
- payload: {"username": "foo", "password": "***"}
- status_code: 429 (Too Many Requests)
{
"error": "Too many requests"
}
- response_time: 0.004804s
2024-01-01T00:00:00.159-0000 - Retry condition matched. Retrying in 5 seconds...
2024-01-01T00:00:05.166-0000 - request: POST http://127.0.0.1:5000/v1/auth/login
2024-01-01T00:00:05.171-0000 - response: 201 (Created)
- request_id: 1a34195e-5cec-4d4e-abe0-61acbfcdae1a
- request: POST http://127.0.0.1:5000/v1/auth/login
- payload: {"username": "foo", "password": "***"}
- status_code: 201 (Created)
- response: {
"token": "ImJjMjA5YjkxLTEzYmItNDE3Yi1iN2ExLTdhNmFlNWFjYjFiNSI.ZuSs3Q.KD-zA6KxCUEC14YUAuCcuQNGNoulp2POuKq0yMIfkGIS0E--V473kssohnvGoIHo97FwwSeOV94NnwaZdGtSxA"
}
- response_time: 0.005143s
>>>
>>> r.request.retried.request_id
'1b028ff7-0880-430c-b5a3-12aa057892cf'
Each endpoint is represented as an EndpointModel
dataclass model, which holds various context around each
parameter (eg. type annotation).
>>> model = client.AUTH.login.endpoint.model
>>> print(model)
<class 'types.AuthAPILoginEndpointModel'>
>>> pprint(model.__dataclass_fields__, sort_dicts=False)
{'username': Field(name='username',type=<class 'str'>,default=<object object at 0x107b410b0>,default_factory=<dataclasses._MISSING_TYPE object at 0x107ea61d0>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=True,_field_type=_FIELD),
'password': Field(name='password',type=<class 'str'>,default=<object object at 0x107b410b0>,default_factory=<dataclasses._MISSING_TYPE object at 0x107ea61d0>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=True,_field_type=_FIELD)}
If a parameter type is documented as object
(or array
of objects) with properties
in the OpenAPI specs, it will
be represented as a ParamModel
dataclass model. These models will be automatically generated as a model file (<your_module_dir>/clients/<client_name>/models/<api_tag>.py
) for each API class.
eg.
- API function definition (note that the
metadata
parameter is of typeMetadata
)
# openapi_test_client/clients/demo_app/api/users.py
from typing import Annotated, Literal
from common_libs.clients.rest_client import RestResponse
from openapi_test_client.clients.demo_app.api.base import DemoAppBaseAPI
from openapi_test_client.libraries.api.api_functions import endpoint
from openapi_test_client.libraries.api.types import Constraint, Format, Optional, Unset
from ..models.users import Metadata
class UsersAPI(DemoAppBaseAPI):
TAGs = ("Users",)
@endpoint.post("/v1/users")
def create_user(
self,
*,
first_name: Annotated[str, Constraint(min_len=1, max_len=255)] = Unset,
last_name: Annotated[str, Constraint(min_len=1, max_len=255)] = Unset,
email: Annotated[str, Format("email")] = Unset,
role: Literal["admin", "viewer", "support"] = Unset,
metadata: Optional[Metadata] = Unset,
**kwargs,
) -> RestResponse:
"""Create a new user"""
...
- Parameter model definition
# openapi_test_client/clients/demo_app/models/users.py
from dataclasses import dataclass
from typing import Annotated, Literal
from openapi_test_client.libraries.api.types import Constraint, Format, Optional, ParamModel, Unset
@dataclass
class Preferences(ParamModel):
theme: Optional[Literal["light", "dark", "system"]] = Unset
language: Optional[str] = Unset
font_size: Optional[Annotated[int, Constraint(min=8, max=40, multiple_of=2)]] = Unset
@dataclass
class SocialLinks(ParamModel):
facebook: Optional[Annotated[str, Format("uri"), Constraint(min_len=1)]] = Unset
instagram: Optional[Annotated[str, Format("uri"), Constraint(min_len=1)]] = Unset
linkedin: Optional[Annotated[str, Format("uri"), Constraint(min_len=1)]] = Unset
github: Optional[Annotated[str, Format("uri"), Constraint(min_len=1)]] = Unset
@dataclass
class Metadata(ParamModel):
preferences: Optional[Preferences] = Unset
social_links: Optional[SocialLinks] = Unset
Note
Same like API function parameters:
- All model attributes will be defined with a default value of
Unset
. Any parameters with this sentinel value will be excluded from an API call payload - Parameters defined as optional in the OpenAPI spec will be annotated with
Optional[]
in the model definition
A parameter model is a customized dataclass
that is intended to function exactly the same as a dictionary under the hood.
Unlike the native dataclass, a parameter model instance will only hold fields that were explicitly given at instantiation.
This is very important when you want to differentiate test scenarios for "not including a parameter in the payload" v.s.
"explicitly giving a null
value for a parameter".
>>> from dataclasses import asdict, is_dataclass
>>> from openapi_test_client.clients.demo_app.models.users import *
>>> metadata = Metadata(preferences=Preferences(theme='dark'))
>>> print(metadata)
Metadata(preferences=Preferences(theme='dark'))
>>> print(metadata.preferences)
Preferences(theme='dark')
>>> print(metadata["preferences"])
Preferences(theme='dark')
>>> is_dataclass(metadata) and isinstance(metadata, dict)
True
>>> metadata.items()
dict_items([('preferences', Preferences(theme='dark'))])
>>> asdict(metadata)
{'preferences': {'theme': 'dark'}}
Parameter models function both as a dataclass and a dictionary, and a change made to one will be reflected in the other.
>>> # Update a model field value
>>> metadata.preferences.theme = "light"
>>> print(metadata)
Metadata(preferences=Preferences(theme='light'))
>>> asdict(metadata)
{'preferences': {'theme': 'light'}}
>>>
>>> # Update as a dictionary
>>> metadata.preferences["theme"] = "dark"
>>> print(metadata)
Metadata(preferences=Preferences(theme='dark'))
>>> asdict(metadata)
{'preferences': {'theme': 'dark'}}
You can also dynamically add or delete model fields similar to how you can add or remove a key from a dictionary, if you need to.
>>> # Add a new field to the model
>>> metadata.new_attr1 = 123
>>> print(metadata)
Metadata(preferences=Preferences(theme='dark'), new_attr1=123)
>>> asdict(metadata)
{'preferences': {'theme': 'dark'}, 'new_attr1': 123}
>>>
>>> # Add a new field to the dictionary
>>> metadata["new_attr2"] = 456
>>> print(metadata)
Metadata(preferences=Preferences(theme='dark'), new_attr1=123, new_attr2=456)
>>> asdict(metadata)
{'preferences': {'theme': 'dark'}, 'new_attr1': 123, 'new_attr2': 456}
>>>
>>> # Delete a field from the model
>>> delattr(metadata, "new_attr2")
>>> print(metadata)
Metadata(preferences=Preferences(theme='dark'), new_attr1=123)
>>> asdict(metadata)
{'preferences': {'theme': 'dark'}, 'new_attr1': 123}
>>>
>>> # Delete a field from the dictionary
>>> del metadata["new_attr1"]
>>> print(metadata)
Metadata(preferences=Preferences(theme='dark'))
>>> asdict(metadata)
{'preferences': {'theme': 'dark'}}
By default, our client models will not perform any validation for you, since they are based on dataclass
. This allows
the test client to send whatever invalid values for a parameter to verify the server-side validation logic.
To expand the use case of the client beyond testing, you can optionally enable "validation mode". When enabled, all of
our dataclass models (Endpoint/Param models) will be automatically converted into native Pydantic models that inherit
this custom base model class:
from typing import ClassVar
from pydantic import BaseModel, ConfigDict
class PydanticModel(BaseModel):
"""Base class for Pydantic endpoint/param models"""
model_config: ClassVar = ConfigDict(extra="forbid", validate_assignment=True, strict=True)
With this dynamic conversion, request parameters will be automatically validated in Pydantic strict mode before an API request call.
Here are some comparisons between regular models and pydantic models:
- Regular dataclass model (Validation will be done on the server-side)
>>> # Model definition
>>> model = client.USERS.create_user.endpoint.model
>>> print(model)
<class 'types.UsersAPICreateUserEndpointModel'>
>>> pprint(model.__dataclass_fields__, sort_dicts=False)
{'first_name': Field(name='first_name',type=typing.Annotated[str, Constraint(min=None, max=None, multiple_of=None, min_len=1, max_len=255, nullable=None, pattern=None, exclusive_minimum=None, exclusive_maximum=None)],default=<object object at 0x107b410b0>,default_factory=<dataclasses._MISSING_TYPE object at 0x107ea61d0>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=True,_field_type=_FIELD),
'last_name': Field(name='last_name',type=typing.Annotated[str, Constraint(min=None, max=None, multiple_of=None, min_len=1, max_len=255, nullable=None, pattern=None, exclusive_minimum=None, exclusive_maximum=None)],default=<object object at 0x107b410b0>,default_factory=<dataclasses._MISSING_TYPE object at 0x107ea61d0>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=True,_field_type=_FIELD),
'email': Field(name='email',type=typing.Annotated[str, Format(value='email')],default=<object object at 0x107b410b0>,default_factory=<dataclasses._MISSING_TYPE object at 0x107ea61d0>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=True,_field_type=_FIELD),
'role': Field(name='role',type=typing.Literal['admin', 'viewer', 'support'],default=<object object at 0x107b410b0>,default_factory=<dataclasses._MISSING_TYPE object at 0x107ea61d0>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=True,_field_type=_FIELD),
'metadata': Field(name='metadata',type=typing.Optional[openapi_test_client.clients.demo_app.models.users.Metadata],default=<object object at 0x107b410b0>,default_factory=<dataclasses._MISSING_TYPE object at 0x107ea61d0>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=True,_field_type=_FIELD)}
>>>
>>> # Make an API request with the invalid parameter values
>>> r = client.USERS.create_user(first_name=123, email="foo", role="something", metadata=Metadata(social_links=SocialLinks(facebook="test")), extra=123)
2024-01-01T00:00:00.741-0800 - The request contains one or more parameters UsersAPI.create_user() does not expect:
- extra
2024-01-01T00:00:00.742-0800 - request: POST http://127.0.0.1:5000/v1/users
2024-01-01T00:00:00.752-0800 - response: 400 (Bad Request)
- request_id: 91b4ac05-1280-473b-ac41-0be30903c448
- request: POST http://127.0.0.1:5000/v1/users
- payload: {"first_name": 123, "email": "foo", "role": "something", "metadata": {"social_links": {"facebook": "test"}}, "extra": 123}
- status_code: 400 (Bad Request)
- response: {
"error": {
"code": 400,
"message": [
{
"type": "string_type",
"loc": [
"first_name"
],
"msg": "Input should be a valid string",
"input": 123,
"url": "https://errors.pydantic.dev/2.6/v/string_type"
},
{
"type": "missing",
"loc": [
"last_name"
],
"msg": "Field required",
"input": {
"first_name": 123,
"email": "foo",
"role": "something",
"metadata": {
"social_links": {
"facebook": "test"
}
},
"extra": 123
},
"url": "https://errors.pydantic.dev/2.6/v/missing"
},
{
"type": "value_error",
"loc": [
"email"
],
"msg": "value is not a valid email address: The email address is not valid. It must have exactly one @-sign.",
"input": "foo",
"ctx": {
"reason": "The email address is not valid. It must have exactly one @-sign."
},
"url": "https://errors.pydantic.dev/2.6/v/value_error"
},
{
"type": "enum",
"loc": [
"role"
],
"msg": "Input should be 'admin', 'viewer' or 'support'",
"input": "something",
"ctx": {
"expected": "'admin', 'viewer' or 'support'"
},
"url": "https://errors.pydantic.dev/2.6/v/enum"
},
{
"type": "url_parsing",
"loc": [
"metadata",
"social_links",
"facebook"
],
"msg": "Input should be a valid URL, relative URL without a base",
"input": "test",
"ctx": {
"error": "relative URL without a base"
},
"url": "https://errors.pydantic.dev/2.6/v/url_parsing"
}
],
"request_id": "91b4ac05-1280-473b-ac41-0be30903c448"
}
}
- response_time: 0.010107s
- Pydantic model (Validation will be done on the client-side)
>>> # Model definition
>>> pydantic_model = client.USERS.create_user.endpoint.model.to_pydantic()
>>> print(pydantic_model)
<class 'types.UsersAPICreateUserEndpointModel'>
>>> pprint(pydantic_model.model_fields, sort_dicts=False)
{'first_name': FieldInfo(annotation=str, required=True, metadata=[MinLen(min_length=1), MaxLen(max_length=255), Constraint(min=None, max=None, multiple_of=None, min_len=1, max_len=255, nullable=None, pattern=None, exclusive_minimum=None, exclusive_maximum=None)]),
'last_name': FieldInfo(annotation=str, required=True, metadata=[MinLen(min_length=1), MaxLen(max_length=255), Constraint(min=None, max=None, multiple_of=None, min_len=1, max_len=255, nullable=None, pattern=None, exclusive_minimum=None, exclusive_maximum=None)]),
'email': FieldInfo(annotation=EmailStr, required=True, metadata=[Format(value='email')]),
'role': FieldInfo(annotation=Literal['admin', 'viewer', 'support'], required=True),
'metadata': FieldInfo(annotation=Union[Metadata, NoneType], required=False, default=None)}
>>>
>>> # Make an API request with the same invalid parmeter values, but with validate=True option
>>> r = client.USERS.create_user(first_name=123, email="foo", role="something", metadata=Metadata(social_links=SocialLinks(facebook="test")), extra=123, validate=True)
2024-01-01T00:00:00.830-0800 - The request contains one or more parameters UsersAPI.create_user() does not expect:
- extra
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
<snip>
raise ValueError(
ValueError: Request parameter validation failed.
6 validation errors for UsersAPICreateUserEndpointModel
first_name
Input should be a valid string [type=string_type, input_value=123, input_type=int]
For further information visit https://errors.pydantic.dev/2.6/v/string_type
last_name
Field required [type=missing, input_value={'first_name': 123, 'emai... 'test'}}, 'extra': 123}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.6/v/missing
email
value is not a valid email address: The email address is not valid. It must have exactly one @-sign. [type=value_error, input_value='foo', input_type=str]
role
Input should be 'admin', 'viewer' or 'support' [type=literal_error, input_value='something', input_type=str]
For further information visit https://errors.pydantic.dev/2.6/v/literal_error
metadata.social_links.facebook
Input should be a valid URL, relative URL without a base [type=url_parsing, input_value='test', input_type=str]
For further information visit https://errors.pydantic.dev/2.6/v/url_parsing
extra
Extra inputs are not permitted [type=extra_forbidden, input_value=123, input_type=int]
For further information visit https://errors.pydantic.dev/2.6/v/extra_forbidden
Tip
The above example enables validation mode "per request" by passing validate=True
to the API function call.
Alternatively, you can also set VALIDATION_MODE='true'
environment variable to globally enable the validation mode.