Utilized Python Chronicles: A Light Intro to Pydantic | by Ilija Lazarevic | Jul, 2024

What about default values and argument extractions?

from pydantic import validate_call
@validate_call(validate_return=True)
def add(*args: int, a: int, b: int = 4) -> int:
return str(sum(args) + a + b)
# ----
add(4,3,4)
> ValidationError: 1 validation error for add
a
Lacking required key phrase solely argument [type=missing_keyword_only_argument, input_value=ArgsKwargs((4, 3, 4)), input_type=ArgsKwargs]
For additional data go to 
# ----
add(4, 3, 4, a=3)
> 18
# ----
@validate_call
def add(*args: int, a: int, b: int = 4) -> int:
return str(sum(args) + a + b)
# ---- 
add(4, 3, 4, a=3)
> '18'

Takeaways from this instance:

• You’ll be able to annotate the kind of the variable variety of arguments declaration (*args).
• Default values are nonetheless an choice, even in case you are annotating variable information sorts.
• validate_call accepts validate_return argument, which makes perform return worth validation as effectively. Knowledge kind coercion can be utilized on this case. validate_return is about to False by default. Whether it is left as it’s, the perform might not return what is asserted in kind hinting.

What about if you wish to validate the info kind but additionally constrain the values that variable can take? Instance:

from pydantic import validate_call, Area
from typing import Annotated 
type_age = Annotated[int, Field(lt=120)]
@validate_call(validate_return=True)
def add(age_one: int, age_two: type_age) -> int:
return age_one + age_two
add(3, 300)
> ValidationError: 1 validation error for add
1
Enter ought to be lower than 120 [type=less_than, input_value=200, input_type=int]
For additional data go to 

This instance exhibits:

• You need to use Annotated and pydantic.Area to not solely validate information kind but additionally add metadata that Pydantic makes use of to constrain variable values and codecs.
• ValidationError is but once more very verbose about what was fallacious with our perform name. This may be actually useful.

Right here is yet one more instance of how one can each validate and constrain variable values. We’ll simulate a payload (dictionary) that you simply wish to course of in your perform after it has been validated:

from pydantic import HttpUrl, PastDate
from pydantic import Area
from pydantic import validate_call
from typing import Annotated
Identify = Annotated[str, Field(min_length=2, max_length=15)]
@validate_call(validate_return=True)
def process_payload(url: HttpUrl, identify: Identify, birth_date: PastDate) -> str:
return f'{identify=}, {birth_date=}'
# ----
payload = {
'url': 'httpss://instance.com',
'identify': 'J',
'birth_date': '2024-12-12'
}
process_payload(**payload)
> ValidationError: 3 validation errors for process_payload
url
URL scheme ought to be 'http' or 'https' [type=url_scheme, input_value='httpss://example.com', input_type=str]
For additional data go to 
identify
String ought to have at the very least 2 characters [type=string_too_short, input_value='J', input_type=str]
For additional data go to 
birth_date
Date ought to be up to now [type=date_past, input_value='2024-12-12', input_type=str]
For additional data go to 
# ----
payload = {
'url': '',
'identify': 'Joe-1234567891011121314',
'birth_date': '2020-12-12'
}
process_payload(**payload)
> ValidationError: 1 validation error for process_payload
identify
String ought to have at most 15 characters [type=string_too_long, input_value='Joe-1234567891011121314', input_type=str]
For additional data go to 

This was the fundamentals of the right way to validate perform arguments and their return worth.

Now, we are going to go to the second most vital manner Pydantic can be utilized to validate and course of information: by way of defining fashions.

This half is extra attention-grabbing for the needs of knowledge processing, as you will notice.

To this point, we have now used validate_call to brighten capabilities and specified perform arguments and their corresponding sorts and constraints.

Right here, we outline fashions by defining mannequin lessons, the place we specify fields, their sorts, and constraints. That is similar to what we did beforehand. By defining a mannequin class that inherits from Pydantic BaseModel, we use a hidden mechanism that does the info validation, parsing, and serialization. What this offers us is the flexibility to create objects that conform to mannequin specs.

Right here is an instance:

from pydantic import Area
from pydantic import BaseModel
class Particular person(BaseModel):
identify: str = Area(min_length=2, max_length=15)
age: int = Area(gt=0, lt=120)
# ----
john = Particular person(identify='john', age=20)
> Particular person(identify='john', age=20)
# ----
mike = Particular person(identify='m', age=0)
> ValidationError: 2 validation errors for Particular person
identify
String ought to have at the very least 2 characters [type=string_too_short, input_value='j', input_type=str]
For additional data go to 
age
Enter ought to be larger than 0 [type=greater_than, input_value=0, input_type=int]
For additional data go to 

You need to use annotation right here as effectively, and you may as well specify default values for fields. Let’s see one other instance:

from pydantic import Area
from pydantic import BaseModel
from typing import Annotated 
Identify = Annotated[str, Field(min_length=2, max_length=15)]
Age = Annotated[int, Field(default=1, ge=0, le=120)]
class Particular person(BaseModel):
identify: Identify
age: Age
# ----
mike = Particular person(identify='mike')
> Particular person(identify='mike', age=1)

Issues get very attention-grabbing when your use case will get a bit advanced. Keep in mind the payload that we outlined? I’ll outline one other, extra advanced construction that we’ll undergo and validate. To make it extra attention-grabbing, let’s create a payload that we’ll use to question a service that acts as an middleman between us and LLM suppliers. Then we are going to validate it.

Right here is an instance:

from pydantic import Area
from pydantic import BaseModel
from pydantic import ConfigDict
from typing import Literal
from typing import Annotated
from enum import Enum
payload = {
"req_id": "take a look at",
"textual content": "It is a pattern textual content.",
"instruction": "embed",
"llm_provider": "openai",
"llm_params": {
"llm_temperature": 0,
"llm_model_name": "gpt4o"
},
"misc": "what"
}
ReqID = Annotated[str, Field(min_length=2, max_length=15)]
class LLMProviders(str, Enum):
OPENAI = 'openai'
CLAUDE = 'claude'
class LLMParams(BaseModel):
temperature: int = Area(validation_alias='llm_temperature', ge=0, le=1)
llm_name: str = Area(validation_alias='llm_model_name', 
serialization_alias='mannequin')
class Payload(BaseModel):
req_id: str = Area(exclude=True)
textual content: str = Area(min_length=5)
instruction: Literal['embed', 'chat']
llm_provider: LLMProviders
llm_params: LLMParams
# model_config = ConfigDict(use_enum_values=True)
# ----
validated_payload = Payload(**payload)
validated_payload
> Payload(req_id='take a look at', 
textual content='It is a pattern textual content.', 
instruction='embed', 
llm_provider=, 
llm_params=LLMParams(temperature=0, llm_name='gpt4o'))
# ----          
validated_payload.model_dump()
> {'textual content': 'It is a pattern textual content.',
'instruction': 'embed',
'llm_provider': ,
'llm_params': {'temperature': 0, 'llm_name': 'gpt4o'}}
# ----
validated_payload.model_dump(by_alias=True)
> {'textual content': 'It is a pattern textual content.',
'instruction': 'embed',
'llm_provider': ,
'llm_params': {'temperature': 0, 'mannequin': 'gpt4o'}}
# ----
# After including 
#     model_config = ConfigDict(use_enum_values=True)
# in Payload mannequin definition, you get
validated_payload.model_dump(by_alias=True)
> {'textual content': 'It is a pattern textual content.',
'instruction': 'embed',
'llm_provider': 'openai',
'llm_params': {'temperature': 0, 'mannequin': 'gpt4o'}}

A few of the vital insights from this elaborated instance are:

• You need to use Enums or Literal to outline an inventory of particular values which might be anticipated.
• In case you wish to identify a mannequin’s discipline otherwise from the sector identify within the validated information, you should utilize validation_alias. It specifies the sector identify within the information being validated.
• serialization_alias is used when the mannequin’s inner discipline identify is just not essentially the identical identify you wish to use whenever you serialize the mannequin.
• Area could be excluded from serialization with exclude=True.
• Mannequin fields could be Pydantic fashions as effectively. The method of validation in that case is finished recursively. This half is absolutely superior, since Pydantic does the job of going into depth whereas validating nested constructions.
• Fields that aren’t taken into consideration within the mannequin definition should not parsed.

Right here I’ll present you the snippets of code that present the place and the way you should utilize Pydantic in your day-to-day duties.

Say you’ve gotten information it is advisable validate and course of. It may be saved in CSV, Parquet recordsdata, or, for instance, in a NoSQL database within the type of a doc. Let’s take the instance of a CSV file, and let’s say you wish to course of its content material.

Right here is the CSV file (take a look at.csv) instance:

identify,age,bank_account
johnny,0,20
matt,10,0
abraham,100,100000
mary,15,15
linda,130,100000

And right here is how it’s validated and parsed:

from pydantic import BaseModel
from pydantic import Area 
from pydantic import field_validator
from pydantic import ValidationInfo
from typing import Checklist
import csv
FILE_NAME = 'take a look at.csv'
class DataModel(BaseModel):
identify: str = Area(min_length=2, max_length=15)
age: int = Area(ge=1, le=120)
bank_account: float = Area(ge=0, default=0)
@field_validator('identify')
@classmethod
def validate_name(cls, v: str, information: ValidationInfo) -> str:
return str(v).capitalize()
class ValidatedModels(BaseModel):
validated: Checklist[DataModel]
validated_rows = []
with open(FILE_NAME, 'r') as f:
reader = csv.DictReader(f, delimiter=',')
for row in reader:
strive:
validated_rows.append(DataModel(**row))
besides ValidationError as ve:
# print out error
# disregard the file
print(f'{ve=}')
validated_rows
> [DataModel(name='Matt', age=10, bank_account=0.0),
DataModel(name='Abraham', age=100, bank_account=100000.0),
DataModel(name='Mary', age=15, bank_account=15.0)]
validated = ValidatedModels(validated=validated_rows)
validated.model_dump()
> {'validated': [{'name': 'Matt', 'age': 10, 'bank_account': 0.0},
{'name': 'Abraham', 'age': 100, 'bank_account': 100000.0},
{'name': 'Mary', 'age': 15, 'bank_account': 15.0}]}

FastAPI is already built-in with Pydantic, so this one goes to be very transient. The best way FastAPI handles requests is by passing them to a perform that handles the route. By passing this request to a perform, validation is carried out routinely. One thing just like validate_call that we talked about initially of this text.

Instance of app.py that’s used to run FastAPI-based service:

from fastapi import FastAPI
from pydantic import BaseModel, HttpUrl
class Request(BaseModel):
request_id: str
url: HttpUrl
app = FastAPI()
@app.submit("/search/by_url/")
async def create_item(req: Request):
return merchandise

Pydantic is a very highly effective library and has a whole lot of mechanisms for a mess of various use circumstances and edge circumstances as effectively. At this time, I defined essentially the most fundamental elements of how you must use it, and I’ll present references beneath for many who should not faint-hearted.

Go and discover. I’m certain it should serve you effectively on completely different fronts.