Easy methods to Implement Structured Output with Native LLMs

Earlier than Q, Okay, and V: Reconstructing the Transformer

Matplotlib vs Plotly: Which Python Chart Software Ought to You Select?

, native LLMs are a lovely choice.

They permit us to maintain our delicate knowledge and scale back our dependency on cloud APIs.

Nonetheless, operating the mannequin domestically is barely step one. In a sensible utility, the native LLM is normally half of a bigger workflow. This implies its responses typically must be consumed by one other part.

In these conditions, free-form textual content will be very troublesome to work with. We would like the output to comply with some predictable constructions.

That’s precisely what Structured Output is for.

We will obtain that by first defining the anticipated form, or schema, prematurely. The native serving runtime then constrains the LLM era to comply with that schema. Lastly, the LLM would give us an everyday Python object that our code can simply parse.

On this submit, we’ll illustrate this sample by way of a concrete case research. We’ll use Gemma 4 as our native LLM, Ollama because the serving runtime, and Pydantic to outline and validate the output schema.

1. How Do We Implement Structured Output with a Native LLM?

1.1 A Good-Residence Case Examine

Suppose we’re constructing a smart-home utility. The person asks a easy query:

Ought to the dishwasher run now or later?

Earlier than answering, the appliance must extract gadget data, timing constraints, and electrical energy tariffs from family notes.

Since these notes comprise non-public data, an area LLM is a pure match as step one. It could actually rework the unique notes right into a structured object that retains solely the details wanted for scheduling whereas eradicating pointless private particulars.

We will then cross this sanitized object to a extra succesful cloud LLM for reasoning and scheduling. Right here, let’s deal with the native transformation step.

The next is the family context we’ll use:

USER_QUESTION = "Ought to the dishwasher run now or later?"

SMART_HOME_CONTEXT = """
It's presently 18:30.

The exercise log data that the robotic vacuum accomplished at this time's kitchen cross
at 16:10 and returned to its dock. No extra vacuuming is required at this time.

The dishwasher's earliest begin is eighteen:30. A cycle takes 90 minutes and makes use of about 1.2 kWh.
It should be full earlier than breakfast at 06:30. As a result of the dishwasher is beside
the bedrooms, it should cease operating by 22:30.

The EV charger's earliest begin is eighteen:30. Charging will take 120 minutes and use about
14 kWh. The automotive should be charged earlier than its driver leaves at 07:00.

The dryer's earliest begin is nineteen:00. Its cycle takes 75 minutes and makes use of about
3.2 kWh. It comprises the soccer equipment, which should be dry by 23:00. The dryer is
too loud later within the night, so it should cease operating by 21:30.

The washer's earliest begin is 20:00. Its cycle takes 60 minutes and
makes use of about 0.9 kWh. It comprises tomorrow's work garments and should end by 05:30.

A kitchen cross with the robotic vacuum takes 45 minutes and makes use of about 0.2 kWh.
The vacuum's earliest begin was 15:00.

The house vitality controller permits just one versatile load to run at a time.
Electrical energy prices 0.45 per kWh from 17:00 to twenty:00, 0.22 from 20:00 to 00:00,
0.12 from 00:00 to 06:00, and 0.25 from 06:00 to 17:00.
""".strip()

The purpose of the native LLM is to retain the scheduling details whereas leaving these private particulars behind.

1.2 Outline the Anticipated Construction

Subsequent, we have to outline what the sanitized object ought to seem like.

The downstream part wants the present time, the gadget talked about within the query, the controller capability, and the electrical energy costs. It additionally wants the gadgets that also require scheduling, along with their runtime and timing necessities.

We will symbolize this utilizing the next Pydantic fashions:

from typing import Annotated

from pydantic import BaseModel, Subject

ClockTime = Annotated[
    str,
    Field(
        min_length=5,
        max_length=5,
        description="Clock time in HH:MM format.",
    ),
]


class DeviceToSchedule(BaseModel):
    device_name: str
    duration_minutes: int
    energy_kwh: float
    earliest_start: ClockTime
    finish_by: ClockTime | None


class SchedulingContext(BaseModel):
    current_time: ClockTime
    focus_device: str
    max_concurrent_devices: int

    current_price_per_kwh: float
    off_peak_start: ClockTime
    off_peak_end: ClockTime
    off_peak_price_per_kwh: float

    devices_to_schedule: listing[DeviceToSchedule] = Subject(
        description=(
            "Units that haven't accomplished their work "
            "and nonetheless must be scheduled."
        )
    )

Be aware that we’ve a nested schema, however the construction is comparatively straightforward to comply with. SchedulingContext comprises the shared family details and a listing of DeviceToSchedule objects.

That’s the form we would like the native LLM to output.

1.3 Setting Ollama and Native LLM

Earlier than transferring ahead, make it possible for Ollama is put in and operating domestically. You’ll be able to set up Ollama on Home windows:

winget set up Ollama.Ollama

On macOS or Linux, run:

"curl -fsSL https://ollama.com/set up.sh | sh"

As soon as Ollama is put in, we will pull the Gemma 4 mannequin:

ollama pull gemma4:e4b

We additionally want the Ollama Python shopper and Pydantic:

pip set up ollama pydantic

Right here, we use the compact 4B variant of the Gemma 4 mannequin for our present case research.

1.4 Join Pydantic to Ollama

Now, we join the schema to our native mannequin.

Right here is how we will obtain that:

import ollama

def call_local_llm(schema, directions, immediate):
    response = ollama.chat(
        mannequin="gemma4:e4b",
        messages=[
            {"role": "system", "content": instructions},
            {"role": "user", "content": prompt},
        ],
        suppose="medium",
        format=schema.model_json_schema(),
    )

    return schema.model_validate_json(response.message.content material)

Two vital issues value mentioning right here:

model_json_schema() converts our Pydantic mannequin into the schema, after which handed into Ollama through the format argument.
model_validate_json() parses the response into the identical Pydantic mannequin. This permits straightforward consumption within the downstream steps.

1.5 Make the Structured-Output Name

Now we will ask Gemma 4 to remodel the family notes.

The instruction is straightforward:

STRUCTURING_INSTRUCTIONS = """
Convert the provided supply materials into the structured scheduling context.
Don't resolve or suggest a schedule.
""".strip()


def build_structuring_prompt(source_material):
    return f"""
Consumer query:
{USER_QUESTION}

Supply materials:
{source_material}
""".strip()

Lastly, we cross the whole SchedulingContext schema to the native mannequin:

one_step_context = call_local_llm(
    SchedulingContext,
    STRUCTURING_INSTRUCTIONS,
    build_structuring_prompt(SMART_HOME_CONTEXT),
)

That’s it.

To realize structured output with an area LLM, we first outline the anticipated construction with Pydantic fashions, then use it to constrain the mannequin era, and at last parse the response again.

That’s the psychological mannequin you want for structured output.

2. Legitimate Construction, Improper Content material

Now, let’s put it into follow and see what Gemma 4 returns.

We run the one-step name and examine the returned object:

print(kind(one_step_context).__name__)

print([
    device.device_name
    for device in one_step_context.devices_to_schedule
])

Listed below are the outcomes:

SchedulingContext

[
    "Dishwasher",
    "EV Charger",
    "Washing Machine",
    "Robot Vacuum (Kitchen Pass)"
]

On the floor, every thing appeared to work as anticipated.

Gemma 4 returned legitimate JSON that follows our schema. Pydantic additionally efficiently parsed it right into a SchedulingContext object.

Nonetheless, there may be one downside: the robotic vacuum shouldn’t be included.

Within the family notes, it clearly states that the robotic vacuum accomplished its kitchen cross at 16:10 and that no extra vacuuming is required at this time. Due to this fact, it doesn’t belong in devices_to_schedule.

That is an attention-grabbing outcome, and truly it gave us an vital distinction in follow:

Structured output solely enforces the form of the response. It doesn’t, by it self, assure that the mannequin places the fitting data inside that form.

So why did the mannequin get it fallacious?

In that one-step name, Gemma 4 has to carry out a number of duties on the similar time, as required by the schema:

Decide which gadgets nonetheless want scheduling.
Extract the related details for these gadgets.
Map these details to the proper schema fields.
Assemble the ultimate nested object.

That’s plenty of work to do for such a small native LLM!

Due to this fact, because the schema turns into extra advanced, the mannequin is beneath stress to coordinate extra choices inside a single era, which makes it extra more likely to make errors.

So how can we tackle this problem?

3. Decompose the Activity

One sensible technique to tackle this problem is to decompose the duty.

In our present case research, as an alternative of asking Gemma 4 to do every thing in a single go, we will break the duty into two steps:

Decide which gadgets nonetheless want scheduling.
Extract the scheduling details for these chosen gadgets.

Step 1: Decide the Scheduling Scope

For step one, we’d like a brand new however small schema:

class SchedulingScope(BaseModel):
    focus_device: str
    device_names_to_schedule: listing[str]

We’ve got the corresponding instruction:

SCOPE_INSTRUCTIONS = """
Establish the main target gadget and the family gadgets that also want scheduling.
Don't resolve or suggest a schedule.
""".strip()

We cross the identical person query and family notes to Gemma 4:

scope = call_local_llm(
    SchedulingScope,
    SCOPE_INSTRUCTIONS,
    build_structuring_prompt(SMART_HOME_CONTEXT),
)

print(scope.model_dump_json(indent=2))

This time, the result’s:

{
  "focus_device": "Dishwasher",
  "device_names_to_schedule": [
    "Dishwasher",
    "EV charger",
    "Washing machine"
  ]
}

Be aware that the robotic vacuum is not included. Gemma 4 appropriately identifies that solely the dishwasher, EV charger, and washer nonetheless want scheduling.

Step 2: Fill the Closing Schema

Subsequent, we ask Gemma 4 to extract the remaining details and fill the ultimate SchedulingContext:

DETAILS_INSTRUCTIONS = """
Convert the provided supply materials into the structured scheduling context
for the provided gadgets. Don't resolve or suggest a schedule.
""".strip()


details_prompt = f"""
Chosen gadgets:
{json.dumps(scope.device_names_to_schedule)}

Consumer query:
{USER_QUESTION}

Supply materials:
{SMART_HOME_CONTEXT}
""".strip()


decomposed_context = call_local_llm(
    SchedulingContext,
    DETAILS_INSTRUCTIONS,
    details_prompt,
)

We embrace the gadget listing produced in step one along with the unique query and supply materials within the immediate above.

Now, let’s examine the whole outcome:

print(decomposed_context.model_dump_json(indent=2))

That is what I obtained:

{
  "current_time": "18:30",
  "focus_device": "Dishwasher",
  "max_concurrent_devices": 1,
  "current_price_per_kwh": 0.45,
  "off_peak_start": "00:00",
  "off_peak_end": "06:00",
  "off_peak_price_per_kwh": 0.12,
  "devices_to_schedule": [
    {
      "device_name": "Dishwasher",
      "duration_minutes": 90,
      "energy_kwh": 1.2,
      "earliest_start": "18:30",
      "finish_by": "06:30"
    },
    {
      "device_name": "EV charger",
      "duration_minutes": 120,
      "energy_kwh": 14.0,
      "earliest_start": "18:30",
      "finish_by": "07:00"
    },
    {
      "device_name": "Washing machine",
      "duration_minutes": 60,
      "energy_kwh": 0.9,
      "earliest_start": "20:00",
      "finish_by": "05:30"
    }
  ]
}

This time, the whole result’s appropriate. The robotic vacuum is excluded, all three gadget data match the unique family notes, and the private information can also be gone.

Due to this fact, we will conclude that the direct method returned a legitimate construction however with incorrect content material, whereas our staged method returned each a legitimate construction and proper content material.

4. Closing Ideas

Native LLMs are a lovely choice when an utility works with delicate knowledge. With structured output, we will combine native LLMs into a bigger workflow, the place the downstream elements can simply devour LLMs’ outcomes.

On this submit, we present that the implementation is easy. We begin by defining the anticipated schema with Pydantic, then passing it to Ollama, and at last parsing the response again right into a validated Python object.

In follow, nonetheless, one catch you need to all the time have in mind is that legitimate construction doesn’t assure appropriate content material. In our instance, the direct name adopted the schema however nonetheless produced the fallacious outcomes.

We successfully solved this downside by adopting a staged method to separate scope willpower from reality extraction, which led to appropriate outcomes.

So, when a small native LLM struggles with a comparatively advanced schema, decomposition is one sensible technique value attempting.