Picture by Writer
# Introduction
Do you know that a big portion of useful data nonetheless exists in unstructured textual content? For instance, analysis papers, scientific notes, monetary reviews, and many others. Extracting dependable and structured data from these texts has all the time been a problem. LangExtract is an open-source Python library (launched by Google) that solves this downside utilizing giant language fashions (LLMs). You may outline what to extract by way of easy prompts and some examples, after which it makes use of LLMs (like Google’s Gemini, OpenAI, or native fashions) to drag out that data from paperwork of any size. One other factor that makes it helpful is its help for very lengthy paperwork (by means of chunking and multi-pass processing) and interactive visualization of outcomes. Let’s discover this library in additional element.
# 1. Putting in and Setting Up
To put in LangExtract regionally, first guarantee you’ve got Python 3.10+ put in. The library is on the market on PyPI. In a terminal or digital setting, run:
For an remoted setting, you might first create and activate a digital setting:
python -m venv langextract_env
supply langextract_env/bin/activate # On Home windows: .langextract_envScriptsactivate
pip set up langextract
There are different choices from supply and utilizing Docker as nicely that you would be able to verify from right here.
# 2. Setting Up API Keys (for Cloud Fashions)
LangExtract itself is free and open-source, however in case you use cloud-hosted LLMs (like Google Gemini or OpenAI GPT fashions), you should provide an API key. You may set the LANGEXTRACT_API_KEY setting variable or retailer it in a .env file in your working listing. For instance:
export LANGEXTRACT_API_KEY="YOUR_API_KEY_HERE"
or in a .env file:
cat >> .env << 'EOF'
LANGEXTRACT_API_KEY=your-api-key-here
EOF
echo '.env' >> .gitignore
On-device LLMs by way of Ollama or different native backends don’t require an API key. To allow OpenAI, you’d run pip set up langextract[openai], set your OPENAI_API_KEY, and use an OpenAI model_id. For Vertex AI (enterprise customers), service account authentication is supported.
# 3. Defining an Extraction Job
LangExtract works by you telling it what data to extract. You do that by writing a transparent immediate description and supplying a number of ExampleData annotations that present what an accurate extraction seems to be like on pattern textual content. As an illustration, to extract characters, feelings, and relationships from a line of literature, you may write:
import langextract as lx
immediate = """
Extract characters, feelings, and relationships so as of look.
Use actual textual content for extractions. Don't paraphrase or overlap entities.
Present significant attributes for every entity so as to add context."""
examples = [
lx.data.ExampleData(
text="ROMEO. But soft! What light through yonder window breaks? ...",
extractions=[
lx.data.Extraction(
extraction_class="character",
extraction_text="ROMEO",
attributes={"emotional_state": "wonder"}
),
lx.data.Extraction(
extraction_class="emotion",
extraction_text="But soft!",
attributes={"feeling": "gentle awe"}
)
]
)
]
These examples (taken from LangExtract’s README) inform the mannequin precisely what sort of structured output is predicted. You may create related examples on your area.
# 4. Working the Extraction
As soon as your immediate and examples are outlined, you merely name the lx.extract() operate. The important thing arguments are:
text_or_documents: Your enter textual content, or an inventory of texts, or perhaps a URL string (LangExtract can fetch and course of textual content from a Gutenberg or different URL).prompt_description: The extraction directions (a string).examples: A listing ofExampleDatathat illustrate the specified output.model_id: The identifier of the LLM to make use of (e.g."gemini-2.5-flash"for Google Gemini Flash, or an Ollama mannequin like"gemma2:2b", or an OpenAI mannequin like"gpt-4o").- Different non-compulsory parameters:
extraction_passes(to re-run extraction for larger recall on lengthy texts),max_workers(to do parallel processing on chunks),fence_output,use_schema_constraints, and many others.
For instance:
input_text=""'JULIET. O Romeo, Romeo! wherefore artwork thou Romeo?
Deny thy father and refuse thy title;
Or, if thou wilt not, be however sworn my love,
And I will now not be a Capulet.
ROMEO. Shall I hear extra, or shall I communicate at this?
JULIET. 'Tis however thy title that's my enemy;
Thou artwork thyself, although not a Montague.
What’s in a reputation? That which we name a rose
By every other title would odor as candy.'''
end result = lx.extract(
text_or_documents=input_text,
prompt_description=immediate,
examples=examples,
model_id="gemini-2.5-flash"
)
This sends the immediate and examples together with the textual content to the chosen LLM and returns a Consequence object. LangExtract routinely handles tokenizing lengthy texts into chunks, batching calls in parallel, and merging the outputs.
# 5. Dealing with Output and Visualization
The output of lx.extract() is a Python object (usually referred to as end result) that incorporates the extracted entities and attributes. You may examine it programmatically or reserve it for later. LangExtract additionally supplies helper features to save lots of outcomes: for instance, you may write the outcomes to a JSONL (JSON Strains) file (one doc per line) and generate an interactive HTML assessment. For instance:
lx.io.save_annotated_documents([result], output_name="extraction_results.jsonl", output_dir=".")
html = lx.visualize("extraction_results.jsonl")
with open("viz.html", "w") as f:
f.write(html if isinstance(html, str) else html.information)
This writes an extraction_results.jsonl file and an interactive viz.html file. The JSONL format is handy for giant datasets and additional processing, and the HTML file highlights every extracted span in context (color-coded by class) for straightforward human inspection like this:
# 6. Supporting Enter Codecs
LangExtract is versatile about enter. You may provide:
- Plain textual content strings: Any textual content you load into Python (e.g. from a file or database) will be processed.
- URLs: As proven above, you may move a URL (e.g. a Mission Gutenberg hyperlink) as
text_or_documents="https://www.gutenberg.org/information/1513/1513-0.txt". LangExtract will obtain and extract from that doc. - Record of texts: Cross a Python checklist of strings to course of a number of paperwork in a single name.
- Wealthy textual content or Markdown: Since LangExtract works on the textual content stage, you would additionally feed in Markdown or HTML in case you pre-process it to uncooked textual content. (LangExtract itself doesn’t parse PDFs or photos, it is advisable to extract textual content first.)
# 7. Conclusion
LangExtract makes it simple to show unstructured textual content into structured information. With excessive accuracy, clear supply mapping, and easy customization, it really works nicely when rule-based strategies fall brief. It’s particularly helpful for complicated or domain-specific extractions. Whereas there may be room for enchancment, LangExtract is already a powerful device for extracting grounded data in 2025.
Kanwal Mehreen is a machine studying engineer and a technical author with a profound ardour for information science and the intersection of AI with drugs. She co-authored the e book “Maximizing Productiveness with ChatGPT”. As a Google Technology Scholar 2022 for APAC, she champions variety and educational excellence. She’s additionally acknowledged as a Teradata Range in Tech Scholar, Mitacs Globalink Analysis Scholar, and Harvard WeCode Scholar. Kanwal is an ardent advocate for change, having based FEMCodes to empower girls in STEM fields.
Picture by Writer
# Introduction
Do you know that a big portion of useful data nonetheless exists in unstructured textual content? For instance, analysis papers, scientific notes, monetary reviews, and many others. Extracting dependable and structured data from these texts has all the time been a problem. LangExtract is an open-source Python library (launched by Google) that solves this downside utilizing giant language fashions (LLMs). You may outline what to extract by way of easy prompts and some examples, after which it makes use of LLMs (like Google’s Gemini, OpenAI, or native fashions) to drag out that data from paperwork of any size. One other factor that makes it helpful is its help for very lengthy paperwork (by means of chunking and multi-pass processing) and interactive visualization of outcomes. Let’s discover this library in additional element.
# 1. Putting in and Setting Up
To put in LangExtract regionally, first guarantee you’ve got Python 3.10+ put in. The library is on the market on PyPI. In a terminal or digital setting, run:
For an remoted setting, you might first create and activate a digital setting:
python -m venv langextract_env
supply langextract_env/bin/activate # On Home windows: .langextract_envScriptsactivate
pip set up langextract
There are different choices from supply and utilizing Docker as nicely that you would be able to verify from right here.
# 2. Setting Up API Keys (for Cloud Fashions)
LangExtract itself is free and open-source, however in case you use cloud-hosted LLMs (like Google Gemini or OpenAI GPT fashions), you should provide an API key. You may set the LANGEXTRACT_API_KEY setting variable or retailer it in a .env file in your working listing. For instance:
export LANGEXTRACT_API_KEY="YOUR_API_KEY_HERE"
or in a .env file:
cat >> .env << 'EOF'
LANGEXTRACT_API_KEY=your-api-key-here
EOF
echo '.env' >> .gitignore
On-device LLMs by way of Ollama or different native backends don’t require an API key. To allow OpenAI, you’d run pip set up langextract[openai], set your OPENAI_API_KEY, and use an OpenAI model_id. For Vertex AI (enterprise customers), service account authentication is supported.
# 3. Defining an Extraction Job
LangExtract works by you telling it what data to extract. You do that by writing a transparent immediate description and supplying a number of ExampleData annotations that present what an accurate extraction seems to be like on pattern textual content. As an illustration, to extract characters, feelings, and relationships from a line of literature, you may write:
import langextract as lx
immediate = """
Extract characters, feelings, and relationships so as of look.
Use actual textual content for extractions. Don't paraphrase or overlap entities.
Present significant attributes for every entity so as to add context."""
examples = [
lx.data.ExampleData(
text="ROMEO. But soft! What light through yonder window breaks? ...",
extractions=[
lx.data.Extraction(
extraction_class="character",
extraction_text="ROMEO",
attributes={"emotional_state": "wonder"}
),
lx.data.Extraction(
extraction_class="emotion",
extraction_text="But soft!",
attributes={"feeling": "gentle awe"}
)
]
)
]
These examples (taken from LangExtract’s README) inform the mannequin precisely what sort of structured output is predicted. You may create related examples on your area.
# 4. Working the Extraction
As soon as your immediate and examples are outlined, you merely name the lx.extract() operate. The important thing arguments are:
text_or_documents: Your enter textual content, or an inventory of texts, or perhaps a URL string (LangExtract can fetch and course of textual content from a Gutenberg or different URL).prompt_description: The extraction directions (a string).examples: A listing ofExampleDatathat illustrate the specified output.model_id: The identifier of the LLM to make use of (e.g."gemini-2.5-flash"for Google Gemini Flash, or an Ollama mannequin like"gemma2:2b", or an OpenAI mannequin like"gpt-4o").- Different non-compulsory parameters:
extraction_passes(to re-run extraction for larger recall on lengthy texts),max_workers(to do parallel processing on chunks),fence_output,use_schema_constraints, and many others.
For instance:
input_text=""'JULIET. O Romeo, Romeo! wherefore artwork thou Romeo?
Deny thy father and refuse thy title;
Or, if thou wilt not, be however sworn my love,
And I will now not be a Capulet.
ROMEO. Shall I hear extra, or shall I communicate at this?
JULIET. 'Tis however thy title that's my enemy;
Thou artwork thyself, although not a Montague.
What’s in a reputation? That which we name a rose
By every other title would odor as candy.'''
end result = lx.extract(
text_or_documents=input_text,
prompt_description=immediate,
examples=examples,
model_id="gemini-2.5-flash"
)
This sends the immediate and examples together with the textual content to the chosen LLM and returns a Consequence object. LangExtract routinely handles tokenizing lengthy texts into chunks, batching calls in parallel, and merging the outputs.
# 5. Dealing with Output and Visualization
The output of lx.extract() is a Python object (usually referred to as end result) that incorporates the extracted entities and attributes. You may examine it programmatically or reserve it for later. LangExtract additionally supplies helper features to save lots of outcomes: for instance, you may write the outcomes to a JSONL (JSON Strains) file (one doc per line) and generate an interactive HTML assessment. For instance:
lx.io.save_annotated_documents([result], output_name="extraction_results.jsonl", output_dir=".")
html = lx.visualize("extraction_results.jsonl")
with open("viz.html", "w") as f:
f.write(html if isinstance(html, str) else html.information)
This writes an extraction_results.jsonl file and an interactive viz.html file. The JSONL format is handy for giant datasets and additional processing, and the HTML file highlights every extracted span in context (color-coded by class) for straightforward human inspection like this:
# 6. Supporting Enter Codecs
LangExtract is versatile about enter. You may provide:
- Plain textual content strings: Any textual content you load into Python (e.g. from a file or database) will be processed.
- URLs: As proven above, you may move a URL (e.g. a Mission Gutenberg hyperlink) as
text_or_documents="https://www.gutenberg.org/information/1513/1513-0.txt". LangExtract will obtain and extract from that doc. - Record of texts: Cross a Python checklist of strings to course of a number of paperwork in a single name.
- Wealthy textual content or Markdown: Since LangExtract works on the textual content stage, you would additionally feed in Markdown or HTML in case you pre-process it to uncooked textual content. (LangExtract itself doesn’t parse PDFs or photos, it is advisable to extract textual content first.)
# 7. Conclusion
LangExtract makes it simple to show unstructured textual content into structured information. With excessive accuracy, clear supply mapping, and easy customization, it really works nicely when rule-based strategies fall brief. It’s particularly helpful for complicated or domain-specific extractions. Whereas there may be room for enchancment, LangExtract is already a powerful device for extracting grounded data in 2025.
Kanwal Mehreen is a machine studying engineer and a technical author with a profound ardour for information science and the intersection of AI with drugs. She co-authored the e book “Maximizing Productiveness with ChatGPT”. As a Google Technology Scholar 2022 for APAC, she champions variety and educational excellence. She’s additionally acknowledged as a Teradata Range in Tech Scholar, Mitacs Globalink Analysis Scholar, and Harvard WeCode Scholar. Kanwal is an ardent advocate for change, having based FEMCodes to empower girls in STEM fields.
