, I constructed a completely automated clever doc processing (IDP) system that ran within the cloud on Amazon Net Companies. The system was designed to permit Freedom of Data-type requests from Irish residents eager to know whether or not their particulars (title, tackle, and so on.) had been recorded in a big insurance coverage claims database maintained by my employer.
At a high-level, the workflow schedule was this,
- Consumer solicitors despatched an e mail to a particular Outlook tackle requesting a search of the shopper’s particulars within the database. For the search, we would have liked the primary and final title of the shopper, along with their date of beginning and present tackle
- A search might solely proceed if there have been corroborating paperwork connected to the e-mail to “show” who the shopper was.
- The attachments would sometimes be PDFs containing pictures of passports, driving licences, and different official paperwork. As well as, we might additionally obtain MS Phrase paperwork or PDFs of financial institution statements and utility payments
- At a specified time every day, the IDP processed these emails, extracted and categorised the attachments and extracted any related Personally Identifiable Data (PII), which was despatched to a claims database for looking.
- If declare particulars had been discovered within the database, a password-protected PDF containing these particulars was created and despatched again to the solicitor, together with a separate e mail containing the password to unlock it. If no outcomes had been discovered, a easy e mail was despatched again indicating this.
Aside from a Human-In-The-Loop (HIL) course of that allowed customers to carry out a last cross-check as as to if the PII extracted from completely different paperwork matched, the method was absolutely automated.
By the way, this new course of proved roughly 90% extra environment friendly than the guide system it changed.
On this article, I’m going to indicate you the steps I took to implement this technique on the AWS cloud platform, albeit a barely simplified model of it.
Be aware that aside from being a consumer of Amazon Net Companies (AWS), I’ve no affiliation or affiliation with the corporate.
In the true system I developed, our supply of emails got here from a company Outlook e mail account. I don’t have entry to a kind of now, so as a substitute, I’ll use my private Gmail as our e mail server and assume every e mail incorporates just one attachment. I’m assuming that any attachment will at all times be a picture file (JPG or PNG) of a passport, driving licence, or financial institution assertion. Additionally, the ultimate step in our course of will probably be a easy outcome file on S3 that signifies the classification and PII extraction values for the attachment.
As a result of size of the code snippets and so on., I’ll embody a hyperlink on the finish of the article to a GitHub repo, the place you could find all of the Lambda code, the Step operate code, IAM permissions, and all the things else required for this undertaking.
As that is primarily in regards to the AWS facet of issues, I’m going to imagine that all the things is about up in your e mail shopper to allow an AWS Lambda course of to learn your emails.
For Gmail, briefly, this course of would come with:-
- creating or deciding on a Google Cloud undertaking
- enabling the Gmail API
- enabling OAuth and finishing a one-time Google consent stream.
After that, you’d retailer the returned credentials/refresh tokens securely in one thing like AWS Secrets and techniques Supervisor.
The purpose is to extract the next items of PII from every e mail attachment.
Doc Sort, First Title, Final Title, DoB and Handle
Our take a look at Gmail attachment pictures
I requested Codex to create 3 clearly faux pictures of a passport, a driver’s licence, and a financial institution assertion. Though faux, they comprise the identical info as the true factor. Listed below are these pictures.
I emailed these pictures to my Gmail account and labelled the emails “DocumentProcessing”.
We now have all the things in place to start out creating our IDP.
Listed below are the AWS companies we’ll use to implement our IDP.
- IAM for permissions
- EventBridge for scheduling
- Step operate for orchestration
- Lambda for compute
- S3 for storage
- Secrets and techniques Supervisor for storing OAuth and different secret info
- Textract for OCR
- Bedrock for AI inference
1/ Storing our OAUTH credentials securely
The very first thing we have to do is retailer our Gmail OAuth credentials and refresh token in AWS Secrets and techniques Supervisor. For me, my e mail setup returned a JSON doc that seemed like this.
{
"put in": {
"client_id": "my_client_id.apps.googleusercontent.com",
"client_secret": "MY_CLIENT_SECRET",
"project_id": "my_project_id",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token"
}
}Open the AWS administration console and go to the Secret Supervisor display screen. Click on the “Retailer a brand new secret” button and select “Different sort of secret” as the key sort. Beneath the key/values pairs part, choose the Plaintext TAB and enter your JSON from above. So your display screen ought to seem like this,
Click on Subsequent, then enter an appropriate title and outline on your new secret. Click on Subsequent, then Subsequent once more, earlier than clicking the Retailer button.
2/ Arrange our S3 buckets
We’ll have one general-purpose bucket and 5 folders that can maintain our information.
- uncooked: that is the place the uncooked e mail attachments will probably be downloaded to
- textract: this may maintain the textual content returned by Textract
- outcomes: this may maintain the outcomes of the doc classification and PII info
- audit: logging info
- quarantine: any unclassified pictures/paperwork will probably be positioned right here
3/ Creating our Lambdas
We’ll want three.
i) ingest_email_image retrieves eligible e mail messages and writes every picture to our uncooked folder on Amazon S3.
ii) extract_text sends the S3 pictures to the Amazon Textract service and shops the extracted textual content.
iii) classify_and_extract_pii sends that extracted textual content to Amazon Bedrock Claude Sonnet 4.6, which makes an attempt to categorise the doc sort and extract any related PII and writes the outcome.
Right here’s a simplified diagram of the method we’ll construct.
EventBridge Scheduler
|
v
Commonplace Step Capabilities workflow
|
+-- Ingest Gmail picture attachments -> S3 uncooked/
|
+-- Map: one e mail/picture at a time
|
+-- Textract Lambda -> S3 textract/
|
+-- Bedrock Lambda -> S3 outcomes/
|
+-- Document a hit or failure outcomeLambda 1 – ingest_email_image
The primary Lambda polls our configured Gmail label utilizing read-only OAuth credentials from AWS Secrets and techniques Supervisor. It validates that every e mail incorporates precisely one JPEG or PNG attachment, saves legitimate pictures below the S3 uncooked/ folder, and returns outputs for the following processing stage; invalid or duplicate attachments are reported individually.
After this Lambda runs, it’s best to see the three picture recordsdata saved below the uncooked folder of your S3 bucket.
Additionally observe that the Lambda makes use of the next atmosphere variables.
Variable Worth
-------- ------
DOCUMENT_BUCKET YOUR_OUPUT_BUCKET_NAME
GMAIL_LABEL_NAME YOUR_GMAIl_LABEL_NAME
GMAIL_SECRET_ARN YOUR_SECRETS_MANAGER_ARN
LOG_LEVEL INFO
MAX_IMAGE_BYTES 10000000
MAX_MESSAGES_PER_RUN 25Lambda 2 – extract_text
The second Lambda takes the pictures saved below uncooked/ and runs Amazon Textract’s synchronous document-text detection on them. It saves the extracted textual content, particular person traces, confidence scores, and supply metadata as JSON below the S3 textract/ folder,
Here’s a prettified, shortened model of the standard output after I ran this towards the financial institution assertion picture.
{
"schema_version":1,
"processed_at":"2026-06-25T11:04:19.850574+00:00",
"supply":{
"bucket":"MY_BUCKET",
"image_key":"uncooked/financial institution.png",
"gmail_message_id":"19efe3cd80025b99",
"gmail_attachment_id":"ANGjdJ-NRkTD-45NFgNRZfHyWSKezw5MI03xN7bvTs4PZKuDjbMCZPk--bnSp4lMVldYhISyd64jH9JgN5cJXTm2_KVDFDsrIFDWJAWs7K6CtMJvsb9OYNX7GZSnwptK_U6M3npTu_02mNbOY_Q-PTKt58rcYbIOA7aDbOxlrAqFh3NmoT69FV92oMjy6NqaT1uSXXZY40h62Ofp-JT4s2JGLKCymhi_o49eAPvBj8azguB4WyoowqWiQEh_cL5IdI7uJC7GEIu9LoUZYuxr7AHZBwACeasd-sNbKLYUb7FwDhPpla_iGD9aDyncH8Q",
"mime_type":"picture/png"
},
"textract":{
"model_version":"1.0",
"document_metadata":{
"Pages":1
},
"traces":[
{
"text":"UK Bank Statement",
"confidence":99.98
},
{
"text":"bank",
"confidence":99.99
},
{
"text":"Unastign Uiver Pol, 01426",
"confidence":93.85
},
...
...
...
{
"text":"Terms UK Banks statement fan Their Band condition, dlesstery. AT stalewaly for diescavey fevien 720, 00553 Dear",
"confidence":93.75
},
{
"text":"Contrined Parih Bank ellamty of allimd elines and ptke and corv. llartes hier. uniedarfc lard sand colunrty mp conmontivre",
"confidence":79.02
},
{
"text":"tharter m allar mpercian the Cant'g paymer Is amowe Sane UK Banklila the Serd Promonry à a corditions of larril and",
"confidence":78.91
},
{
"text":"Statemant of Teard of thre D2. Carpltionrnes Other whitan and Contiiomed Earm Conditions of Terms witte pittier tied Oswllas",
"confidence":89.62
},
{
"text":"Terms and Condition, lod Cporl and Balance.",
"confidence":82.19
}
],
"textual content":"UK Financial institution StatementnbanknUnastign Uiver Pol, 01426nUttesline 952016nVobue. 4346nContasc! 84 6207nContasc! 16777982nMame HoldernAccountnJohn SmithnJohn Smithn11 The Excessive StreetnNewTownnNT1 3WEnAccount NU83632nNomenStatement PeriodnSort CodenValidn01/05/2026 to 01/06/2026nStatement PeriodnOpening Balancen01/05/2026 to 01/06/2026nClosing BalancenTransactionsntonDatenDescriptionnAmountnBalancen07906TNnDirect Debitsn£72,000n£59,000n080161BnCard Paymentsn£73,000n£95,000nCard Paynditn090061NnSalary Creditn£95,000n£56,000n064061BnSalary Creditn£23,000n£64,000n095061DnATM Withdrawalsn£60,000n£60,000n075061NnATM With drawalyn£60,000n£64,000n374091DnATM Vithdrawalyn£60,000n£63,000n340061DnATM Withdrawalsn£60,000n£73,000n153381DnATM Withdrawalyn£60,000n£13,000n150491DnATM Withdrawalyn£10,000n£13,000n169010PnATM Withdrawalsn£11,000n£19,000n188191DnATM Withdrawalsn£10,000n£95,000n11111TDnSalary Creditn£60,000n£55,000n11211TDnATM Withdrawalsn£60,000n£95,000nPage NumbernTerms UK Banks assertion fan Their Band situation, dlesstery. AT stalewaly for diescavey fevien 720, 00553 DearnContrined Parih Financial institution ellamty of allimd elines and ptke and corv. llartes hier. uniedarfc lard sand colunrty mp conmontivrentharter m allar mpercian the Cant'g paymer Is amowe Sane UK Banklila the Serd Promonry à a corditions of larril andnStatemant of Teard of thre D2. Carpltionrnes Different whitan and Contiiomed Earm Situations of Phrases witte pittier tied OswllasnTerms and Situation, lod Cporl and Stability."
}
}Lambda 3 – classify_and_extract_pii
The ultimate Lambda takes every block of JSON textual content that Textract scraped from the enter pictures and makes use of AWS Bedrock to categorise the doc sort and extract any PII it could glean. The important thing to success on this step is the mannequin used and the immediate handed to it. For my work undertaking, I used Claude Sonnet 4.6 and a really massive immediate as a result of it needed to deal with extra complicated inputs. However for this extra simple instance, whereas we’ll nonetheless use Sonnet 4.6, we are able to get away with a a lot easier immediate.
First, we have to observe the mannequin ARN related to Sonnet 4.6 within the area you’re working in. You do that from the Bedrock console, so choose that, then click on on the Inference Profiles hyperlink on the left-hand menu bar. Seek for Sonnet 4.6 and observe the suitable ARN. As that is an inference profile, additionally, you will have to be aware of the areas that AWS can route inference calls by means of, as these will must be added to your IAM permissions along with the inference profile permission.
Listed below are the ultimate outputs I acquired after processing the financial institution, passport and driver’s licence attachments.
Financial institution attachment
{
"schema_version":1,
"processed_at":"2026-06-25T11:04:23.221887+00:00",
"supply":{
"bucket":"MY_BUCKET",
"textract_key":"textract/financial institution.png.json"
},
"mannequin":{
"model_id":"us.anthropic.claude-sonnet-4-6"
},
"classification":"bank_statement",
"pii":{
"first_name":"John",
"last_name":"Smith",
"date_of_birth":null,
"tackle":"11 The Excessive Road, NewTown, NT1 3WE"
}
}Passport attachment
{
"schema_version":1,
"processed_at":"2026-06-25T11:04:21.829715+00:00",
"supply":{
"bucket":"MY_BUCKET",
"textract_key":"textract/passport.jpg.json"
},
"mannequin":{
"model_id":"us.anthropic.claude-sonnet-4-6"
},
"classification":"passport",
"pii":{
"first_name":"AVERY",
"last_name":"EXAMPLE",
"date_of_birth":"1991-08-30",
"tackle":"14 Instance Lane, Testford, TE1 2ST"
}
}Driver’s licence attachment
{
"schema_version":1,
"processed_at":"2026-06-25T11:04:22.507427+00:00",
"supply":{
"bucket":"MY_BUCKET",
"textract_key":"textract/driving.jpg.json"
},
"mannequin":{
"model_id":"us.anthropic.claude-sonnet-4-6"
},
"classification":"driving_licence",
"pii":{
"first_name":"AVERY",
"last_name":"EXAMPLE",
"date_of_birth":"1991-08-30",
"tackle":"14 Instance Lane, Testford, TE1 2ST"
}
}This Lambda makes use of the next atmosphere variables.
Variable Worth
-------- ------
BEDROCK_MODEL_ID us.anthropic.claude-sonnet-4-6
DOCUMENT_BUCKET MY_BUCKET
LOG_LEVEL INFO
MAX_INPUT_CHARS 30000
MAX_JOBS_PER_RUN 25The Step Operate
Though for this straightforward instance we might in all probability simply daisy-chain the three Lambdas to run one after one other in code, the most effective observe is to make use of AWS’s orchestration device, referred to as Step. Step capabilities have the added benefit of being retryable after failures and elegantly dealing with errors and timeouts.
AWS makes use of a language referred to as the Amazon State Language (ASL) to outline their Step Capabilities. The ASL code is within the repo. Inside Step, you’ll be able to render a course of right into a helpful stream diagram, which, in our case, seems like this.
It would look a bit difficult, however plenty of that’s the typical scaffolding that surrounds a course of like this reminiscent of error dealing with. It’s mainly simply orchestrating the three Lambdas by calling them one after the opposite.
The EventBridge scheduler
To tie all the things collectively, I wanted a option to kick off the Step operate each day. A simple method to do this on AWS is by way of a service referred to as EventBridge. EventBridge is able to many issues, however certainly one of its most helpful capabilities is to arrange a cron primarily based timing occasion that may name different AWS techniques at a specified date and time. On my work undertaking, I set a scheduled process to run at 11:55 PM every evening to catch all emails within the inbox for that day, so I’ll repeat that right here.
Click on on the EventBridge service from the principle console, then click on the Schedule menu merchandise on the left-hand bar. On the display screen show, click on the “Create schedule” button. It is best to see a display screen like this
Sort in a reputation and outline on your schedule. Go away the “Schedule group” area because the default, then select the “Recurring schedule” possibility. Choose your required Timezone and go for the cron-based schedule. From right here, fill out the fields as you’d for a cron job.
For our instance of Monday by means of Friday at 11:55 PM, this might be
Subsequent, set a versatile time window if you need, together with non-compulsory begin/finish dates. Click on the Subsequent button and choose “AWS Step Capabilities” as your goal. Choose the Step operate title you need to run from the drop-down listing and enter any inputs.
Click on Subsequent. On this display screen, enter NONE for the motion to take after the occasion has completed. You too can enter a lifeless letter queue (DLQ), encryption particulars, and IAM permissions.
Click on Subsequent as soon as once more to go to the Overview display screen, and when you’re pleased with all the small print, click on the “Create schedule” button.
Your schedule will now be “reside”, and your Step operate will execute each weekday at 11:55 PM.
What’s wanted to show this right into a manufacturing system?
So, what I’ve proven you on this article is the naked bones of an automatic IDP system. Wanting again on the real-life IDP system I carried out, listed here are the modifications you would want to correctly productionise this.
- Deal with extra varieties of attachment paperwork, e.g., MS Phrase, plain textual content, PDF, pictures, and handwritten materials.
- Deal with a number of attachments per e mail.
- Code a front-end HIL in order that customers can cross-check the returned PII from completely different paperwork to make sure they match.
- Have the HIL entrance finish routinely set off the following automated stage, e.g. a separate Step operate that implements …
- Sending the PII to the back-end database for search.
- Automating the manufacturing of PDFs containing database search outcomes.
- Routinely password-protecting the PDFs containing the database search outcomes.
- Automating the sending of the PDFs and the separate password file again to the requester.
- Making certain all the things is logged
Be aware that there’s a value related to constructing and growing the system I’ve described above so when you do observe alongside and create actual sources on AWS, please concentrate on this and delete something you now not require to keep away from shock expenses.
Abstract
This text walks by means of the method of constructing a simplified Clever Doc Processing system on AWS. The system begins with emails with a particular label in Gmail, extracts picture attachments to Amazon S3, makes use of Amazon Textract to learn the textual content, after which sends that textual content to Amazon Bedrock with Claude Sonnet to categorise the doc and extract key PII reminiscent of title, date of beginning, and tackle. This work is carried out by AWS Lambda capabilities.
The Lambda calls are orchestrated with a Step Operate, scheduled by EventBridge, and secured with IAM permissions and Secrets and techniques Supervisor. The result’s a sensible AWS pipeline that demonstrates how email-based doc consumption, classification and information extraction could be automated whereas remaining production-aware in design.
We lined plenty of floor and doubtless over-engineered the answer primarily based on the “simplified” enter pictures we had been attempting to course of. For our take a look at instances, we might in all probability have bypassed the AWS Textract step altogether and simply had Bedrock classify and extract the data straight from the pictures.
I wanted the Textract step in my work undertaking as a result of the enter I used to be coping with was extra complicated, together with PDFs and pictures of hand-written textual content. I feel it was value maintaining in that step to indicate that you’ve got choices in case your inputs are lower than simple.
Lastly, I discussed ed some steps you would want to take to show the system I described into a correct, production-ready course of.
For all of the code and ancillary recordsdata, try my GitHub repo on the hyperlink under. Be aware that I’ve redacted ARNs, bucket names, AWS account numbers, and the rest that will pose a privateness or safety threat.
https://github.com/taupirho/document-intake-aws-pipeline
PS I’m available in the market for contract work simply now. Should you or somebody you already know is in search of an skilled information engineer, both distant or Edinburgh, UK-based, with abilities in AWS, AI, Python, SQL, PySpark, DuckDB, and so on., let me know. Yow will discover me on LinkedIn.
