Document Extraction: Getting Facts from PDFs

---

Tamil Ancestry Research | Family Tree Guide

---

In this article:

1. What “document extraction” actually means
2. Which documents are worth extracting from (and which are just noise)
3. The four layers: image → text → facts → links
4. What to extract: a small, brutal checklist
5. How to store extracted facts + confidence in TamizhConnect
6. Common mistakes that quietly corrupt your data
7. A practical extraction workflow for one messy PDF

---

1. What “document extraction” actually means

Most families do this:

• Scan birth certificates, pattas, temple books, e-rolls, letters, school records.
• Dump them into Google Drive / WhatsApp / “Documents” folder.
• Feel satisfied because “everything is digitised”.

That’s not genealogy. That’s digital hoarding.

Document extraction is:

Taking a document (scan, PDF, photo) and pulling out specific, checkable facts – names, dates, places, relationships, roles – then storing those facts as structured data linked back to the original.

You’re not trying to rewrite the whole document.
You’re trying to:

• extract the signal,
• keep the original as evidence,
• and make search + analysis possible.

If you’re not doing extraction, your PDFs are just prettier piles of paper.

---

2. Which documents are worth extracting from (and which are just noise)

Not every document deserves the same effort. Prioritise.

2.1. High-value documents

These directly affect your tree and timelines:

• Vital records
  • birth, baptism, NIC/Aadhaar, marriage, death certificates
• Land and property
  • pattas, sale deeds, inām records, lease documents, mortgage papers
• Migration and ID
  • passports, visas, PR cards, ship lists, estate registers, ration cards
• Education & employment
  • school/college records, appointment orders, service books
• Temple / church / mosque
  • pooja registers, donor lists, trust minutes when they name your people.

These are worth systematic extraction.

2.2. Medium-value documents

Good for context and cross-checking:

• court case copies,
• association membership lists,
• old letters with dates/addresses,
• newspaper obituaries,
• festival souvenir books.

Extract selectively: only the pieces that touch your people.

2.3. Low-value documents

Mostly noise:

• generic religious pamphlets,
• random motivational PDFs,
• WhatsApp forwards,
• undated political rants,
• adverts / marketing junk.

Don’t waste time “extracting” from these. If they don’t add verifiable facts, ignore them.

---

3. The four layers: image → text → facts → links

If you don’t separate these layers, you’ll make a mess.

3.1. Layer 1 – Image

• Raw scan / photo / PDF page.
• This is your evidence.
• You never edit history here; you only attach metadata.

3.2. Layer 2 – Text

• Either:
  • OCR output (Tamil / English / Sinhala etc.), or
  • manually typed transcription.
• Still close to the original:
  • includes errors, formatting issues, noise.
• Purpose:
  • make the document searchable,
  • make manual extraction faster.

3.3. Layer 3 – Facts

From the text, you pull specific units like:

• “Person X born on Y at Place Z.”
• “Person A married Person B on Date, at Place.”
• “Person C listed as patta holder of Survey S in Village V in year Y.”
• “Household listed at address A in e-roll year Y.”

Each fact gets:

• a type,
• structured fields,
• a confidence level,
• a link back to the document.

3.4. Layer 4 – Links into TamizhConnect

Facts then hook into:

• Person profiles
• Places (villages/towns/temples)
• Land parcels
• Events (birth, marriage, migration, land transfer, etc.)

That’s where your tree and maps become grounded in something more than memory.

---

4. What to extract: a small, brutal checklist

Stop trying to “extract everything”. You’ll burn out. Pull what actually matters.

4.1. From birth / baptism / naming records

Extract:

• person’s full name as written,
• date of birth / baptism,
• place of birth (village, town, hospital if given),
• parents’ names as written,
• religion / temple / church (if explicit),
• any informants or witnesses.

Ignore long boilerplate legal text. It’s the same on every certificate.

4.2. From marriage records

Extract:

• both partners’ names,
• date and place of marriage,
• each partner’s age, occupation, and address at time of marriage (if given),
• parents’ names and whether alive/deceased (if present),
• witness names.

These are gold for linking branches and locations.

4.3. From death records

Extract:

• name of deceased (as written),
• date and place of death,
• age at death,
• cause of death (if meaningful),
• informant’s name and relationship,
• last known address.

Useful for bounding birth year and tracking address changes.

4.4. From pattas / land deeds

Extract:

• land parcel ID: survey number + village,
• extent and type (wet/dry/house-site),
• holder/owner names,
• date of transaction/issue,
• type of transaction: sale, gift, inheritance, partition, lease, etc.,
• any clear relationships (s/o, w/o, etc.).

Don’t get lost in long legal clauses unless they introduce new people or dates.

4.5. From e-rolls / voter lists

Extract:

• name as written,
• relation name (father/husband/mother),
• age or year of birth,
• sex,
• house number / address fragment,
• part/section and constituency,
• voter ID.

Plus the year of the roll itself.

4.6. From temple/church/mosque records

Extract:

• devotee/worshipper/donor name (as written),
• any gothram / nakshatram / rasi if present,
• date or festival/year,
• offering / role (donation, lamp, pooja, trustee, worker),
• village or address if written.

Everything else is context.

---

5. How to store extracted facts + confidence in TamizhConnect

If you don’t model extraction properly, you can’t trust your own data.

5.1. Source object

For each document:

• sourceId
• sourceType:
  • "birth-cert", "marriage-cert", "death-cert", "patta", "sale-deed", "e-roll", "temple-record", "school-record", "passport", etc.
• titleOrDescription: "Birth certificate of X", "Patta #123 for Survey 45/2"
• yearApprox
• fileLink (PDF/image)
• language
• ocrStatus (if relevant).

5.2. Fact object

For each extracted fact:

• factId
• factType:
  • "birth", "marriage", "death", "land-holding", "residence", "education", "occupation", "donation", etc.
• sourceId
• pageOrLocation: "page 3, entry 12" or similar
• structured fields depending on type, e.g.:

For a birth fact:

• personId (if linked)
• nameAsWritten
• dateOfBirth (or yearRange if approximate)
• placeId (if mapped)
• fatherNameAsWritten, motherNameAsWritten
• notes.

For a land-holding fact:

• personId[] (list of holders)
• landParcelId
• role ("holder", "co-holder", "tenant")
• effectiveYear.

5.3. Confidence and review

Every fact needs:

• confidence: "high" | "medium" | "low"
• extractionMethod: "manual-typed" | "ocr-reviewed" | "ocr-raw"
• reviewStatus: "not-reviewed" | "reviewed"
• reviewer (optional)
• reviewDate.

Rules of thumb:

• High confidence: typed from clear document; double-checked.
• Medium: OCR + human glance; or old handwriting you’re fairly sure about.
• Low: hard-to-read, partial, or conflicting sources.

High-confidence facts can drive merges and major tree edits.
Medium/low should push you to look for more evidence.

---

6. Avoiding Data Corruption: Common Mistakes in Document Extraction

If you do any of these, fix your habits.

6.1. “Cleaning” names instead of keeping originals

Bad:

• Reading MUTUSAMI and storing only Muthusamy without keeping the exact original spelling as written.

Correct:

• nameAsWritten: "MUTUSAMI"
• nameNormalized: "Muthusamy"

Original stays; your “cleaned” version is an extra, not a replacement.

6.2. Mixing multiple documents into one vague note

Bad:

“He was born around 1950 in Trichy, worked in Chennai, moved to Canada.”

With no source references.

Correct:

• create separate facts for:
  • birth (from cert),
  • address (from e-roll / ration card),
  • migration (from passport/visa),
• each with its own sourceId.

6.3. Guessing dates instead of marking them as approximate

Bad:

• Document says “age 30 in 1978” → you record DOB as 1948-01-01.

Correct:

• store yearOfBirthRange: 1947–1949,
• or approxYearOfBirth: 1948,
• plus note: “derived from age 30 in 1978 death certificate.”

Don’t manufacture exact days out of nothing.

6.4. Over-trusting OCR

Bad:

• Using raw OCR names directly to merge person profiles.
• Assuming every OCR “Nadarajan” is the same “Natarajan”.

Correct:

• for names and places, always visually check against the scan before merging,
• especially if the match changes relationships in the tree.

---

7. A practical extraction workflow for one messy PDF

Take one real-world mess: a 40-page scanned PDF of old records (say, a school or temple book). Here’s how to not screw it up.

Step 1 – Decide scope

Ask:

• “What am I trying to get from this?”

Example:

• “Names, fathers’ names, and villages for anyone related to our three core branches.”

Ignore everything else.

Step 2 – Prepare the document

• Run OCR if the print is reasonably clear.
• If it’s handwriting / terrible print, skip OCR and just view the images.

Step 3 – Make a simple extraction template

Open a spreadsheet or table with columns like:

• SourceId, Page, LineOrEntry,
• NameAsWritten, FatherOrHusbandName,
• VillageOrAddress,
• YearOrDate,
• Notes.

No fancy schema yet. Just structured rows.

Step 4 – Go page by page and log only relevant rows

• For each page, skim for known surnames, initials, or villages.
• When you hit a relevant line:
  • type it into the template,
  • don’t try to reformat; keep it close to what’s on the page.

Step 5 – Import into TamizhConnect

Once you have a few dozen rows:

• create source entry for the document,
• for each row:
  • create fact objects (birth, education, donation, etc.),
  • link to person profiles when you are confident,
  • leave as unlinked if you’re not sure yet.

Step 6 – Review links and adjust tree

Only after facts exist:

• consider merging person profiles,
• update life events (birthplace, schooling, temple roles),
• log conflicts and uncertainties explicitly.

Step 7 – Stop, don’t “perfect” everything

Once you’ve extracted the high-value bits:

• move on to the next document.
• You can always come back later if you discover a new branch that ties into unused pages.

---

If you treat “document extraction” as some vague tech buzzword, you’ll stay stuck at the “We have so many PDFs” stage forever.

If you treat it as:

• a disciplined process of pulling out minimal, high-value facts,
• tying them to people, places, and events,
• and always keeping a clear link back to the original,

then TamizhConnect stops being a pretty file cabinet and actually becomes what it’s supposed to be:

a hard, evidence-backed map of who your people were, where they lived, and how their lives changed over time.