salikhussain71-code / change-order-extractor

# Change-Order Extractor
**A Python pipeline for extracting structured data from construction change orders into validated JSON, with field-level confidence scores, supporting evidence, and human-review warnings.**
Built for the Sledge AI Engineer assignment.
All sample documents and evaluation labels in this repository are synthetic and were created for development and testing.
## Overview
Construction change orders can contain inconsistent formatting, ambiguous values, multiple monetary amounts, and incomplete information. Extracting these fields reliably requires more than matching numbers and labels.
This project implements a layered extraction pipeline that combines deterministic rules, value normalization, validation, evidence tracking, heuristic confidence scoring, and an optional large language model (LLM) fallback.
The central design principle is **trustworthy extraction over forced completion**: when the available evidence is insufficient, the system can return `null` and flag the document for review instead of inventing a value.
## Key Features
- **Multi-format input:** Processes text files and PDFs with extractable text.
- **Structured JSON:** Extracts nine change-order fields into a consistent schema.
- **Evidence tracking:** Preserves supporting source text for individual field values.
- **Field-level confidence:** Assigns heuristic confidence scores to extracted values.
- **Value normalization:** Handles supported date formats, monetary amounts, credits, and approval statuses.
- **Validation and warnings:** Flags missing information, ambiguous values, competing amounts, and potentially unsafe input.
- **Conservative PDF handling:** Flags scanned PDFs without a usable text layer for OCR rather than pretending extraction succeeded.
- **Optional LLM fallback:** Supports configured Gemini or Anthropic providers for eligible unresolved fields.
- **Automated testing:** Includes unit, regression, safety-related, CLI, and mocked LLM fallback tests.
- **Evaluation tooling:** Measures extraction results against labelled synthetic datasets.
- **Reproducible workflow:** Includes sample documents, evaluation labels, dependency files, and generated JSON outputs.
## Quick Start
### Requirements
- Python
- pip
### 1. Clone the repository
Open PowerShell or a terminal:
 [code] 
### 2. Create and activate a virtual environment
On Windows PowerShell:
 [code] 
### 3. Install dependencies
 [code] 
### 4. Run the automated tests
 [code] 
**Verified result:** 49 tests passed in the reported local run.
### 5. Process the sample documents
 [code] 
The command processes the supported sample files and writes generated JSON outputs to `outputs/`.
### 6. Process a single document
 [code] 
### 7. Run the development evaluation
 [code] 
### 8. Run the held-out evaluation
 [code] 
Install dependencies in the environment where you run the project. In particular, `pdfplumber` is required for PDF text extraction.
## Architecture
The pipeline follows a layered workfl