S
ACTIVEAI/ML Engineer · AI extraction serviceProfessionalProduction

InvoicePilot

AI-powered invoice processing platform that extracts, validates, and analyzes invoices while detecting duplicates and anomalies to streamline financial document workflows.

FastAPIPythonMindee OCRPostgreSQLRedis StreamsCloudflare R2

[ 01 ] · Context

Problem

  • Bookkeeping firms process invoices by hand: transcribing totals, matching purchase orders, and hunting for duplicates, an error-prone paper trail that takes hours and leaves no proof of what was actually done.
  • Duplicate payments and fraud slip through because there is no structured audit trail, and any claim that an invoice was processed rests on someone's word.
  • Generic extraction tools pull fields but cannot vouch that the extracted data matches the file, so automation stops before approval, exactly where the hours get saved.

[ 02 ] · Response

Solution

InvoicePilot is a team-built platform for audit-ready invoice operations. A React frontend handles upload, extraction review, and approval; a Fastify API coordinates the workflow; the FastAPI invoice-ai-service performs OCR extraction, normalization, and validation; BullMQ workers run duplicate checks, fraud checks, and the on-chain seal; and a SolidInvoice sync worker hands approved invoices to billing. Auth is Supabase; the database is PostgreSQL with row-level security.

One end-to-end pipeline runs per document: capture → extraction → normalization → duplicate, business, and fraud checks → human approval → on-chain cryptographic seal → accounting sync. A sha-256 of the exact uploaded bytes is the document's identity, re-verified by the AI service, so a seal can never cover a file that doesn't match its hash.

OCR and sealing are deliberately one pipeline: what the OCR extracts is what gets sealed on-chain. Two ingest routes (sync and async) fold onto the same invoices row through a single shared, frozen mapper, so the same document always produces the same on-chain hash.

Async work runs on BullMQ queues (extraction, duplicate-check, anomaly-check, blockchain-seal, solidinvoice-sync) while PostgreSQL row-level security FORCE policies keep every tenant's data isolated at the database, not the ORM.

[ 03 ] · My work

My contribution

  • I built the invoice-ai-service, the platform's extraction backbone: a FastAPI pipeline that turns an uploaded invoice into a structured, validated record: R2 ingest → OCR → transform → normalize → validate → persist.
  • Designed the provider-agnostic OCR layer (base provider plus the Mindee V2 adapter; teammates later added a local Tesseract provider) so the extraction engine can swap OCR vendors without touching app code, and integrated the OCR path with Cloudflare R2 storage.
  • Authored the structured invoice schema and the normalization/transformer services (money normalization, field coercion), including the {value, confidence, present} field contract the frontend's extraction review consumes.
  • Implemented the validation layer and the 12-rule business scoring (empty invoice, non-positive totals, date/currency mismatches, OCR-confidence floors, duplicate line items, and more).
  • Implemented duplicate detection (exact match on normalized vendor + invoice number + date) and the fraud rule set: 9 historical/statistical rules with risk levels that flag amount spikes, invoice-number/counterpart mismatches, and repeated-volume patterns once enough history exists.
  • My invoice-ai-service history was merged into the team monorepo and continues as the extraction backbone; the rest of the stack (frontend, API, worker, contracts, billing sync) was built by teammates.

[ 04 ] · Structure

Architecture

  Frontend (Vite + React) · Supabase auth · Cloudflare R2
        │  multipart upload / presigned R2 URL
        ▼
  Fastify API · JWT (Supabase JWKS) · RBAC · rate-limit · audit log
        │                          │
        │ sync /invoices/process   │ async /invoices → BullMQ
        ▼                          ▼
  invoice-ai-service (FastAPI)     Worker queues
   verify sha-256 · OCR (Mindee)    invoice-extraction · duplicate-check
   transform · normalize ·          anomaly-check · blockchain-seal
   business rules (12) · fraud (9)  solidinvoice-sync
   classify · validate · persist
        │                          │
        ▼──────────────────────────▼
  PostgreSQL 17 · RLS FORCE · NOBYPASSRLS service role
        │
        ▼
  SealRegistry.sol (Base Sepolia)
   { recordHash, recordType, sealedAt, sealedBy } · append-only, no PII

[ 05 ] · Trade-offs

Key engineering decisions

  1. 01

    Hash-only on-chain (privacy by design) (team)

    Built by my teammates on the platform side; my AI service supplies the fields it seals. The contract stores only {recordHash, recordType, sealedAt, sealedBy}, never amounts, PII, or vendor data. Each hash is sha-256 over RFC 8785 (JCS) canonical JSON (NFC-normalized, money in integer minor units) tagged with a versioned domain string, so a document can be verified without any sensitive data leaving its owner's control.

  2. 02

    The frozen anti-fabrication mapper (team)

    Built by my teammates on the platform side; my AI service supplies the fields it seals. The AI service emits {value: 0.0, present: false} for a field it never found. The one shared mapper omits any key the extractor did not produce instead of writing a zero, so a $0 invoice invented from a default can never reach an approver, let alone a seal. It is deliberately frozen in shared-types so both ingest routes seal identical bytes.

  3. 03

    RLS FORCE + service-role split (team)

    Built by my teammates on the platform side; my AI service supplies the fields it seals. Tenant isolation is enforced at the database, not the ORM. The API sets per-request GUCs, the worker connects with app.current_role='service' under a NOBYPASSRLS production role, and FORCE ROW LEVEL SECURITY keeps reads tenant-scoped however the query is written.

  4. 04

    AES-256-GCM for PII at rest (team)

    Built by my teammates on the platform side; my AI service supplies the fields it seals. Vendor bank details and firm tax IDs are encrypted field-level with aes-256-gcm, stored hex-serialized, and tied to a referenced KMS key alias. Decryption happens only where a route genuinely needs the plaintext. The rest of the system never sees it.

  5. 05

    Versioned, frozen approval hashes (team)

    Built by my teammates on the platform side; my AI service supplies the fields it seals. The hashing kernel lives in shared-types behind a versioned domain tag (invoicepilot/approval/v1). v1 seals must stay verifiable forever, so a changed sealed-field set ships as approval-v2 rather than an edit to the payload builder.

  6. 06

    Verify never writes (team)

    Built by my teammates on the platform side; my AI service supplies the fields it seals. The public verify path recomputes sha-256 in-process and calls a keyless view function on-chain: no DB write, no audit row, no signer. Every outcome (match, mismatch, not-yet-sealed, chain error, payload error) resolves explicitly and never throws.

[ 06 ] · Under the hood

Engineering highlights

The hash kernel: one payload becomes a bytes32
// shared-types/src/canonical.ts: the backend-agnostic seal core.
// recordHash = '0x' + sha256( domainTag + '\n' + canonicalJSON(payload) )
export function computeRecordHash(
  payload: unknown,
  domainTag: string,
): `0x${string}` {
  return `0x${sha256Hex(domainTag + '\n' + canonicalJSON(payload))}`;
}
Teammate code, shown for context: RFC 8785 (JCS) canonical JSON + a versioned domain tag make the hash deterministic across languages: the Python service extracts, TypeScript seals, Solidity verifies.
The anti-fabrication rule (frozen mapper)
// AI emits { value: 0.0, present: false } for fields it never found.
// A mapper checking only typeof value === 'number' writes amount 0,
// marks the invoice approvable; then it gets SEALED ON-CHAIN.
// So every key is OMITTED unless the extractor produced it:
export interface ExtractedInvoiceFields {
  vendorName?: string;       // nfc'd, varchar(255)
  invoiceNumber?: string;    // nfc'd, varchar(128)
  amount?: string;           // fixed-4 decimal, numeric(19,4)
  currency?: string;         // /^[A-Z]{3}$/
  lineItems?: NormalizedLineItem[]; // jsonb, sealed as-stored
}
Teammate code, shown for context: An invoice with no extracted total is never handed to an approver: the exact $0-invoice-sealed bug this branch exists to kill.
The single on-chain write, with a fee policy
// seal.ts: the only tx this worker ever pays for.
const [block, suggestedTip] = await Promise.all([
  publicClient.getBlock(),
  publicClient.estimateMaxPriorityFeePerGas(),
]);
const fees = computeSealFees({
  baseFeePerGas: block.baseFeePerGas,
  suggestedPriorityFeePerGas: suggestedTip,
  maxPriorityFeeWei: config.maxPriorityFeeWei,
  baseFeeMultiplier: config.baseFeeMultiplier,
  maxFeeCapWei: config.maxFeeCapWei,
});
if (fees.maxFeePerGas > config.maxFeeCapWei) {
  throw new Error('seal: chain is expensive; not submitting, retry later');
}
const txHash = await walletClient.writeContract({
  address: SEAL_CONTRACT_ADDRESS, abi: sealRegistryAbi,
  functionName: 'sealRecord',
  args: [recordHash, RECORD_TYPE_APPROVAL],
  maxFeePerGas: fees.maxFeePerGas,
  maxPriorityFeePerGas: fees.maxPriorityFeePerGas,
});
Teammate code, shown for context: The tip is our number, not the RPC's. A seal is never urgent, so over a hard fee cap the job waits for a cheaper block instead of overpaying.

[ 07 ] · Outcomes

Results

700
API tests (teammates' Fastify suite)
160
AI-service tests (team repo)
12+9
business + fraud rules
4
BullMQ queues (teammates' worker)
78.6k
gas per on-chain seal (teammates' contract)
2
OCR providers (my Mindee adapter + a teammate's local one)

[ 08 ] · Visuals

Screenshots

[ 09 ] · Where it's going

Status & next steps

  • Reconcile schema drift: base SQL defines 7 invoice_status values, the code enum has 12, the SOC 2 spec lists 15. Align before the base schema is ever re-imported.
  • Add a per-currency minor-unit exponent table (JPY=0, BHD=3) before any non-2-decimal currency reaches production.
  • Backfill pre-006 rows and replace the rule-based anomaly placeholder with the planned Isolation Forest path.
  • Public demo environment with seeded sample invoices so the pipeline is explorable without credentials.

[ 10 ] · Keep exploring

More projects

Back to all projects