> ## Documentation Index
> Fetch the complete documentation index at: https://docs.parsefy.io/llms.txt
> Use this file to discover all available pages before exploring further.
# Schema Basics
> Learn how to create effective JSON Schemas for financial document extraction
## What is a Schema?
A schema defines the **structure of data** you want to extract from your documents. Parsefy uses [JSON Schema](https://json-schema.org/) to understand exactly what fields to extract, their types, and any validation rules.
If you're using our SDKs, you can define schemas using **Pydantic models** (Python) or **Zod schemas** (TypeScript) instead of raw JSON Schema.
## Basic Structure
Every Parsefy schema is a JSON object with these key properties:
```json theme={null}
{
"type": "object",
"properties": {
"field_name": {
"type": "string",
"description": "What this field contains"
}
},
"required": ["field_name"]
}
```
### Schema Properties
| Property | Required | Description |
| ------------ | -------- | ------------------------------------- |
| `type` | Yes | Always `"object"` for the root schema |
| `properties` | Yes | Object containing field definitions |
| `required` | No | Array of required field names |
Add `description` to each **field** to help the AI understand what to extract. Field-level descriptions are much more valuable than top-level schema descriptions.
## ⚠️ Required vs Optional Fields (Critical for Billing)
**All fields are required by default** in both SDKs. This significantly impacts your costs because required fields that return `null` or fall below the confidence threshold trigger the expensive fallback model.
### How It Works
| User writes (SDK) | SDK converts to (JSON Schema) | API interprets as |
| ----------------------------- | ----------------------------- | ------------------------------------------------- |
| `name: z.string()` | `required: ["name"]` | **Required**: triggers fallback if low confidence |
| `name: z.string().optional()` | `required: []` | **Optional**: won't trigger fallback |
| `name: str` (Python) | `required: ["name"]` | **Required**: triggers fallback if low confidence |
| `name: str \| None = None` | `required: []` | **Optional**: won't trigger fallback |
### Why This Matters
If a **required** field returns `null` or falls below the `confidence_threshold`:
1. The API automatically triggers the **fallback model** (Tier 2)
2. Tier 2 is significantly more expensive
3. Your costs increase unexpectedly
### Best Practice: Mark Optional Fields
```typescript TypeScript (Zod) theme={null}
const invoiceSchema = z.object({
// REQUIRED - Core financial data that's always present
invoice_number: z.string().describe('The invoice number'),
total: z.number().describe('Total amount including tax'),
// OPTIONAL - May not appear on all invoices
vendor: z.string().optional().describe('Vendor name'),
tax_id: z.string().optional().describe('Tax ID or VAT number'),
due_date: z.string().optional().describe('Payment due date'),
notes: z.string().optional().describe('Additional notes'),
});
```
```python Python (Pydantic) theme={null}
from pydantic import BaseModel, Field
class Invoice(BaseModel):
# REQUIRED - Core financial data that's always present
invoice_number: str = Field(description="The invoice number")
total: float = Field(description="Total amount including tax")
# OPTIONAL - May not appear on all invoices
vendor: str | None = Field(default=None, description="Vendor name")
tax_id: str | None = Field(default=None, description="Tax ID or VAT number")
due_date: str | None = Field(default=None, description="Payment due date")
notes: str | None = Field(default=None, description="Additional notes")
```
```json JSON Schema theme={null}
{
"type": "object",
"properties": {
"invoice_number": {
"type": "string",
"description": "The invoice number"
},
"total": {
"type": "number",
"description": "Total amount including tax"
},
"vendor": {
"type": "string",
"description": "Vendor name"
},
"tax_id": {
"type": "string",
"description": "Tax ID or VAT number"
},
"due_date": {
"type": "string",
"description": "Payment due date"
},
"notes": {
"type": "string",
"description": "Additional notes"
}
},
"required": ["invoice_number", "total"]
}
```
**Rule of thumb:** If a field might be missing in >20% of your documents, mark it as optional.
## Field Types
Parsefy supports all standard JSON Schema types:
```json theme={null}
{
"invoice_number": {
"type": "string",
"description": "The invoice or receipt number"
}
}
```
```json theme={null}
{
"total_amount": {
"type": "number",
"description": "Total amount due in dollars"
}
}
```
For integers only:
```json theme={null}
{
"quantity": {
"type": "integer",
"description": "Number of items"
}
}
```
```json theme={null}
{
"is_paid": {
"type": "boolean",
"description": "Whether the invoice has been paid"
}
}
```
```json theme={null}
{
"line_items": {
"type": "array",
"description": "List of items on the invoice",
"items": {
"type": "object",
"properties": {
"description": {"type": "string"},
"quantity": {"type": "integer"},
"price": {"type": "number"}
}
}
}
}
```
```json theme={null}
{
"vendor": {
"type": "object",
"description": "Vendor information",
"properties": {
"name": {"type": "string"},
"address": {"type": "string"},
"phone": {"type": "string"}
}
}
}
```
## Complete Financial Document Schema
Here's a comprehensive invoice extraction schema with proper required/optional fields:
```typescript TypeScript (Zod) theme={null}
import * as z from 'zod';
const vendorSchema = z.object({
name: z.string().describe('Company name'),
address: z.string().optional().describe('Full address'),
phone: z.string().optional().describe('Phone number'),
email: z.string().optional().describe('Email address'),
tax_id: z.string().optional().describe('Tax ID or VAT number'),
});
const lineItemSchema = z.object({
description: z.string().describe('Item description'),
quantity: z.number().describe('Number of items'),
unit_price: z.number().describe('Price per unit'),
amount: z.number().describe('Line total'),
});
const invoiceSchema = z.object({
// REQUIRED - Core financial fields
invoice_number: z.string().describe('The invoice number'),
total: z.number().describe('Total amount due including tax'),
currency: z.string().describe('3-letter currency code (USD, EUR, etc.)'),
line_items: z.array(lineItemSchema).describe('List of line items'),
// OPTIONAL - May not be on all invoices
date: z.string().optional().describe('Invoice date in YYYY-MM-DD format'),
due_date: z.string().optional().describe('Payment due date'),
vendor: vendorSchema.optional().describe('Vendor information'),
subtotal: z.number().optional().describe('Subtotal before tax'),
tax: z.number().optional().describe('Tax amount'),
payment_terms: z.string().optional().describe('e.g., Net 30'),
});
```
```python Python (Pydantic) theme={null}
from pydantic import BaseModel, Field
class Vendor(BaseModel):
name: str = Field(description="Company name")
address: str | None = Field(default=None, description="Full address")
phone: str | None = Field(default=None, description="Phone number")
email: str | None = Field(default=None, description="Email address")
tax_id: str | None = Field(default=None, description="Tax ID or VAT number")
class LineItem(BaseModel):
description: str = Field(description="Item description")
quantity: int = Field(description="Number of items")
unit_price: float = Field(description="Price per unit")
amount: float = Field(description="Line total")
class Invoice(BaseModel):
# REQUIRED - Core financial fields
invoice_number: str = Field(description="The invoice number")
total: float = Field(description="Total amount due including tax")
currency: str = Field(description="3-letter currency code (USD, EUR, etc.)")
line_items: list[LineItem] = Field(description="List of line items")
# OPTIONAL - May not be on all invoices
date: str | None = Field(default=None, description="Invoice date in YYYY-MM-DD format")
due_date: str | None = Field(default=None, description="Payment due date")
vendor: Vendor | None = Field(default=None, description="Vendor information")
subtotal: float | None = Field(default=None, description="Subtotal before tax")
tax: float | None = Field(default=None, description="Tax amount")
payment_terms: str | None = Field(default=None, description="e.g., Net 30")
```
```json JSON Schema theme={null}
{
"type": "object",
"properties": {
"invoice_number": {
"type": "string",
"description": "The invoice number"
},
"date": {
"type": "string",
"description": "Invoice date in YYYY-MM-DD format"
},
"due_date": {
"type": "string",
"description": "Payment due date"
},
"vendor": {
"type": "object",
"description": "Vendor information",
"properties": {
"name": {"type": "string", "description": "Company name"},
"address": {"type": "string", "description": "Full address"},
"phone": {"type": "string", "description": "Phone number"},
"email": {"type": "string", "description": "Email address"},
"tax_id": {"type": "string", "description": "Tax ID or VAT number"}
}
},
"line_items": {
"type": "array",
"description": "List of line items",
"items": {
"type": "object",
"properties": {
"description": {"type": "string"},
"quantity": {"type": "integer"},
"unit_price": {"type": "number"},
"amount": {"type": "number"}
}
}
},
"subtotal": {"type": "number", "description": "Subtotal before tax"},
"tax": {"type": "number", "description": "Tax amount"},
"total": {"type": "number", "description": "Total amount due including tax"},
"currency": {"type": "string", "description": "3-letter currency code"},
"payment_terms": {"type": "string", "description": "e.g., Net 30"}
},
"required": ["invoice_number", "total", "currency", "line_items"]
}
```
## Best Practices
Always add `description` fields. They help the AI understand what to look for and where.
"Invoice date in YYYY-MM-DD format" is better than just "date".
Fields missing in >20% of documents should be optional to avoid costly fallbacks.
Use `number` for amounts, `integer` for counts, `string` for text.
### Do's and Don'ts
```json theme={null}
{
"total_amount": {
"type": "number",
"description": "The final total amount due, including tax"
},
"invoice_date": {
"type": "string",
"description": "The date the invoice was issued, preserve original format"
}
}
```
```json theme={null}
{
"total": {
"type": "string"
},
"date": {
"type": "string"
}
}
```
* Missing descriptions
* `total` should be `number`, not `string`
* Vague field names without context
## The `_meta` Field
Parsefy automatically injects a `_meta` field into every extraction response with field-level confidence:
```json theme={null}
{
"invoice_number": "INV-2024-001",
"total": 1500.00,
"_meta": {
"confidence_score": 0.95,
"field_confidence": [
{ "field": "$.invoice_number", "score": 0.98, "reason": "Exact match", "page": 1, "text": "INV-2024-001" },
{ "field": "$.total", "score": 0.92, "reason": "Exact match", "page": 1, "text": "Total: $1,500.00" }
],
"issues": []
}
}
```
You don't need to include `_meta` in your schema; it's added automatically.
## Next Steps
Learn how to use custom rules to improve accuracy
Understanding the confidence scoring system