## Why AI Testing Is Different
AI applications are non-deterministic — the same input can produce different outputs. Traditional unit testing (exact match) doesn't work. Instead, we test for properties, patterns, and quality ranges.
### Testing Pyramid for AI
``` / E2E Tests \ (Full pipeline) / Integration \ (API + model) / Component Tests \ (Prompts, parsing) / Unit Tests \ (Functions, schemas) ```
### Types of AI Tests
- Deterministic tests: Validate schemas, parsing, tool call formats
- Property tests: Check outputs have required properties (length, format, tone)
- Comparison tests: Verify output quality vs. a baseline
- Regression tests: Ensure changes don't degrade known-good outputs
### Deterministic Testing
```typescript // Test that structured output matches schema test("product extraction returns valid JSON", async () => { const result = await extractProduct("iPhone 15 Pro, $999, 256GB"); expect(result).toHaveProperty("name"); expect(result).toHaveProperty("price"); expect(typeof result.price).toBe("number"); expect(result.price).toBeGreaterThan(0); }); ```
### Property-Based Testing
```typescript test("summary is shorter than input", async () => { const input = generateLongText(5000); const summary = await summarize(input); expect(summary.length).toBeLessThan(input.length * 0.3); expect(summary.length).toBeGreaterThan(50); }); ```