Receipt-scanning expense apps (Expensify, Zoho Expense, modern fintech wallets) have become routine for business travelers and freelancers. The interview tests whether you understand the OCR pipeline, the categorization layer, and integration with accounting systems.
Functional requirements
- Capture a receipt with the camera
- Extract amount, vendor, date, line items via OCR
- Categorize for expense reports
- Sync to accounting system (QuickBooks, NetSuite)
- Submit for reimbursement
- Mileage tracking via GPS
Architecture
Three pipelines: capture, extract, sync.
Camera capture
Quality matters for OCR accuracy:
- Auto-detect receipt edges and crop
- De-skew and de-warp
- Enhance contrast for faded receipts
- Capture as high-res JPEG (1080p or 4K)
Platform APIs: VisionKit on iOS (DataScannerViewController), ML Kit on Android. Both have receipt-detection models.
OCR
Two strategies:
- On-device: faster, private, works offline. Modern phones have on-device text recognition.
- Cloud: better accuracy for tricky receipts. Send the image to a service like Veryfi, AWS Textract, or Google Document AI.
Most production apps use cloud for accuracy with on-device for instant preview.
Field extraction
OCR returns text blocks. Field extraction parses semantic fields:
- Total amount (look for “Total”, “Amount Due”, currency symbols)
- Vendor name (typically the largest text at the top)
- Date (multiple formats; locale-dependent)
- Line items (table-like structure)
- Tax (separate line, often labeled)
Modern systems use LLMs (cloud-hosted) for robust extraction across receipt formats.
Categorization
Map vendor → category:
- Lookup table for common vendors (Starbucks → “Meals”)
- ML categorization for unknown vendors based on keywords and patterns
- User correction trains the model over time
Mileage tracking
For driving expense claims:
- Background location tracking when user starts a trip
- Auto-detect start/stop based on movement patterns
- Calculate distance from GPS path
- Apply IRS or company mileage rate
Battery: the most expensive part of these apps. Be conservative.
Accounting integration
Push expenses to:
- QuickBooks, Xero, NetSuite, FreshBooks (most common)
- Sync via OAuth + REST API
- Map your category to their chart of accounts
- Handle approval workflows
Submitting for reimbursement
For corporate reimbursement:
- Group expenses into a report
- Submit for manager approval
- Approval triggers reimbursement (ACH, payroll, expense card)
Frequently Asked Questions
How accurate is mobile OCR for receipts?
For typical English-language receipts: 95%+ on amount and date. Lower on line items, especially handwritten or thermal-faded receipts.
How is fraud prevented?
Image fingerprinting (same receipt submitted twice triggers flag). Velocity rules. Manager approval as a gate. None of these are perfect.
Why is mileage tracking battery-intensive?
Continuous GPS during the drive. Modern apps use motion-detection to auto-start, then high-accuracy GPS until the user stops moving.