Build a Voice-Powered Expense Reporting System

Off-the-shelf apps like Expensify offer voice entry, but it is a closed system. It requires users to open the app and follow a specific prompt format. If a project manager wants to record a quick memo like "bought 15 2x4s at Home Depot for 123.50 for the Elm Street project," Expensify's rigid structure often fails or requires manual correction. It also cannot handle custom fields like a "Project ID" without a higher-tier plan.

The alternative is a manual process where an admin listens to voice memos and enters them into accounting software. For a team of 10 sales reps each submitting 20 expenses a month, that is 200 reports. If each manual entry takes 3 minutes, an admin spends 10 hours a month just on data entry. Transcription errors, like hearing "fifty" as "fifteen," cause accounting headaches.

A technical team might try to connect Google Drive to OpenAI's Whisper API via a tool like Make.com. This works for transcription, but then they hit a wall. Extracting structured data like `{"vendor": "Home Depot", "amount": 123.50}` requires another LLM call with complex prompting and validation logic. This approach is brittle and expensive in a per-operation model, and it frequently breaks on edge cases.

Syntora's approach to building a custom voice AI for expense report processing would begin with a discovery phase to understand specific client requirements for data fields, integration points, and anticipated usage patterns.

The core technical architecture would involve setting up a dedicated endpoint where users could securely upload audio files (such as m4a, mp3, wav) from their devices. These files would be stored temporarily on AWS S3. An AWS Lambda function would be configured to trigger upon new uploads, initiating a call to an audio processing API for transcription to text. Optimizing this transcription step for financial terminology would be a key focus to ensure accuracy. Syntora utilizes Python with the Boto3 library to manage interactions between S3 and Lambda.

The transcribed text would then be sent to the Claude API. We would craft an engineered prompt designed to instruct the model to act as an accountant, focusing on extracting specific fields: vendor, total amount, transaction date, and expense category. We have built document processing pipelines using Claude API for financial documents, and a similar pattern applies here for accurate expense data extraction. Pydantic would be implemented for robust data validation, ensuring, for example, that the amount is always a float and the date conforms to ISO 8601 format. Entries failing validation would be flagged for manual review, allowing for human oversight.

Once the data is structured and validated, a Python script, potentially using the httpx library, would post this data to the client's accounting software API, such as QuickBooks or Xero. Authentication would be handled through secure secrets management within AWS. The system would create a new expense entry, attach the original transcription as a note for auditability, and archive the audio file. For full transparency and auditability, every step of the process would be logged using structlog to a Supabase database table, providing a comprehensive audit trail.

The proposed system would be deployed as a serverless application utilizing FastAPI and AWS Lambda. This architecture avoids server management overhead and typically results in low operational costs, often under $30 per month for standard volumes. Monitoring would be established using CloudWatch alarms, configured to send notifications (e.g., via Slack) if API error rates exceed a defined threshold or if processing latency for any single file surpasses a specified duration. This proactive monitoring would ensure early detection of any operational issues.