Scalable Serverless Data Platform with Audio Integration

A cloud-native solution for processing massive CSV datasets with advanced audio playback and filtering features — built for speed, scalability, and zero-database overhead.

Project Overview

This project is a modern data processing platform designed to handle large-scale CSV files stored in AWS S3. It enables organizations to:

• Process and merge multiple CSV files dynamically

• Integrate and play corresponding audio files per record

• Filter data in real time by date, name, phone number, or ID

• Manage access through secure, role-based authentication

All of this is achieved using a fully serverless AWS backend and a modern, lightweight frontend architecture.

Complex CSV Processing without Databases

The system is built to process thousands of records per day, with each row in a CSV containing:

• Metadata: phone number, ID, timestamps, etc.

• S3 path to an associated audio file

• References to recording dates and tags

Instead of using traditional databases, the platform reads directly from S3, merges files dynamically, and delivers a unified data experience.

Advanced Filtering and Search

To maintain performance across large datasets, the platform uses:

Date-Based Prefix Filtering

Filters S3 objects using YYYYMMDD format prefixes (e.g., 20250225), allowing quick lookup of files by date.

Real-Time Search

Users can search across all files for specific names, phone numbers, or time ranges — with consistently fast results even across large volumes of data.

Cloud-Native Backend Architecture

Built entirely on AWS using a serverless design for scalability and cost-efficiency:

Python Lambda Functions

Handles all data processing logic, including CSV parsing, merging, and audio URL generation. Lambda memory and time limits are optimized for high throughput.

API Gateway

Provides REST endpoints for frontend-to-backend communication with rate limiting, error handling, and structured responses.

AWS Cognito Authentication

Implements secure authentication with multi-role support:

• Admins: full access

• Playback users: stream audio, view data

• View-only users: read access

Integrated Audio Playback

Each CSV row includes a reference to a stored audio file. Audio is playable directly from the platform using a built-in player with:

• Instant streaming from S3

• Inline playback controls in each data row

• Zero context switching — data and media in the same interface

Frontend & Deployment

React + Vite + Tailwind CSS

A modern frontend built with:

• Fast loading using Vite

• Responsive UI with Tailwind CSS

• Real-time interactivity and seamless UX

AWS S3 + CloudFront

Deployed as a static site on AWS S3 with global distribution via CloudFront:

• High performance worldwide • TLS/SSL encryption by default • Scales instantly to handle traffic spikes

Data Visualization & UX

The platform uses custom data tables designed for usability at scale:

• Pagination, sorting, and virtual scrolling

• Real-time searching across thousands of rows

• Seamless integration of audio with each record

Role-Based Access Control

The permission model supports multiple user types with precise control:

Full Access Users

Can view all data, download records, and stream audio

Playback Users

Can view and stream audio but cannot download

View-Only Users

Can view the data without any interaction capabilities

This structure ensures security, privacy, and operational efficiency across teams.

Performance & Cost Optimization

• Resolved early bottlenecks by optimizing Lambda memory and execution strategy

• Replaced traditional databases with direct S3-based architecture

• Reduced infrastructure costs while improving scalability

Key Innovations

• Fully serverless backend architecture

• Prefix-based S3 filtering for time-based data queries

• Real-time search without traditional indexing

• Seamless integration of media into tabular data views

• Multi-role access control using AWS Cognito

• High-performance static frontend deployment

Business Impact

This platform enables organizations to manage large volumes of structured and unstructured data (audio + metadata) without relying on costly infrastructure. By combining performance, scalability, and usability, the system unlocks new possibilities in operational data handling and reporting.