Stock Market Data Pipeline
End-to-end pipeline on Apache Airflow + Docker to ingest, process & store daily stock data. Dockerized Spark transforms, stored in MinIO (S3-compatible) & PostgreSQL, visualized in Metabase.
I independently own and scale data platforms in fast-paced, lean environments — leading critical migrations and architectural shifts that cut query latency by 90%+ and infrastructure cost by 35–40%. I specialize in cost-efficient, reliable pipelines and real-time systems.
I'm a Data Engineer who likes the hard, unglamorous problems: migrations that can't drop a row, warehouses that need to answer in seconds instead of minutes, and cloud bills that need to come down without anyone noticing a regression.
Moved a cloud-native lakehouse to ClickHouse and 300GB from DocumentDB to Aurora Postgres — structured, zero-downtime, no cost increase.
Consolidated databases and re-architected the medallion lakehouse to drive 35–40% cost reductions across production and lower environments.
Built real-time monitoring dashboards with automated alerting for proactive issue detection and higher data reliability.
Healthcare technology company focused on digital solutions to improve patient engagement and management.
A tech startup focused on enhancing retail & shopping experiences through data-driven insights.
A leading research university specializing in innovation and advanced technology.
End-to-end pipeline on Apache Airflow + Docker to ingest, process & store daily stock data. Dockerized Spark transforms, stored in MinIO (S3-compatible) & PostgreSQL, visualized in Metabase.
AI design system using Neural Style Transfer & GANs to blend artistic styles into unique patterns — reducing iteration time from hours to seconds. Secured 2nd place at NUS for innovation.
Shiv Nadar University · India
Open to data engineering roles & interesting platform problems. The fastest way to reach me is email.