Data Engineer · Bengaluru, India
I build the data systems companies depend on — real-time streaming pipelines, financial data platforms, cloud lakehouses, and AI-assisted engineering workflows — at production scale, with the reliability that holds up under pressure.
About Me
I'm a Data Engineer based in Bengaluru with a track record of building production-grade data infrastructure across four companies — from early-stage startups to global e-commerce platforms. My work spans real-time streaming, financial data systems, cloud architecture, and increasingly AI-assisted engineering.
My technical range covers the full pipeline lifecycle: real-time ingestion with Kafka and SQS, large-scale processing with Databricks, Spark, and PySpark, transformation with dbt and SQL, cloud infrastructure with Terraform and AWS, and analytics delivery with Tableau and Looker. I've operated these systems in production — on-call, at scale, with real accountability.
What makes my profile stand out is deep financial domain expertise: at StockX I design subledger-to-GL pipelines, reconciliation frameworks, and audit-ready data flows that Accounting and Tax teams rely on for global month-end close. I speak the language of both engineers and finance stakeholders, which is rare.
I also integrate AI tooling — Claude Code, prompt engineering, LLM-driven workflows — into daily engineering practice. Not as a novelty, but as a genuine productivity multiplier that compresses delivery cycles without sacrificing quality.
Career Timeline
StockX — Global Marketplace
Data Engineer · Sep 2024 – Present · Financial Data
Zuma — PropTech
Senior Data Engineer · May 2023 – Sep 2024 · Analytics Platform
UNA Brands — E-commerce Roll-up
Senior Data Engineer · Dec 2021 – May 2023 · Streaming + Platform
Sopra Steria — IT Services (Airbus)
Software Engineer – Data · Aug 2019 – Nov 2021 · Telemetry Pipelines
AI in my workflow
I use Claude Code daily — for SQL optimization, root-cause analysis, code generation, and technical documentation. It's become a genuine force multiplier in how fast I can ship without cutting corners on quality.
Technical Skills
Every tool here has been used in production. No tutorial projects.
Languages
Big Data & Data Platforms
Cloud & Infrastructure
Databases & Storage
Analytics & BI
DevOps, CI/CD & Observability
AI-Assisted Engineering
Work History
Four companies, four distinct data challenges — all production, all at scale.
Data Engineer
StockX · Bengaluru
Building the financial data nervous system for one of the world's largest sneaker and collectibles marketplaces — every order, payment, fee, and tax event flows through pipelines I own into audit-ready ledger entries used for global financial close.
Senior Data Engineer
Zuma · Bengaluru
Took ownership of the analytics platform at a fast-moving PropTech startup — stabilised unreliable pipelines, migrated the entire BI layer, and built a data service the engineering team still depends on.
Senior Data Engineer
UNA Brands · Bengaluru
High-throughput pipelines for an e-commerce roll-up managing dozens of brands — real-time Kafka streaming at 50M+ records/day, a Kubernetes-native data platform, and engineering team leadership across a 5-person group.
Software Engineer — Data Platforms
Sopra Steria · Noida
Started my data engineering career building real-time Spark pipelines for Airbus — aircraft engine telemetry with hard latency requirements and zero tolerance for data loss.
Core Expertise
The six areas where I can walk into a codebase on day one and make a real difference.
Kafka to Databricks Spark Structured Streaming — high-throughput, low-latency event processing with 99.9% availability. Built and operated at 50M+ records/day in production, not just in theory.
Deep experience building subledger-to-GL pipelines, account reconciliation frameworks, and SOX-compliant datasets. I understand accounting concepts and the engineering required to make financial data reliable and audit-ready at global scale.
Full IaC ownership on AWS using Terraform — Glue, EMR, ECS/Fargate, Lambda, S3, Redshift. I own the architecture decisions, cost tradeoffs, and deployment governance — not just the resources themselves.
dbt-based transformation layers, dimensional models, CDC pipelines, and SQL that performs under pressure. I write data models that other engineers can maintain and business users can trust — not just ones that pass tests on the first run.
Automated DQ frameworks, reconciliation workflows, and monitoring stacks with CloudWatch, Grafana, Datadog, and PagerDuty. My pipelines don't just run — they surface issues before stakeholders notice something is wrong.
I embed Claude Code and LLM-based workflows into my engineering practice — SQL optimisation, intelligent root-cause analysis, automated documentation, and code generation. AI amplifies the engineering judgement; it doesn't replace it.
Get In Touch
Open to data engineering consulting, freelance pipeline projects, and full-time roles at companies building serious data infrastructure. I respond within 24 hours.
mayur.sonawala13@gmail.com