Data Engineer | Software Developer | Cloud Architect
With 3+ years of hands-on experience building and optimizing scalable, high-performance,
cloud-based applications, I specialize in designing and deploying data pipelines and
distributed systems with a strong focus on real-time data ingestion and analytics.
I’m driven by curiosity and a commitment to continuous improvement, always eager to
deepen my expertise in data engineering and explore new technologies.
I've worked on a few individual and a few team projects over the course of 5 years. Here are a
few!
ETL Pipelines on AWS for Realtime Data Processing
- Architected and implemented a scalable data ingestion and IoT device management system capable of handling 500 messages per second
- Worked with cross functional teams and Subject Matter Experts(SMEs) to design and architect a new data aggregation service using Kotlin, Spring, Kafka Streams and Snowflake that addressed the current pain points and came up with an elegant solution and worked on implementing the same independently.
- Designed architecture for product features like message storage and aggregation, creating API contracts, cache implementation.
- Worked as part of Deployment team and wrote cloud-formation templates (IaC).
- Designed, implemented, and integrated various APIs, micro-services, and interfaces to ingest data from multiple systems.
- Worked collaboratively with DevOps, data scientists, and other internal partners to turn requirements and design documents into scalable, elegant production software.
Emergency Vehicle Pre-emption/Transit Signal Priority (EVP/TSP)
- Deployed and maintained code in 90,000 intersections across North America, all serving preemption/priority to Emergency/Transit vehicles.
- Handled the migration of four families of devices(210x, MP70, Cradlepoint and CEI) for both EVP and TSP from Neptune to DynamoDB while also building in a Redis cache, and a cache invalidation mechanism for all the devices using SNS, SQS and AWS Lambda.
- Introduced a caching mechanism, slashing Lambda response times by over 95% from 1100ms to 50ms using Redis.
- Recognized the need for a centralised Feature Toggle Service and spearheaded its development, enabling seamless feature management across teams and eliminating the need for redundant application stacks or hardcoding feature flags, enabling dynamic feature management and reducing development time for agencies.
- Simultaneously led the development of EVP and TSP solutions, enhancing traffic management and reducing response times for emergency vehicles and public transit in major metropolitan areas across North America.
- Streamlined existing project and improved product performance. Implemented data validation checks at front end and back end layers using Pydantic.
- Mentored junior engineers on the team and created a step by step documentation for new engineers who are joining the team to assist in their onboarding and initial development setup and questions.
Multi-Threaded Client Server Project
• Designed and built a distributed file system using Sockets and Threads in Java
• Implemented multi-threading to handle multiple client requests at the same time
• Enabled Server Directory sync and de-sync on client request and maintained folder
consistency between server and client
• Used Java Swings to build the front end
API based Video Sharing Platform on GCP
• Designed and built multiple API's that interact with Cloud MySQL and Google Cloud Storage and hosted them on Google Functions
• Used Google Kubernetes Engine to host a website built with Python-Flask where users can upload and view videos
• Built interactions between the front end and the backend using RESTful APIs and JSON.
• Created Container images, wrote the YAML files, and deployed them to Kubernetes clusters on the GKE. These were then exposed using a service to the Public Internet.
AWS Project
• Built and hosted a website built from scratch with HTML, CSS and PHP on an AWS EC2
instance
• Implemented load balancing and auto-scaling groups using a saved AMI, triggered
based on network usage
• Used MySQL based Relational DB hosted on IBM Cloud and AWS for data storage.
• Used Chart.js to display charts and other graphic info on the webpages.
Ford iOS Application
• Worked with Core and third-party Swift libraries [Alamofire (for REST API calls),
Google Maps SDK (Geocode and Reverse Geocode)] with Cocoapods as the dependency
manager.
• Developed the application in an Agile environment using MVC design pattern.
• Developed Unit Test cases.
Serve-Me System
• Developed a native Android application using Firebase as the database.
• Designed the User Interface and implemented Core functionality of the
application.
• Responsible for developing 30% of all screens and functionality of the
application.
UTA Vending System
• Built an Android application which uses shows locations of vending machines around
campus
• Designed and used SQLite DB for data storage.
• Worked with Singleton classes and Recycler Views.
NotesHub
• Designed a Web Application using PHP, HTML, MySQL and JavaScript that allows
students of a class to upload pictures of a class’s notes, which can be viewed by
all the students of the same class.
• Built the website to be responsive using CSS and allowed for continuous downward
scrolling to view each picture as an article.
Data Analysis and MIS Report Generation
• Created Analytical and MIS reports using HTML5, DataTables, PHP and JavaScript
which used the incoming real-time data from each car and mobile device to create
useful insights.
• Used Google Charts API to display data in the form of attractive and efficient
charts so that clients and end-users can understand and interpret the incoming data
faster.