SENIORDATAENGINEER Jan. 2020 - May. 2021
• Developed data processing framework for daily pricing of 65 million customers in Production.
• Automated the existing Job flow and enabled logging using Control-M that reduced RCA turnaround time by 50%.
• Optimized the existing production jobs runtime from 3.5 to 2 hours.
• Build an HLD of data processing pipeline in Azure using ADF for loading delta data from multiple sources.
• Helped another team for 2 sprints to write Pyspark code and complete the developer tasks in time.
• Created Jenkins Job for auto-deployment of the framework in multiple environments that enabled faster deployments
