Work Experience AUBURN DIGITAL SOLUTIONS (Data engineer) (Dec 2020-Present) Working as Data Engineer. Design and implement highly scalable and available Big Data solutions using AWS services including S3, Redshift, Athena, Glue, and Lambda. - Develop and deploy ETL pipelines using pyspark, and other Big Data tools to extract, transform and load data into the data Warehouse.
➢ Ingest data from diverse sources into Amazon S3 and subsequently process it for use in analytics and reporting.
➢ Design, develop, and maintain ETL pipelines using AWS Glue for data extraction, transformation, and loading.
➢ Manage and optimize Amazon Redshift data warehouse clusters, including schema design, performance tuning, and query optimization.
➢ Set up and configure CloudWatch alarms and monitoring for ETL jobs, data pipelines, and data warehouse performance.
➢ Identify and implement improvements to enhance data pipeline efficiency.
