Worky | Meet our Talent - Anilkumar Patel

Skills

Data Engineering

About

Anilkumar Patel's skills align with System Developers and Analysts (Information and Communication Technology). Anilkumar also has skills associated with Programmers (Information and Communication Technology). Anilkumar Patel has 6 years of work experience.

Work Experience

Data Engineering Intern

Shamrock Foods Company

May 2023 - August 2023

Created and implemented robust ETL pipelines utilizing Azure Data Factory, transforming the data extraction process for Shamrock stores across different states; reduced data processing time by 90% and increased data accuracy by 25%. - Transformed legacy ETL systems into Databricks Delta Tables, resulting in an impressive 80% decrease in processing time and significant efficiency gains. - Consolidated data from Microsoft REST API and Azure Active Directory API by incorporating Databricks, Spark SQL, and Delta; reduced data errors by 20%, guaranteeing reliable and precise data for augmented analysis. - Spearheaded the implementation of DBT Cloud and Snowflake, resulting in a 75% reduction in ETL code development, streamlining data transformation processes, and improving overall efficiency by creating robust Data Modeling and Data Mart (Star Schema).

Senior Data Engineering

Priceline.com

May 2020 - August 2022

Analyzed and assessed data engineering team's codebase for compliance with coding standards, performance optimization on SQL Queries, and issue identification; provided actionable feedback resulting in a 30% decrease in system downtime and refined overall code quality. - Streamlined the establishment of a state-of-the-art GCP Data Lake-house, transforming data management practices and elevating data-driven decision-making capabilities. - Strategically optimized data workflows, enhanced data integrity, and fortified data security measures to ensure seamless ingress/egress patterns, data governance, cataloging, database design, Dimensional Modeling, Data mart and retention policies. - Headed Geospatial team, migrating data to Cloud SQL Postgres & Big Query for a 40x performance boost, ensuring compatibility with Cloud Composer Airflow on ArcGIS Pipeline. - Architected migration frameworks (on-premises to Big Query, Cloud SQL, Bigtable), and refined GCS storage, resulting in a $10,000 cost reduction. Migrated Onprem Java App/Airflow Dag's to the Cloud, devised a booking propensity model, and Co-ordinated to DevOps CI/CD pipelines.

scheduler, and employed Python/Pyspark

Mactores Cognition

December 2018 - May 2020

, Mumbai, India 12/2018 to 05/2020 - At Seagate Technology (17 Months): Initiated a data migration strategy, using Airflow as the scheduler, and employed Python/Pyspark for backend processing. This led to a significant cut in analytics processing time by 3 days. Leveraged AWS S3 Data Lake, Presto as the Query Engine for long-running queries, and Spark SQL for short-running queries during the successful migration of databases and factory files from on-premises infrastructure. - Godrej Consumer Products (8 Months): Engineered an end-to-end ETL solution, seamlessly transitioning On-Prem data to AWS infrastructure (S3, Aurora, RDS & Athena) and Established Data Mart for Tableau Business Reports via JDBC connector. This comprehensive solution not only purify data accessibility but also resulted in a remarkable 50% reduction in data ingestion time, enhancing overall efficiency. - Planned and executed strategic migration plans in Agile Methodology for Blackhawk Network Holdings, Veritas, and Katerra ensuring a smooth transition of critical data and systems, resulting in improved operational resilience and a 25% reduction in post-migration issues. Data Engineering

Formcept Technologies

July 2017 - December 2018

Bengaluru, India 07/2017 to 12/2018 - Initiated the development of real-time data pipelines utilizing Apache Kafka, with a primary focus on data ingestion and pre-processing, optimizing performance with AWS Cloud services. - Collaborated with front-end engineers, leveraging AWS Cloud services (Lambda, S3, Glue) for efficient data processing, achieving a 20% improvement in overall ingestion speed. - Enriched API infrastructure with AWS API Gateway, Elastic Load Balancer, and CloudFront, optimizing data accessibility by 80% in Data lake. Applied ELK and EFK stacks for log analysis, resulting in a 30% increase in actionable insights from log data. Data Engineering

GCP Professional Data Engineer

Infrasoft Pvt Ltd

June 2016 - July 2017

Mumbai, India 06/2016 to 07/2017 - Revised a Batch data pipeline using Airflow/Streamsets and Python and Apache Kafka, processing over 1 million events daily, resulting in a 30% reduction in processing time. - Engineered a solution reducing data ingestion latency by 40%, optimizing ETL processes for improved reporting accuracy and faster decision-making. - Designed and executed a star schema for a retail analytics platform, resulting in a 40% increase in query performance and managing ETL pipelines for 5+ TB of daily data from sources. Certifications PUBLICATIONS / Blogs - GCP Professional Data Engineer. - AWS Certified Solutions Architect - Associate. - AWS Certified Developer - Associate. - GCP Professional Machine Learning Engineer. - AWS Academy Cloud Architecting. - AWS - Optimizing Presto SQL on Amazon EMR to Deliver Faster Query Processing. - AWS - Lower TCO and Increase Query Performance by Running Hive on Spark in Amazon EMR. - AWS - Migrating from Oracle to Amazon Redshift. - Medium - Configuration Management and Continuous Deployment.

Anilkumar Patel