Mihira Lolla
Development
Ohio, United States
Skills
Data Engineering
About
Mihira Lolla's skills align with System Developers and Analysts (Information and Communication Technology). Mihira also has skills associated with Programmers (Information and Communication Technology). Mihira Lolla has 10 years of work experience.
View more
Work Experience
Senior Data Engineer
Cigna
August 2023 - Present
- Project: FHIR (Fast Healthcare Interoperability Resources) This project deals with processing the healthcare data by making use of HAPI FHIR (Fast Healthcare Interoperability Resources) API which is a standard for exchanging healthcare data. * Designed and implemented an ETL process, optimizing data processing and reducing unnecessary computations for improved performance. * Utilized the HAPI Java API to extract the Admit, Discharge, Transfer data into S3 using Spark scala. * Involved in Planning, Design, Architecture, Development, and Delivery activities of the project. * Key player in bringing performance optimization to ensure better client-focused delivery. * Configured S3 buckets with various lifecycle policies based on business requirement. * Extract, interpret and analyze data to identify key metrics and transform original data into useful information. * Involved in converting hive queries into Spark transformations and loaded the data to deltalake * Utilized Kafka, Spark and data bricks DeltaLake capabilities to efficiently handle incremental data updates, resulting in reduced processing times and improved overall system performance.
Senior Data Engineer
Finning Inc
June 2021 - April 2022
- Project: Condition Monitoring This project deals with data pipelines which load data using PySpark in Azure systems using Azure databricks, SSIS, Azure Blob Storage, Azure Data Factory. * Developed efficient data pipelines to extract, ingest, and clean large sets of data to enable quicker analysis. * Experience in developing data driven applications using Spark, with Spark-SQL in Databricks for data extraction, transformation, and aggregation from multiple formats of data. * Worked on developing and setting up DevOps Pipelines and workflows in Azure Data Factory. * Contributed to optimization of the data pipelines and data models. * Experience in orchestrating the automation of the pipelines with Azure Data Factory
AWS Data Engineer
Zillow Group
July 2020 - June 2021
- Project: Cost Attribution, Customer Journey Analytics * Developed automated processes to ingest, and clean large sets of data to enable quicker analysis. * Automated pipelines to extract, transform and optimize the ingestion of millions of rows of data per day to third party API for Customer Journey Analytics. * Collaborated with Data Scientists and various Stakeholders as a single point of contact in the process of designing a seam less Data Pipeline with Rate Limitations and controlled throughput using AWS, Spark, Airflow to ingest data to a third-party API "Amplitude" * Worked on improving efficiency of pipelines and reduce cost on AWS * Created pipelines with Airflow, Pyspark to extract data from snowflake to perform analytical reporting on tableau * Reduced the cost of storage and compute by 17% through optimization of data stored on S3 and EC2
Senior Data Engineer
Cargill
January 2019 - July 2020
- Project: Dairy Enteligen This project deals with the Data modeling and transformation of data from Cargill Dairy Farms into intelligent reports with Key Performance Indicators that highly helped boost their business. Responsibilities: * Developed data pipelines to fetch Dairy data from Java REST web services into Hive and transform it as per business needs using SCD Type-2 methodology using Spark-scala and write it to Kudu tables for Data visualization. * Designed and built pipelines to automate ingestion of structured and semi-structured data. * Created multiple tables on Hive and Kudu with Cloudera Hue platform. * Created a generic schema flattening reusable script in Spark scala to handle nested JSON objects. * Created Spark jobs that handled data for periodic Incremental Loads. * Performed performance tuning on the data pipeline processes. * Implemented complex logic using Spark and HQL based on the corresponding business requirements. * Performed migration compatible setup in maven and Spark-scala for Cloudera upgrade from 5.9 to 6.3.3 * Created complex Hive scripts to load the report data with different aggregated calculations, joining and filtering transformations to display the data as per business requirements. * Worked as a technical lead to a team of junior developers and trained, delegated the tasks, and monitored them.
Data Engineer
AbbVie
July 2018 - December 2018
- Project 1: POMSNet / Maximo This project deals with the supply chain, manufacturing, inventory, and shipment data of the AbbVie pharmaceuticals. Responsibilities: * Worked in a fast-paced agile development environment to quickly analyze, develop, and test potential use cases for business users. * Developed a reusable framework in Python to perform data load operation for multiple tables using Sqoop. * Performed transformations on the data based on the Business requirements. * Created Spark jobs to load the data with optimization. * Developed automated scripts for data comparison of source and target which has saved hours of time of Support teams. * Developed scalable, reliable services and pipelines using EMR, Kafka, Spark, Hive Project 2: DIA (Proof of Concept) * Developed a Kafka project to stream the data from a data source into various Kafka topics using Spark streaming. * Loaded tables with AWS Athena sourced from S3 buckets. * Built the schemas required for loading data using AWS Glue.
Data Engineer
Walmart
January 2017 - June 2018
- Project 1: Feature Space Sales The project is designed to predict the items that need to be placed in the Featured Space at department level in Walmart Stores across USA Responsibilities: * Developed recommendation functionality to display Featured and non-featured items in the aisles. * Created Hive scripts to obtain data from various tables and filter them as per the required business logic. * Created Spark jobs to load data from various sources like Oracle, MySQL and forecast the Feature Space Recommendations. * Implemented Spark/Hive Scripts from the business model to perform Recommendation function. Project 2: Multi level Loyalty The objective of the project is to calculate the loyalty of the product at department level. Responsibilities: * Involved in creating Hive scripts to create tables. * Developed the transformations needed to calculate loyalty of the products using Spark - Scala. * Involved in converting Hive queries into Spark SQL with Scala.
Data Engineer / Informatica Developer
Woolworths Ltd
April 2014 - December 2016
- Project: Hadoop Migration The project aims to migrate the sales data from Teradata to HDFS. Responsibilities: * Involved in importing data from Teradata to HDFS using Sqoop. * Migrate the existing data from Teradata to Hadoop and perform ETL operations on it. * Created Hive and Pig scripts to load data and create reports depending on business requirements. * Worked as a developer to create mappings using Informatica PowerCenter for fulfilling various business requirements. * Worked on improving the performance of the PL/SQL projects.
JAVA DEVELOPER
Woolworths Ltd
February 2013 - April 2014
- Responsibilities: * Worked on the development of Java Applications for a supply chain management engine that helps forecast sales. * Worked on migration of legacy WebSphere projects to JBoss.
Education
