Vishal Vishal
Development
Dublin, Ireland
Skills
Database Management
About
VISHAL VISHAL's skills align with System Developers and Analysts (Information and Communication Technology). VISHAL also has skills associated with Database Specialists (Information and Communication Technology). VISHAL VISHAL appears to be a low-to-mid level candidate, with 4 years of experience.
View more
Work Experience
Data Engineer
Celestial V Solution Bangaluru
February 2021 - August 2023
- India * Developed and implemented data migration strategies to transfer transactional data from on-premises systems to AWS S3 using Apache NiFi. * Ensured compliance with data privacy regulations such as GDPR and HIPAA. * Implemented data governance policies to maintain data integrity, quality, and security. * Utilized AWS Glue to extract data from various sources, including GCP APIs, and load it into AWS S3. * Established a volume-based data ingestion process to move data into Databricks for further processing. * Conducted performance tuning of Spark jobs to improve processing efficiency and reduce runtime. * Developed a testing framework that ensured data quality and consistency through comprehensive data validation checks. * Implemented CI/CD pipelines in AWS using tools such as Jenkins, AWS Code Pipeline, and GitHub Actions to automate the deployment and integration of data processing jobs. * Utilized Apache Airflow to orchestrate and manage complex data workflows, ensuring efficient and timely execution of ETL processes.
Data Engineer
SBC Tech
August 2019 - February 2021
- * Developed python and PySpark job to load data from various sources like MongoDB, Elasticsearch and Azure blob into Azure Databricks and created common data model based on those data. * Created a robust data pipeline, extracting and processing data from MongoDB, Elastic Search, Azure Blob in AWS S3. * Designed and deployed an EMR Studio machine learning model, enabling seamless detection of fraudulent sites by leveraging processed data from MongoDB stored in AWS S3. * Implemented CI/CD pipelines for Databricks to automate the deployment and integration of data processing jobs, ensuring continuous delivery and integration. * Created static and interactive data visualization dashboards using Power BI for the sales, marketing, and data analyst teams, resulting in data-driven decision making. * Participated in data quality checks and troubleshooting to ensure data accuracy and consistency.
Internship
Peenak Business Solutions Pvt. Ltd
June 2019 - August 2019
- * Built an end-to-end solution by ingesting tables from an on-premise SQL Server database using Azure Data Factory. * Stored ingested data in Azure Data Lake. * Utilized Azure Databricks to transform RAW data into the cleanest form. * Loaded clean data into Azure Synapse Analytics. * Integrated Azure Synapse Analytics with Microsoft Power BI to build interactive dashboards. * Employed Azure Active Directory (AAD) and Azure Key Vault for monitoring and governance purposes. ACADMIC PROJECTS Next Word Prediction with LSTM and CNN Dublin Business School * Conducted research in natural language processing (NLP) and AI, focusing on deep learning techniques, including Recurrent Neural Networks (RNN), LSTM, and Convolutional Neural Networks (CNN). * Implemented and optimized machine learning algorithms for text sentiment analysis, achieving an accuracy rate of 90%. * Collaborated with cross-functional teams to integrate the prediction model into smart speakers and home assistants. * Implemented neural encoder-decoder models to map input text to output text effectively.
Education
Dublin Business School
Master of Science
