Karre Bhavanisankar
Development
Tokyo, Japan
Skills
Data Engineering
About
KARRE BHAVANI SANKAR's skills align with System Developers and Analysts (Information and Communication Technology). KARRE also has skills associated with Database Specialists (Information and Communication Technology). KARRE BHAVANI SANKAR appears to be a low-to-mid level candidate, with 4 years of experience.
View more
Work Experience
Data Engineer
October 2023 - March 2024
- Diverse public data analytics • Formulated data-driven solutions for client-specific requirements, addressing business challenges. • Load structured and unstructured data onto DLI, creating Spark Dataframes and performing transformations. • Create UDF for complex analytical operations, customized functions, and optimized Spark Jobs on DLI. • Load ES Indices data using Python API, creating pandas dataframes with comprehensive processing. • Create Kibana visualizations and dashboards, capturing patterns and trends to meet business requirements. • Build & deploy Docker images via SWR, activating CCE containers and automate CI/CD pipelines using Jenkins and GitLab integration. • Establish and maintain a comprehensive wiki page for each project, ensuring continuous updates with the latest information. Senior Data Engineer Hays Specialist Recruitment Japan K.K. 01/2019 ՞ 09/2023 Tokyo, Japan Telemetry Data analytics • Analyze business problems, ensure data quality, and propose data- driven solutions through standalone meetings with clients on-site. • Configure AWS Eventbridge CRON settings for Lambda-trigger, EMR cluster creation, managing job execution and termination. • Create Spark dataframe on 250֡ TB data from S3 and various db sources. • Perform Spark transformations and actions for in-depth analysis of extensive datasets. • Create UDF for intricate analytical tasks, refine and optimize Spark Jobs, and significantly reduce job turnaround time, resulting in a 20% monthly cost reduction. • Collaborate with the Data Team on a data migration project, overseeing the seamless transfer of data from MsSQL to the Redshift data warehouse. • Perform data quality checks and provide comprehensive reports to business units for diverse business solutions.
Data Engineer
Indo-Sakura Software Japan
December 2016 - December 2018
- Managed cross-domain projects for diverse clients • Understand the Business problem by driving the standalone meetings with Customers on their site. • Propose the best solution to make the difference and add value using big data technologies. • Provide daily updates on task progress to the customer and uphold transparent communication throughout the project. • Leverage AWS cloud and essential cloud services tailored to software and hardware requirements in alignment with business agreements. • Collect and ingest IoT devices data and create Datapipeline ELT/ETL jobs using Hadoop and Spark Frameworks. • Establish an analytics base platform and discern and communicate data requirements to the customer, tailored to specific business use cases. • Automate pipeline jobs for analytical business operations using Pig Latin scripts and Hive QL. • Provide Knowledge Transfer ՄKTՅ to customer data science teams and collaborate with them to create new features for feeding ML models. • Establish an e-Confluence page, incorporating high-level information for each project and ensuring the continuous update of the latest details.
Software Engineer
Mobi Cloud Infosystems Pvt Ltd
May 2015 - November 2016
- Intern: Big Data R&D Engineer and ML Specialist • Explore latest trends in big data and AI algorithms for research. • Collect real time tweets using spark streaming with Kafka. • Identify trends, patterns, and insights within Twitter data through the application of diverse complex analytical operations to derive business conclusions. • Use NLP techniques to transform, tokenize, and analyze each word, computing TF՞IDF to develop tweet sentiment analysis. • Create Classification and regression ML/DL models on available public datasets. • Build a recommendation engine using the ALS matrix factorization model to suggest top N shows for each user. • Create ready to use data sets and share Data Marketplace platform that facilitates offering data to consumers, enabling them to research, sample, compare, and purchase the third-party data.
Education
