Chico Qi
Development
Auckland, New Zealand
Skills
Data Engineering
About
Chico Qi's skills align with Programmers (Information and Communication Technology). Chico also has skills associated with System Developers and Analysts (Information and Communication Technology). Chico Qi has 21 years of work experience, with 8 years of management experience, including a low-level position.
View more
Work Experience
Lead Data Engineer
Westpac
January 2023 - March 2024
- Upgrading Airflow to Docker/K8S environment with DockerOperator/ KubernetesPodOperator • Building up a data mesh product based on SpringBoot, Kafka and Postgresql in AWS to provide API for endusers. • Deploying data product into AWS using Jenkins/CloudBees, Terraform, Helm, Prometheus and Grafana.
Senior Data Engineer
Xero
November 2021 - December 2022
- • Working on the user behavioural event platform • Completed a few CI/CD pipelines including Python testing, TeamCity, Terraform, K8S, AWS, Docker, AWS CloudWatch, Sumo Logic and New Relic
Data Scientist
John Snow Labs
April 2021 - September 2021
- Developing Scala/Python programs for Spark-OCR project including model finetuning, Scala coding, Python coding, preparing demo notebooks, completing Jenkins CI/CD pipeline. • After fine-tuning LayoutLM pre-trained model with SROIE dataset, converted that PyTorch model to TensorFlow model by ONNX, developed VisualDocumentNER based on Spark MLlib, implemented Python wrapper and demo Jupyter notebook • Fine-tuning Signature and Handwriting Detection model based on mmdetection (Cascade RCNN) and YOLOv5 with www.gsa.gov dataset and implemented this function in Spark-OCR following the same development process as above • Completed two Handwritten Text Recognition models (CRNN and SPAN) including fine-tuning and Spark implementations • Have done a table extractor from PPT/PPTX and a pdf convertor based on Apache POI • Resolving Maven/SBT dependency conflicts within Spark-OCR project
Data Engineer
Westpac
December 2020 - April 2021
- Working on Spark/Hadoop/Airflow platform
Big Data Engineer
Vodafone New Zealand
September 2018 - November 2020
- Completed the population movement project with Stats NZ using AWS Glue Scala APIs • Managed the on-premise Hadoop cluster and implemented pipelines to collect a variety of source data using NiFi, Hive and Spark • Migrated the on-premise platform to AWS and Snowflake and then rebuilt new pipelines • Set up RapidMiner in AWS using CloudFormation, ECR, EC2, ASG, ELB etc and then started training and serving machine learning models
Consultant
Enterprise IT Analytics
December 2016 - August 2018
- Configured a secure NiFi with LDAP for SKYTV and then upgraded NiFi against Kerberos Built up a new secure Cloudera cluster against Kerberos with Active Directory, enabled TLS/SSL for SKYTV Completed JAVA programs which transformed multiple source data from Stats NZ into the structured data using Apache Spark for Air New Zealand Developed data manipulating programs using AWS Glue, Athena, DynamoDB for Air New Zealand
Senior Data Mining Engineer
Yixia
December 2015 - October 2016
- Yixia is one of fast-growing live-streaming video application developers. Yixia operates several popular mobile video-sharing and streaming apps including MiaoPai, XiaoKaxiu and Yizhibo. Established the stream data platform with Kafka, Spark and Hadoop as the centralised platform for Yixia Created the user profiling system including the demographics, interests and behaviour Implemented similarity calculating algorithms and ranking algorithms using massive activities data about 100M video view per day for the recommendation system
Algorithm Engineer
Kika
August 2015 - December 2015
- Finished a recommendation system using Spark which finds more candidate apps for existing users
Big Data Engineer
TalkingData
April 2014 - August 2015
- TalkingData is China's leading third-party data intelligence solution provider. As a key member built up the DMP https://www.talkingdata.com/dmp.jsp? languagetype=en_us Implemented a user profiling program based on behavioural data and serving lots of clients until now Developed and deployed near real-time data pipelines using Kafka and Spark so that this project could hand about 8TB of data per day
Project Manager
Nufront
January 2011 - December 2013
- Led the AP and BP projects and designed the specification of a smart phone chipset
Field Application Engineer
LegendSilicon
January 2007 - December 2011
- Technical supporting designed demodulator chips
Embedded Engineer
Vimicro
January 2006 - December 2007
Failure Analysis Team Leader
BOEOT
January 2003 - December 2006
Education
Tsinghua University
Master of mechanical design and theory
January 2000 - January 2003
