Projects

TableCheck_Data Operation Engineer Interview Project | June 2023

• Small project in Data Operation Engieer Interview at TableCheck
• Build a simple ETL pipeline to collect data from csv file, centralize in datawarehouse and build insightful dashboard
• Ubuntu, Shell, Apache Airflow, Python, Pandas, PostgreSQL, Google Looker Studio

MCI_Data Engineer Lecturer Interview Project | June 2023

• Base on provided data, generate the CSV data and send it continuously (streaming). Read CSV files and consume the messages. Then load it within micro-batching to datawarehouse as data mart, finally visualize the data to get insights
• Ubntu, Shell, Docker, Apache Kafka, Apache Spark, Python, Pandas, PostgreSQL, Google Looker Studio