Data Engineer II

Deploying and supporting the end-to-end management of our data, including ETL/ELT, data warehouse/data lake, data staging, data governance, and managing the different layers of data required to ensure a successful BI and reporting for the Procurement Execution Center (PEC).

Build and design ETL/ELT processes with Azure Data Factory (ADF) and/or Python, schedule and execute them daily and weekly.

Assemble large, complex data sets that meet functional and non-functional business requirements.

Build infrastructure for optimal ETL/ELT of data from various sources using Azure SQL and ADF.

Develop data models for DataViz, reporting and advanced analytics, optimizing performance.

Maintain conceptual, logical, and physical data models with metadata.

Manage the DevOps pipeline deployment model, including automated testing.

Deploy data stewardship and governance across the data warehouse, cleansing and enhancing data.