Principal data engineer (databricks)

Role Overview

Reporting to the Data & Analytics Lead, the Principal Data Engineer is a hands-on role, combining the building and optimisation of production data pipelines with the technical leadership needed to raise engineering standards and develop the people around them. The role functions as a key interface between a) business stakeholders and department heads, b) the wider data and analytics team, and c) owners of source systems and data in Finance and IT.

Key Responsibilities Designing and building robust, production-grade data pipelines on Databricks, making appropriate use of current platform capabilities such as Spark Declarative Pipelines (SDP) Working “under the hood” to optimise the platform for performance and cost — tuning compute, jobs and queries, managing storage and table layout, and keeping platform spend under control Defining and maintaining consistent business metrics and a semantic layer (for example, using Metric Views) so reporting is built on trusted,...