Found Description
We are seeking an experienced Data Engineer to own the end-to-end lifecycle of enterprise Databricks environment from initial workspace provisioning and security hardening through ongoing performance tuning and cost optimisation. You will work closely with data engineers, ML engineers, and cloud infrastructure teams to ensure the platform runs reliably, securely, and economically at scale.
Job Responsibilities
Platform Setup & Provisioning
- Deploy and configure Databricks workspaces on AWS, Azure, or GCP using Terraform / Databricks Terraform Provider
- Design multi-workspace topologies (dev / staging / prod) with appropriate network isolation (VPC/VNet injection, Private Link)
- Automate workspace bootstrapping via CI/CD pipelines (GitHub Actions, Azure DevOps, Jenkins)
- Integrate with cloud identity providers (Azure AD / Okta / AWS IAM) for SSO and SCIM provisioning
- Configure customer-managed keys (CMK) for enc...