Found Description
What you will be doing:
- Design, develop, and maintain robust, scalable, and fault-tolerant data pipelines for structured, semi-structured, and unstructured data.
- Build and optimise enterprise data solutions using technologies such as Microsoft Fabric, Azure Databricks, Apache Spark, Python, Scala, and Ab Initio.
- Develop high-performance ETL/ELT pipelines to support enterprise Data Warehouses and Data Lakes.
- Implement DataOps best practices, including automation, orchestration, testing, and CI/CD for batch and streaming data workloads.
- Collaborate with software engineering teams to design and implement secure APIs and data services for real-time data consumption.
- Design, build, and manage hybrid cloud data platforms using Microsoft Azure technologies and on-premises data platforms where required.
- Apply Infrastructure as Code (IaC) principles to automate cloud infrastructure deployment and management.
- Establi...