Databricks Senior Data Engineer Role

Drive the future of data engineering as a Senior Data Engineer at BeachHead. Lead exciting data platform migrations using your Databricks expertise for a prominent financial client.In this role, your skills in developing data ingestion pipelines and optimizing Spark/Scala workflows will be key. You will focus on transitioning trading and market data into robust, modern frameworks, supporting ongoing data engineering tasks and platform initiatives. Collaborate effectively with technical teams to ensure scalable solutions that meet business needs.Key Responsibilities:
• Design and implement batch and real-time data pipelines
• Optimize workflows using Spark and Scala for reliability
• Implement ingestion solutions with Kafka or similar technologies
• Develop practices in medallion architecture
• Support data quality through validation and reconciliation processesRequirements:
• Expert-level experience with Databricks
• Advanced knowledge of Apache Spark and Scala
...