Senior Data Engineer at BeachHead

Elevate your data engineering skills at BeachHead as a Senior Data Engineer specializing in Databricks and Data Platform Migration. Take part in transforming legacy trading and market data into scalable, modern frameworks.In this vital role for a top financial client, you will apply your expertise in Databricks and advanced data pipeline development. Your responsibilities will include creating batch and real-time ingestion pipelines and optimizing data processing workflows using Spark and Scala. You'll ensure high data quality and contribute to essential platform initiatives.Key Responsibilities:
• Design data ingestion pipelines leveraging Databricks
• Optimize Spark/Scala workflows for enhanced performance
• Implement streaming ingestion solutions using Kafka
• Develop bronze, silver, gold medallion architecture
• Ensure data integrity through validation and reconciliationRequirements:
• Hands-on experience with Databricks as an SME
• Advanced skills in Apache Sp...