Senior Data Engineer - Databricks

Essential Technical Skills

• Data Engineering: Strong foundation in data engineering principles, ETL/ELT processes, and data pipeline design patterns

• PySpark: Proven hands-on experience developing data pipelines using PySpark, including DataFrames API, Spark SQL, and performance optimization

• Databricks Platform: Practical experience with Databricks workspace, cluster management, notebooks, and job orchestration

• Workspace AI Agent: Knowledge of Databricks Workspace AI Agent capabilities and integration

• Data Modelling: Experience implementing data models including dimensional modeling, data vault, or lakehouse architectures

• Delta Lake: Understanding of Delta Lake features including ACID transactions, schema evolution, and optimization techniques

• Python: Strong Python programming skills for data processing and automation

Additional Technical Skills

• SQL proficiency for data querying and transformation <...