Found Description
Overview
As a Senior Data Engineer, you’ll be part of a team which works with big data & cloud technologies such as Spark, (Scala & Pyspark), Python, Databricks, Airflow, Hive etc.
Responsibilities
- Responsible for maintaining & assembling data pipelines that consume large datasets, runs the complex data transformations that are needed to generate the facts for the clients.
- Responsible for integrating finished models into the pipelines that also helps in building the insights for the customer.
- Use your spark skills to optimize spark code (including data science models).
- Building required infrastructure for optimal extraction, transformation and loading of data from various data sources using Azure, Databricks and RDMBS technologies.
- Write and maintain documentation for features you work on & transition them to L2 support.
- Ability to write highly efficient code in terms of performance / memory utilization. <...