DATAOPS/CLOUD DATA ENGINEER

Responsibilities

• Design, develop, test, implement, and troubleshoot data pipelines using various tools such as Databricks, Azure Data Factory, AWS Glue
• Complex data transformation procedures
• Data models for data warehouses and lakehouses

Experience Requirements

Data engineering Experience

Programming & scripting : Python, SQL, Linux shell, PowerShell.

Data manipulation/analysis using pandas and pyspark .

Working with XLSX, CSV, JSON files, relational databases, cloud storage, structured and unstructured data.

Cloud experience

40%

• AWS and/or Azure services.
• Cloud data warehouse solutions such as AWS Redshift.
• Data lakehouse solutions such as Databricks (Delta Lake).

Data processing or orchestration/automation

25%

• Data Warehouse ETL tools (such as Azure Data Fac...