DATAOPS/CLOUD DATA ENGINEER

Responsibilities

Design, develop, test, implement, and troubleshoot data pipelines using various tools such as Databricks, Azure Data Factory, AWS Glue

Complex data transformation procedures

Data models for data warehouses and lakehouses

Experience Requirements Data engineering Experience

Programming & scripting : Python, SQL, Linux shell, PowerShell.

Data manipulation/analysis using

pandas

and

pyspark .

Working with XLSX, CSV, JSON files, relational databases, cloud storage, structured and unstructured data.

Cloud experience 40%

AWS

and/or

Azure

services.

Cloud

data warehouse

solutions such as AWS Redshift.

Data lakehouse solutions such as Databricks (Delta Lake).

Data processing or orchestration/automation 25%

Data Warehouse ETL tools (such as Azure Data Factory, AWS Glue) as well as cloud agnostic...