Found Description
Responsibilities
- Develop tools to improve data flows between internal/external systems and the data lake/warehouse.
- Work with stakeholders to understand needs for data structure, availability, scalability, and accessibility.
- Build robust and reproducible data ingest pipelines to collect, clean, harmonize, merge, and consolidate data sources.
- Understanding existing data applications and infrastructure architecture.
- Build and support new data feeds for various Data Management layers and Data Lakes.
- Evaluate business needs and requirements.
- Support migration of existing data transformation jobs in Oracle, and MS SQL to Snowflake.
- Lead the migration of the existing data transformation jobs in Oracle, Hive, Impala etc. into Spark, Python on Glue etc.
- Able to document the processes and steps.
- Develop and maintain datasets.
- Improve data quality and efficiency.
- Le...