Found Description
Overview
DeepLight AI is a specialist AI and data consultancy with extensive experience implementing intelligent enterprise systems across multiple industries, with particular depth in financial services and banking.
Responsibilities
- Build scalable ingestion pipelines using AWS Glue, PySpark, and ETL pipeline development.
- Implement AWS Glue jobs for Bronze layer ingestion, applying defined standards and templates.
- Design and execute historical loading mechanisms to bring legacy data into the Lakehouse.
- Optimize Glue job performance (DPU allocation, parallelization, partitioning) according to best practices.
- Collaborate with platform teams to ensure tooling and optimization alignment.
- Automate migration of source tables to Bronze layer, leveraging AI-enabled acceleration.
- Version-control jobs and automate production deployment via Git and Terraform.
- Establish source system connectivity int...