Project overview: The program is a multi phase data migration initiative replacing legacy capital markets systems with a modern platform ecosystem. It includes critical datasets across custody, clearing and settlement, derivatives processing, and CCP operations. The project spans 18 months and follows an incremental approach, with a strong emphasis on data integrity, reconciliation, and traceability.
- Position overview: We are looking for a Senior Data Engineer to develop and optimize data pipelines within a large scale data migration program in a capital markets environment. This is a hands on, developer oriented role for someone with a strong software engineering background who has moved into Data Engineering and cloud data migration on AWS.
-
You will design and implement robust, scalable, production grade data pipelines, ensuring data quality, consistency, reconciliation, and traceability across multiple systems.
Technology stack: Java, Python, SQL (Oracle and PostgreSQL), AWS (Glue, Athena, Redshift), dbt, Maven, Git
- Responsibilities: Design and implement scalable ETL/ELT pipelines using AWS Glue, Python, and dbt.
-
Take a highly hands on role, spending most of your time coding and reviewing code.
-
Integrate data from multiple Oracle and PostgreSQL legacy systems into AWS.
-
Write and optimize complex SQL and Java/Python data transformation logic.
-
Define and enforce coding standards, testing practices, code reviews, CI/CD processes, and deployment procedures.
-
Guide and mentor mid level engineers through technical direction and code reviews.
-
Ensure data quality, consistency, integrity, reconciliation, and traceability.
-
Collaborate with data architects on target data models and transformation strategies.
-
Optimize pipelines for performance, scalability, and cost efficiency across AWS Glue, Athena, and Redshift.
-
Implement error handling, logging, monitoring, and observability.
-
Collaborate with data analysts and business stakeholders to ensure migrated data meets business requirements.
-
Troubleshoot complex data and code related issues and drive root cause analysis.
-
Contribute to data platform architecture and document technical solutions.
- Requirements: Advanced Java skills with strong, production grade development experience.
-
Advanced SQL skills with Oracle and PostgreSQL, along with strong database knowledge.
-
Advanced Python skills for data processing and transformation.
-
Strong hands on experience with dbt or equivalent transformation frameworks.
-
Experience with Maven for build and dependency management.
-
Experience with Git and modern version control workflows.
-
Proven experience building production grade ETL/ELT pipelines at scale.
-
Strong experience with data migration to AWS Cloud, combined with a solid software development background.
-
Solid understanding of data modeling and data warehouse architectures.
-
Experience implementing CI/CD practices in data environments.
-
Strong problem solving skills and the ability to design scalable, maintainable solutions.
-
Fluency in Spanish, both written and spoken.
- Nice to have: Experience with Spring or Spring Boot.
-
Experience with AWS Data Engineering services, including Glue, Athena, and Redshift.
-
Experience with Bash scripting.
-
Experience with Docker or Podman.
-
Experience with API design and integration.
-
Experience in capital markets, including custody, clearing, settlement, derivatives, or CCP.
-
Experience working on large scale data migration or transformation programs.
-
Experience with Oracle legacy systems.
-
Understanding of data governance, data lineage, and metadata management.
-
Experience with observability and monitoring solutions.
