Description
We are seeking a highly skilled Сonsultant Data Engineer to be the driving force behind this massive architectural shift. You will be hands-on in moving a highly complex, highly regulated data ecosystem from an on-premise Teradata environment to a cutting-edge AWS “Lakehouse” architecture.
In this role, you will focus on building robust data pipelines, modernizing legacy code, and ensuring flawless data delivery. You will work closely with System Analysts, Automation Engineers, and Data Architects to convert hundreds of legacy scripts into modern dbt models, orchestrate them via Astronomer (Airflow), and establish governed warehousing in Databricks.
Requirements
Required Skills & Technologies
- Cloud Platforms: Hands-on engineering experience within the AWS ecosystem (S3, IAM, Glue) and deep expertise in Databricks (SparkSQL, PySpark, governed cataloging).
- Modern Data Stack: Strong proficiency in dbt (Core or Cloud) for data transformation and understanding of modern table formats, specifically Apache Iceberg.
- Orchestration: Experience building and maintaining data pipelines using Apache Airflow (Astronomer experience is a strong plus).
- Languages: Expert-level SQL (ability to read Teradata SQL/BTEQ and write highly optimized Spark/Databricks SQL) and strong Python skills for pipeline development and Airflow DAGs.
- Legacy Systems: Familiarity with Teradata architecture and utilities (FastExport, BTEQ) to effectively understand the source systems being decommissioned.
Experience & Qualifications
- Experience: 7+ years of dedicated Data Engineering experience, with a proven track record in complex data migration projects (On-Premise to Cloud).
- Financial Services Background: Experience working in a highly regulated banking or financial services environment is strongly preferred, particularly within Risk or Finance domains.
- Engineering Best Practices: Strong grasp of CI/CD for data pipelines, Git version control, data modeling (Kimball/Inmon), and infrastructure as code (Terraform).
- Communication: Ability to work collaboratively in a fast-paced, cross-functional agile team, effectively communicating technical blockers to technical leads and project managers.
Job responsibilities
- Lakehouse Engineering: Build and maintain scalable data pipelines to migrate Consumer Risk datasets from Teradata to Amazon S3, utilizing Apache Iceberg for the lakehouse layer and Databricks for governed warehousing.
- Code Modernization (BTEQ to dbt): Perform complex code conversions, transforming up to 500 legacy Teradata BTEQ scripts into modular, version-controlled, and highly performant dbt (data build tool) models.
- Pipeline Orchestration: Design, develop, and monitor complex Directed Acyclic Graphs (DAGs) using Astronomer (Apache Airflow) to orchestrate end-to-end data workflows.
- Data Lineage & Transparency: Integrate pipelines with OpenLineage to ensure complete, end-to-end data traceability, providing the transparency required for regulatory compliance.
- Reconciliation & Parity: Work alongside the QA/Automation team to execute automated reconciliation frameworks, debugging and resolving data discrepancies to ensure strict attribute-level parity and timing equivalence during governed parallel runs
#LI-NV1
#LI-Hybrid
What we offer
Our goal is to build an inclusive positive culture where everyone can feel comfortable being themselves, empowering people to create their own high standards and therefore more value. We work together to promote fairness while recognising, valuing and embracing differences – providing a transparent support structure and generous training budget to help our people develop skills to progress their career. Our region supports a hybrid model that can flex across a wide spectrum of working options determined by business, customer and individual needs.
You'll benefit from a comprehensive health and wellness plan, private healthcare (clinical and mental wellbeing), discounted gym memberships, and in office yoga sessions and massages. We offer a fantastic benefits package including a competitive pension scheme and recognition schemes through bonus/reward initiatives. Colleagues are entitled to an annual volunteering day – so you can take time to support a cause close to your heart. We also love to stay social with trips to the zoo, quiz nights, sports events, theatre trips and much more.
We are an equal opportunities employer. It is our policy to promote an environment free from discrimination, harassment and victimisation, and we actively welcome applications from people with a disability or long-term health condition. As users of the Disability Confident scheme, we guarantee to interview all disabled applicants who meet the minimum essential criteria for this vacancy.
We are committed to providing reasonable adjustments. If you require any adjustments for the application process, the interview, or the job itself, or if you need this information in an accessible format, please contact uki-careers@globallogic.com.
About GlobalLogic
GlobalLogic, a Hitachi Group Company, is a trusted digital engineering partner to the world’s largest and most forward-thinking companies. Since 2000, we’ve been at the forefront of the digital revolution – helping create some of the most innovative and widely used digital products and experiences. Today we continue to collaborate with clients in transforming businesses and redefining industries through intelligent products, platforms, and services.
