Job Search
Senior Data Engineer IRC203967
| Job: | IRC203967 |
| Location: | United Kingdom - Manchester |
| Designation: | Senior Software Engineer |
| Experience: | 5-10 years |
| Function: | Engineering |
| Skills: | Airflow, AWS, Docker, Kubernetes, S3, Scala, Spark |
| Work Model: | Remote, On-Site/Office, Hybrid |
Description:
We are GlobalLogic, a Hitachi Group Company.
Based across 19 countries, GlobalLogic is a leader in digital engineering. We help brands across the globe design and build innovative products, platforms, and digital experiences for the modern world.
By integrating experience design, complex engineering, and data expertise – we help our clients imagine what’s possible and accelerate their transition into tomorrow’s digital businesses. At GlobalLogic, we have a phenomenal track record of successfully delivering ground-breaking Cloud & Digital Transformation programmes and we have an international reach working with a wide variety of projects and customers. Our approach enables companies “build the exceptional” and be fit for purpose in the 21st century.
We hire based on personality, potential, and enthusiasm to make a difference, then we give you the tools and skills you need to follow your own path. You’ll benefit by gaining exposure to a wide range of tools and technologies that you can then put into practice and become certified on various Cloud (and related) technologies that will help you to develop your own toolkit.
Our client is the world’s leading global information services company, unlocking the power of data to create more opportunities for consumers, businesses, and society. FORTUNE named them in one of the 100 Best Companies to work for. In addition, for the last five years they’ve been name in the 100 “World’s Most Innovative Companies” by Forbes Magazine.
As a Senior Data Engineer you’ll use your skills to transform data as well as developing data pipelines and visualisation solutions using industry recognised common tooling and expertise.
Requirements:
- DBT core experience is must
- Container Images: Docker
- Container Registry: TBD
- Compute Orchestrator: Kubernetes (Managed version EKS)
- Data Pipeline Orchestrator: Airflow | Kestra
- Data Transformation Frameworks: Spark & DBT
- Programing Language (Data Pipeline): Scala & sbt
- Configuration Management: AWS App Config | AWS Parameter Store
- Code Repository: Github / git
- Object Storage: S3
- Table Metastore: AWS Glue
- Open Table Format: Iceberg
- DBT Compute Engine: AWS Athena (possibility of also covering Snowflake)
- Visualization: Looker | Tableau
Job Responsibilities:
- Development of a File Ingestion Framework (Spark/Scala)
– Read files from object storage (landing zone).
– Convert file to open table format Iceberg.
– Perform some column label categorization.
– Schema Inference, pushed to central repository and data services. - Development of Transformation Application (Spark/Scala)
– Read Iceberg tables (raw/refined zone).
– Translate legacy application code into next-gen tech stack (see above).
– Write datasets back into proper lake zone. - Development of DBT data models/macros, setup standards of development for other less-technical teams to continue expanding the code-library following proper procedures.
- Work collaboratively with Software Engineers around the Orchestration component of our applications.
- Development of Export Framework
– Easily export files from our lake into agreed upon channels
– IAM Controls.
What We Offer
Why work at GlobalLogic
Our goal is to build an inclusive positive culture where everyone can feel comfortable being themselves, empowering our people to create their own high standards and therefore more value. We work together to promote fairness while recognising, valuing and embracing differences – providing a transparent support structure and generous training budget to help our people develop skills to progress their career. Our region also supports a hybrid model which can flex across a wide spectrum of working options determined by our business, customer and individual needs.
You’ll benefit from a comprehensive health and wellness plan, private healthcare (clinical and mental wellbeing), and discounted gym memberships. We offer a fantastic benefits package including a competitive pension scheme and recognition schemes through bonus/reward initiatives. Colleagues are entitled to an annual volunteering day – so you can take time to support a cause close to your heart. We also love to stay social at our trips to the zoo, quiz nights, sports events, theatre trips and much more.
We are an equal opportunities employer. It is our policy to promote an environment free from discrimination, harassment and victimisation. We are proud signatories to the Tech Talent Charter which was created because solving the diversity problem in tech requires a collective effort across companies and sectors – https://www.techtalentcharter.co.uk/.