Job Search
Principal Data Engineer (Python, Spark, Databricks, Azure) IRC139545
| Job: | IRC139545 |
| Location: | Ukraine - Kyiv, Lviv, Mykolaiv, Kharkiv |
| Designation: | Lead Software Engineer |
| Experience: | 3-5 years |
| Function: | Engineering |
| Skills: | Spark |
Description:
The Client is the largest cloud computing and virtualization technology company.
Our team is working on the development of a sophisticated Analytics as a Service (AaaS) solution.
As a part of the team, you will contribute to the creation of means for data ingestion, validation/verification, multiple storage zones, processing, and analytics on top of that. The final solution can be used by teams of various maturity or size for smaller teams looking for the basics, or large teams interested in advanced APIs and tooling.
Requirements:
• 4+ years building conceptual, logical and/or physical database designs, or the pipelines required to build them
• 2+ years of hands-on Python experience (especially WHL and packed code)
• Expert level knowledge of using SQL to write complex, highly-optimized queries across large volumes of data.
• 2+ years of Kimball (Dimensional Modeling) Expertise
• 2+ years of Spark experience (especially Databricks Spark and Delta Lake)
• Experience with source control (git) on the command line
• Strong hands-on experience implementing big-data solutions in the Azure Data Lake ecosystem (Azure Data Lake, Databricks).
• Ability to work independently and provide guidance to junior data engineers.
• (preferred) Experience with big data pipelines or DAG Tools (Dbt, Data Factory, Airflow, or similar)
• (preferred) Experience with Kafka or other streaming technology (or a willingness to learn)
• (preferred) Experience with database deployment pipelines (i.e., dacpac’s or similar technology)
• (preferred) Once or more unit testing or data quality frameworks
• (bonus) Experience with MLFlow and other MLOps pipeline technology
Preferences:
Python, azure data factory
Job Responsibilities:
- Take major part in requirements definition, architecture, and design;
- Architect of subsystems and understand the impact across products;
- Design and implement core data flow components in accordance with project standards;
- Regular communications with the client and teams;
- Ensure project delivery in a timely matter;
- Lead engineering efficiency efforts. Improve operational efficiency;
- Daily project activities (daily standups, technical advisory, reporting, etc);
- Collaborating with the Dev Leads and Product Manager to ensure that the requirements;
- Managing daily project activities.
What We Offer
Exciting Projects: Come take your place at the forefront of digital transformation! With clients across all industries and sectors, we offer an opportunity to participate in creating market-defining products using the latest technologies.
Collaborative Environment: Expand your skills by collaborating with a diverse team of highly talented people in an open, laidback environment — or even abroad in one of our global centers or client facilities!
Work-Life Balance: GlobalLogic prioritizes work-life balance, which is why we offer flexible opportunities and options.
Professional Development: Our dedicated Learning & Development team regularly organizes certification and technical / soft skill training to help you realize your professional goals.
Excellent Benefits: We provide our consultants with competitive compensation and benefits
Fun Perks: We want you to love where you work, which is why we host sports classes, cultural, social and team building activities such as sports competitions and end-of-year corporate parties. Our vibrant offices also include dedicated GL Zones and rooftop decks where you can drink coffee or tea with your colleagues over a game of table football or darts!