Senior/Principal Data Engineer (Python, Spark, Databricks, Azure) IRC152786
|Location:||Poland - Krakow|
|Designation:||Lead Software Engineer|
|Skills:||Azure Data Lake Store, Databricks, Python, Spark, SQL|
Our client is one of the leaders in Software as a service in the world of business applications.
You will be working on a data platform.
About the platform:
A suite of products to deliver a flexible modern data estate in the customer’s Azure tenant without the major expense and effort of a large buildout or data program.
We build, deliver, and update a data platform for customers to build on top of, without the hassle and expense of an implementation.
Customers see data available in their environment week one, with sophisticated history and query technology – no engineering!
• 4+ years building conceptual, logical and/or physical database designs, or the pipelines required to build them
• 2+ years of hands-on Python experience (especially WHL and packed code)
• Expert level knowledge of using SQL to write complex, highly optimized queries across large volumes of data.
• 2+ years of Kimball (Dimensional Modeling) Expertise
• 2+ years of Spark experience (especially Databricks Spark and Delta Lake)
• Experience with source control (git) on the command line
• Strong hands-on experience implementing big-data solutions in the Azure Data Lake ecosystem (Azure Data Lake, Databricks).
• Ability to work independently and provide guidance to junior data engineers.
• (preferred) Experience with big data pipelines or DAG Tools (Dbt, Data Factory, Airflow, or similar)
• (preferred) Experience with Kafka or other streaming technology (or a willingness to learn)
• (preferred) Experience with database deployment pipelines (i.e., dacpac’s or similar technology)
• (preferred) Once or more unit testing or data quality frameworks
• (bonus) Experience with MLFlow and other MLOps pipeline technology
azure data factory, Kafka
Take major part in requirements definition, architecture, and design;
Architect of subsystems and understand the impact across products;
Design and implement core data flow components in accordance with project standards;
Regular communications with the client and teams;
Ensure project delivery in a timely matter;
Lead engineering efficiency efforts. Improve operational efficiency;
Daily project activities (daily standups, technical advisory, reporting, etc);
Collaborating with the Dev Leads and Product Manager to ensure that the requirements;
Managing daily project activities.
What We Offer
Exciting Projects: With clients across all industries and sectors, we offer an opportunity to work on market-defining products using the latest technologies.
Collaborative Environment: You can expand your skills by collaborating with a diverse team of highly talented people in an open, laidback environment — or even abroad in one of our global centers or client facilities!
Work-Life Balance: GlobalLogic prioritizes work-life balance, which is why we offer flexible work schedules.
Professional Development: We develop paths suited to your individual talents through international knowledge exchanges and professional certification opportunities.
Excellent Benefits: We provide our employees with private medical care, sports facilities cards, group life insurance, travel insurance, relocation package, food subsidies and cultural activities.
Fun Perks: We want you to feel comfortable in your work, which is why we create good working environment with relax zones, host social and teambuilding activities and stock our kitchen with delicious teas and coffees!
About GlobalLogicGlobalLogic is a leader in digital engineering. We help brands across the globe design and build innovative products, platforms, and digital experiences for the modern world. By integrating experience design, complex engineering, and data expertise—we help our clients imagine what’s possible, and accelerate their transition into tomorrow’s digital businesses. Headquartered in Silicon Valley, GlobalLogic operates design studios and engineering centers around the world, extending our deep expertise to customers in the automotive, communications, financial services, healthcare and life sciences, manufacturing, media and entertainment, semiconductor, and technology industries. GlobalLogic is a Hitachi Group Company operating under Hitachi, Ltd. (TSE: 6501) which contributes to a sustainable society with a higher quality of life by driving innovation through data and technology as the Social Innovation Business.