Data Scientist – Model Evaluation, ML Evaluation IRC264520

Description

We are seeking a talented and motivated Machine Learning Engineer to join our team. In this role, you will be responsible for designing, implementing, and running a comprehensive benchmarking framework for our existing forecasting model hosted on Google Cloud Platform (GCP). Your primary goal will be to evaluate the model’s performance rigorously using diverse external datasets, identify areas for improvement, and detect potential anomalous behaviors, drawing inspiration from established evaluation methodologies while tailoring them to our specific needs.

Requirements

Ideally the candidate has experience with data science or ML experience beyond calling a large language model endpoint to produce predictions. Dealing with models directly and experience debugging them would be very beneficial.

• Strong programming skills with proven experience in Python, including relevant data science libraries (e.g., Pandas, NumPy, Scikit-learn).
• Solid understanding of machine learning concepts, particularly evaluation metrics and methodologies for forecasting or time-series models.
• Experience in data manipulation, data cleaning, and feature engineering.
• Excellent analytical and problem-solving abilities.
• Strong written and verbal communication skills.
• Ability to work independently and1 manage project timelines effectively.

Preferred Qualifications:

• Hands-on experience with Google Cloud Platform (GCP), especially the Vertex AI platform (e.g., Vertex AI Workbench, Pipelines, Training Jobs).
• Experience in developing, training, and/or evaluating Neural Network models.
• Specific experience working with time-series forecasting techniques and models.
• Proven ability to identify, diagnose, and report anomalous behavior in machine learning models.
• Familiarity with MLOps principles and tools such as Docker.

 

Job responsibilities

• Design and develop a scalable and reproducible benchmarking pipeline primarily using Python.
• Implement and execute benchmark tests against the forecasting model, leveraging GCP infrastructure where applicable.
• Analyze benchmarking results, evaluate model performance across various metrics, data slices, and scenarios.
• Investigate and document instances of anomalous model behavior, prediction errors, or performance degradation under specific conditions.
• Develop clear reports and visualizations to communicate findings, insights, and recommendations to both technical and potentially non-technical stakeholders.
• Stay updated on best practices for model evaluation and benchmarking in the machine learning field.

What we offer

Culture of caring. At GlobalLogic, we prioritize a culture of caring. Across every region and department, at every level, we consistently put people first. From day one, you’ll experience an inclusive culture of acceptance and belonging, where you’ll have the chance to build meaningful connections with collaborative teammates, supportive managers, and compassionate leaders. 

Learning and development. We are committed to your continuous learning and development. You’ll learn and grow daily in an environment with many opportunities to try new things, sharpen your skills, and advance your career at GlobalLogic. With our Career Navigator tool as just one example, GlobalLogic offers a rich array of programs, training curricula, and hands-on opportunities to grow personally and professionally.

Interesting & meaningful work. GlobalLogic is known for engineering impact for and with clients around the world. As part of our team, you’ll have the chance to work on projects that matter. Each is a unique opportunity to engage your curiosity and creative problem-solving skills as you help clients reimagine what’s possible and bring new solutions to market. In the process, you’ll have the privilege of working on some of the most cutting-edge and impactful solutions shaping the world today.

Balance and flexibility. We believe in the importance of balance and flexibility. With many functional career areas, roles, and work arrangements, you can explore ways of achieving the perfect balance between your work and life. Your life extends beyond the office, and we always do our best to help you integrate and balance the best of work and life, having fun along the way!

High-trust organization. We are a high-trust organization where integrity is key. By joining GlobalLogic, you’re placing your trust in a safe, reliable, and ethical global company. Integrity and trust are a cornerstone of our value proposition to our employees and clients. You will find truthfulness, candor, and integrity in everything we do.

About GlobalLogic

GlobalLogic, a Hitachi Group Company, is a trusted digital engineering partner to the world’s largest and most forward-thinking companies. Since 2000, we’ve been at the forefront of the digital revolution – helping create some of the most innovative and widely used digital products and experiences. Today we continue to collaborate with clients in transforming businesses and redefining industries through intelligent products, platforms, and services.