Data Engineer (AI/ML & Knowledge Foundation) IRC282068

職種概要

The project is a multi-vendor, multi-phase initiative focused on building a suite of purpose-built AI Agents for industrial equipment maintenance. These agents must deliver reliable, no-hallucinated recommendations, which fundamentally relies on a high-quality, structured Knowledge Base and Reliable AI foundation.

 

#LI-YZ3

必要条件

• Data Engineering Fundamentals: Strong proven experience in the design, implementation, and maintenance of scalable data pipelines.
• Cloud Expertise: Extensive familiarity with and a proven record of building efficient data pipelines using the Azure cloud data tech stack and associated services.
• AI/LLM Data Prep: Direct experience in data processing and preparation for AI and LLM consumption, including handling large volumes of unstructured text (e.g., technical manuals).
• RAG Architecture: Hands-on experience with technologies and methods supporting embedding and vector search for implementing robust Retrieval-Augmented Generation (RAG) applications.
• Big Data Processing: Experience in managing and processing large data sets efficiently, focusing on performance optimization and scalability.
• Domain Knowledge (Preferred): Familiarity with data challenges related to industrial sensor data, time-series data, or complex operational logs.

職務内容

The Data Engineer will play a crucial role in building the robust data ingestion and processing pipelines necessary to feed the Knowledge Base, support Retrieval-Augmented Generation (RAG) architecture, and integrate diverse data sources including sensor data and large-scale technical documentation.

Key Responsibilities:

The Data Engineer will design, implement, and optimize data infrastructure specifically tailored for the AI/LLM components of the platform.

Data Pipeline Development and Management

• Pipeline Creation and Maintenance: Design, build, and maintain production-grade data pipelines, including the Manuals and Safety protocols Ingestion Pipeline, ensuring continuous alignment and updates of the knowledge base.
• Large Dataset Management: Process and manage large datasets derived from complex sources, including maintenance logs, historical data, safety documents, and real-time sensor streams (vibration, temperature, flow, etc.).
• Data Ingestion and Extraction: Implement data extraction tools and translation tools to transform unstructured manuals and protocols into structured data suitable for AI consumption.
• Data Alignment: Ensure the ingestion of maintenance data aligns with and extends the knowledge base with detailed information about equipment internals.

Data Preparation for AI and LLM Consumption

• RAG System Support: Prepare and process data specifically for use in RAG AI apps (Retrieval-Augmented Generation), which are critical for the Reliable AI foundation to achieve validated, zero-hallucination output.
• Embedding and Vector Search: Implement and manage processes for generating embeddings and storing them for vector search to enable the AI agents to retrieve relevant, grounded information from the Knowledge Base.
• Data Validation and Quality: Work closely with the AI/ML teams to ensure the ingested data supports the Validation of the Agents based on Reliable AI, providing the necessary evidence-based validation that underpins the assertion of zero hallucination.
Optimization and Cloud Technology
• Data Processing Optimization: Continuously monitor and optimize existing data pipelines to improve latency, throughput, and cost efficiency, especially for systems integrating real-time ML sensor data analysis.
• Azure Cloud Data Stack: Leverage Azure cloud data services (such as Azure Data Factory, Azure Synapse Analytics, Azure Cosmos DB, etc.) to build efficient, scalable, and resilient data infrastructure within the HTA cloud environment.
• Integration with ML/Sensor Data: Facilitate and manage the data flow for the Integration with ML sensor data analysis, connecting agents with device sensor data, ML analytics, and models.

私たちが提供するもの

Culture of caring. At GlobalLogic, we prioritize a culture of caring. Across every region and department, at every level, we consistently put people first. From day one, you’ll experience an inclusive culture of acceptance and belonging, where you’ll have the chance to build meaningful connections with collaborative teammates, supportive managers, and compassionate leaders. 

Learning and development. We are committed to your continuous learning and development. You’ll learn and grow daily in an environment with many opportunities to try new things, sharpen your skills, and advance your career at GlobalLogic. With our Career Navigator tool as just one example, GlobalLogic offers a rich array of programs, training curricula, and hands-on opportunities to grow personally and professionally.

Interesting & meaningful work. GlobalLogic is known for engineering impact for and with clients around the world. As part of our team, you’ll have the chance to work on projects that matter. Each is a unique opportunity to engage your curiosity and creative problem-solving skills as you help clients reimagine what’s possible and bring new solutions to market. In the process, you’ll have the privilege of working on some of the most cutting-edge and impactful solutions shaping the world today.

Balance and flexibility. We believe in the importance of balance and flexibility. With many functional career areas, roles, and work arrangements, you can explore ways of achieving the perfect balance between your work and life. Your life extends beyond the office, and we always do our best to help you integrate and balance the best of work and life, having fun along the way!

High-trust organization. We are a high-trust organization where integrity is key. By joining GlobalLogic, you’re placing your trust in a safe, reliable, and ethical global company. Integrity and trust are a cornerstone of our value proposition to our employees and clients. You will find truthfulness, candor, and integrity in everything we do.

GlobalLogicについて

GlobalLogic, a Hitachi Group Company, is a trusted digital engineering partner to the world’s largest and most forward-thinking companies. Since 2000, we’ve been at the forefront of the digital revolution – helping create some of the most innovative and widely used digital products and experiences. Today we continue to collaborate with clients in transforming businesses and redefining industries through intelligent products, platforms, and services.