DS Consultant – AI Quality & Feature Eng(NCS/Job/ 1870)

Data Science consultants for AI Quality and AI features - 4-6 years experienced Data Science consultants to support urgent bandwidth needs across AI feature quality, model evaluation, and tooling efforts. These consultants will work on high-impact LLM prompt tuning.

R&R

• Support the development and fine-tuning of Prompt APIs for key Copilot features.
• Run controlled experiments (A/B testing, Shadow testing) for validating prompt and model performance.
• Help build and validate prompts optimized for different industries, locations, and meeting scenarios.
• Create and maintain offline evaluation tools for prompt validation and quality benchmarking.
• Assist in debugging skills and regression detection across model iterations.
• Contribute to training/testing/validation datasets and frameworks for prompt-level evaluation.
• Enable skill localization by geography, industry verticals, and domain-specific intents.
• Work with team leads to identify LLM scaling opportunities and embed segmentation-based fine-tuning.
• Monitor and drive improvements in DSAT, NSAT, and engagement KPIs.
• Generate and analyze impact reports tied to skill usage, revenue drivers, and customer feedback.
• Support tasks from Prague-based teams, including output readiness reviews, documentation, and prompt audits.
• Participate in regular sync-ups with partner engineering and PM teams.

Required Experience

• 3+ years in data science, ML operations, or AI-driven feature development.
• Strong hands-on experience in Python, SQL, and evaluation metrics design.
• Familiarity with prompt engineering, LLMs, or NLP models (OpenAI, T5, etc.).
• Understanding of offline testing tools, A/B test frameworks, and prompt performance diagnostics.
• Experience working with AI skill pipelines, telemetry, or meeting productivity tools is a plus.