ML Eng – AI Ops & Model Infrastructure (NCS/Job/ 1873)

• Build and maintain serving infrastructure for ONNX models, Augloop, SLM-based inference, and future LLM/SLM pipelines.
• Integrate models into scalable APIs for online prediction and retrieval-augmented generation workflows.
• Set up and run real-time A/B experiments on production Copilot features.
• Implement alerting, logging, and telemetry tools to monitor model drift, latency, and regressions.
• Develop dashboards for automated quality monitoring and error detection in inference traffic.
• Optimize inference latency and cost across CPU/GPU environments.
• Build internal tools for performance analysis, model comparison, and troubleshooting.
• Work on batch and streaming inference frameworks, ensuring SLA adherence.
• Implement resource orchestration and utilization tracking across CPU/GPU workloads.
• Contribute to tools that monitor uptime, throughput, container health, and job scaling.
• Ensure scalability and reliability of model APIs, with clear SLAs around latency, throughput, cost, and memory footprint.
• Profile models and infra for cold start issues, load testing, and concurrency handling.
• Integrate Responsible AI checks for fairness, explainability, and performance variance.
• Address AI injection attacks, inference sandboxing, and privacy guardrails.
• Contribute to regression pipelines for SLA, PII, and compliance validation across Copilot features.

Required Experience

• 3–6 years of hands-on experience as an ML SWE or MLOps Engineer in production AI systems.
• Strong coding skills in Python, C++, or Go, with experience in TensorRT, ONNX Runtime, or similar.
• Experience with ML Ops tools: Azure ML, Kubernetes, Prometheus, Grafana, MLflow, Airflow, etc.
• Hands-on with monitoring systems, load testing tools, and infra debugging utilities.
• Familiarity with model security, compliance frameworks, or Responsible AI practices is a plus.

Soft Expectations

• Able to work independently and deliver code-quality infrastructure within agile cycles.
• Document architecture, assumptions, and SLA metrics clearly.
• Comfort in collaborating with both AI scientists and infra/DevOps teams.
• Availability for overlap with Prague or Redmond teams preferred.