Full stack Context Engineer (RARR Job 5821)

We are seeking a highly experienced and innovative Senior AI Engineer with a strong background in Copilot agent development and context engineering. This role is pivotal in designing intelligent, context-aware AI systems that enhance user productivity and decision-making across enterprise applications.

Key Responsibilities

Agent Architecture & Build

• Design and build multi-agent workflows for generation, testing, review, and fixing, with human approval via pull requests (PRs).

• Implement agents as GitHub Apps and Actions that:

  • Read issues and specifications

  • Propose code changes

  • Create PRs with context, branch protections, and code owner rules

• Build context retrieval systems, including:

  • Code indexing

  • Semantic search over repositories

  • API specifications and schemas to ground agent outputs

Integration & Automation

• Integrate agents into GitHub Actions for gated execution, including:

  • Linting

  • SAST

  • License checks

  • Testing and coverage

• Integrate with Jira, Azure Boards, and GitHub Issues to provide story context and acceptance criteria.

• Automate generation of:

  • Documentation

  • API docs

  • ADR notes

  • Changelogs

  • Release notes

Guardrails, Security & Governance

• Implement prompt and response filtering, including:

  • PII redaction

  • Secure secret handling

  • Data egress controls

• Enforce policy gates such as:

  • CodeQL and SAST

  • Dependency scanning

  • IaC checks

  • OWASP rules

• Define risk tiers for agent actions (e.g., tests vs. business logic vs. infrastructure) and establish escalation paths.

Evaluation, Observability & Cost Control

• Build offline and online evaluation harnesses to track:

  • Agent PR acceptance rates

  • Rework and iteration metrics

• Instrument agents using OpenTelemetry, with dashboards in Grafana and Datadog, and structured logging.

• Optimize latency and cost through:

  • Prompt caching

  • Using smaller models for simple tasks

  • Smart chunking strategies

Change Management & Enablement

• Coach developers on:

  • Writing agent-friendly stories and specifications

  • Effectively reviewing AI-generated PRs

• Create and maintain:

  • Playbooks

  • Runbooks

  • Prompt libraries

  • Policy packs for ongoing operations

Required Qualifications

• 8+ years of software engineering experience, including 3+ years building LLM-powered automation or bots at production scale.

• Strong expertise in:

  • Java, Spring Boot 3

  • Angular/ React, TypeScript

  • REST and GraphQL APIs

• Extensive experience with AWS, including:

  • ECS, EKS, Lambda

  • API Gateway, S3

  • RDS, DynamoDB

  • CloudWatch

GitHub Ecosystem Expertise

• GitHub Copilot Enterprise and Copilot Chat configuration and adoption.

• GitHub Actions authoring, including:

  • Composite actions

  • Reusable workflows

  • Self-hosted runners

• GitHub Apps development using:

  • REST and GraphQL APIs

  • Checks API

  • PR review automation

AI & Agent Frameworks

• Experience with at least one agentic orchestration framework, such as:

  • LangGraph

  • LangChain

  • Microsoft AutoGen

  • Semantic Kernel

  • crewAI