| Section | Weight | Objectives |
|---|
| Evaluate, optimize, and monitor multi-agent solutions | 20-25% | - Optimize prompt and model performance
- 1. Diagnose context window and retrieval issues
- 2. Implement continuous improvement workflows
- 3. Optimize task duration, parallelism, and rate limits
- Design and implement evaluation and validation strategies
- 1. Implement human review processes using Microsoft Foundry
- 2. Evaluate memory, knowledge, tools, prompts, and solution quality
- Implement observability and monitoring
- 1. Monitor token usage, cost, quotas, and performance
- 2. Monitor agent health, workflow failures, tracing, and quality regression
|
| Architect multi-agent solutions | 15-20% | - Specify technology components for multi-agent solutions
- 1. Select developer tools and SDLC environment components
- 2. Design Zero Trust security components and identity boundaries
- 3. Select communication, integration, compute, persistence, observability, and monitoring components
- Design logical architecture for multi-agent solutions
- 1. Specify agent personas, scopes, boundaries, autonomy levels, and behavioral guidelines
- 2. Design memory architectures including short-term, long-term, and context sharing
- 3. Design workflows including agents, subagents, control loops, and human-in-the-loop processes
- 4. Decompose goals and objectives into workflows, agents, and tools
|
| Develop multi-agent solutions in Azure | 30-35% | - Implement multi-agent orchestration
- 1. Implement orchestration patterns including hub-and-spoke, sequential, parallel, and peer-to-peer
- 2. Implement orchestration frameworks including Microsoft Agent Framework, LangChain, and LangGraph
- 3. Implement human-in-the-loop approval workflows
- Implement agent memory, context management, and knowledge integration
- 1. Implement multi-agent memory strategies and lifecycle management
- 2. Integrate knowledge sources including search, MCP, and semantic search
- 3. Design and implement multi-agent RAG architectures
- Design and implement advanced prompt engineering strategies
- 1. Implement fine-tuning strategies for agents and models
- 2. Implement dynamic context injection and prompt lifecycle management
- 3. Design context-aware multi-agent behaviors
- Build and integrate tool ecosystems
- 1. Integrate external resources using function calling and tool usage
- 2. Build MCP servers and clients
- 3. Design tool error handling and fallback mechanisms
|
| Secure, govern, and deploy multi-agent solutions | 20-25% | - Design and implement guardrails
- 1. Design custom domain-specific guardrails
- 2. Implement guardrails for inputs, tool calls, responses, and outputs
- Deploy multi-agent solutions to Azure
- 1. Implement testing, CI/CD, and infrastructure-as-code deployment strategies
- 2. Choose release methodologies including DTAP, blue/green, and canary
- Design and implement security for multi-agent solutions
- 1. Manage secrets using Azure Key Vault
- 2. Apply shift-left security principles
- 3. Implement identity, access control, network boundaries, and authentication
|
