AI Toolkit is a powerful extension for Visual Studio Code that streamlines agent development. With AI Toolkit, you can:
- 🔍 Explore and evaluate models from a wide range of providers—including Anthropic, OpenAI, GitHub—or run models locally using ONNX and Ollama.
- ⚡ Build and test agents in minutes with prompt generation, quick starters, and seamless MCP tool integrations.
Complete features include:
| Feature |
Description |
Screenshot |
| Model Catalog |
Discover and access AI models from multiple sources including GitHub, ONNX, Ollama, OpenAI, Anthropic, and Google. Compare models side-by-side and find the perfect fit for your use case. |
 |
| Playground |
Interactive chat environment for real-time model testing. Experiment with different prompts, parameters, and multi-modal inputs including images and attachments. |
 |
| Agent Builder |
Streamlined prompt engineering and agent development workflow. Create sophisticated prompts, integrate MCP tools, and generate production-ready code with structured outputs. |
 |
| Bulk Run |
Execute batch prompt testing across multiple models simultaneously. Ideal for comparing model performance and testing at scale with various input scenarios. |
 |
| Model Evaluation |
Comprehensive model assessment using datasets and standard metrics. Measure performance with built-in evaluators (F1 score, relevance, similarity, coherence) or create custom evaluation criteria. |
 |
| Fine-tuning |
Customize and adapt models for specific domains and requirements. Train models locally with GPU support or leverage Azure Container Apps for cloud-based fine-tuning. |
 |
| Model Conversion |
Convert, quantize, and optimize machine learning models for local deployment. Transform models from Hugging Face and other sources to run efficiently on Windows with CPU, GPU, or NPU acceleration. |
 |
| Tracing |
Monitor and analyze the performance of your AI applications. Collect and visualize trace data to gain insights into model behavior and performance. |
 |
Getting started
We recommend starting with models hosted by GitHub.
- Follow the installation guide to set up AI Toolkit for your device.
- From the extension tree view, select CATALOG > Models to explore models available. We recommend to getting started with models hosted by GitHub.
- From the model card, select Try in Playground to start experimenting the capability of an AI Model.
Build AI agents
The key feature of AI Toolkit is to build AI agents. The agent builder provides a set of tools to help you create and optimize your AI agents. You can use the agent builder to:
- ✨ Generate and improve prompts with natural language
- 🔁 Iterate and refine prompts based on model responses
- 🧩 Break down tasks with prompt chaining and structured outputs
- ⚡ Test integrations with real-time runs and tool use such as MCP servers
- 💡 Generate production-ready code for rapid app development
- 🧷 Use variables in prompts
- 🧪 Run agents with test cases to validate your agent easily
- 📊 Evaluate the accuracy and performance of your agent with built-in or custom metrics
- 🔗 Function calling support: Enable agents to invoke external functions dynamically
- 🗂️ Agent versioning and version comparison for evaluation results
- 🐞 Local tracing and debugging of agents
- 🚀 Deploy your models and agents to Azure AI Foundry
And a lot of features are coming soon, stay tuned for:
- ☁️ Deploy your agent to the cloud
Agents can now connect to external tools through MCP (Model Control Protocol) servers, enabling them to perform real-world actions like querying a database, accessing APIs, or executing custom logic.
| Feature |
Description |
Screenshot |
| Connect to an Existing MCP Server |
Use tools from command(stdio) or HTTP (server-sent event) |
 |
| Build and Scaffold a New MCP Server |
Start creating your own MCP server from a simple scaffold and test in Agent Builder |
 |
Feedback and resources
We value your feedback to help shape our roadmap. Explore our developer documentation for more features, open issues or share suggestions on GitHub, or join our Discord community to connect with other developers.
AI Toolkit ❤️ Developer Community.
Data and telemetry
The AI Toolkit for Visual Studio Code collects usage data and sends it to Microsoft to help improve our products and services. Read our privacy statement to learn more. This extension respects the telemetry.enableTelemetry setting which you can learn more about at disable telemetry reporting.
