AI and Machine Learning on Microsoft Azure Specialization

Program guide, audit checklist, and FAQ

V1.5
Valid January 1, 2024 -June 30, 2024
Program updates and announcements
Module B – Jan 1, 2024
V1.5 AI and Machine Learning on Microsoft Azure Specialization is published. Changes include:
The Azure AI portfolio in controls 1.1 (Assessment), control 2.1 (Solution Design) control 2.3 (Proof of
Concept) and control 3.1 (Deployment) have been updated to current portfolio product names:
• AI Cognitive Services has been renamed Azure AI Services; Azure Cognitive Search has been
renamed Azure AI Search
• Azure Data Warehouse has been renamed Azure Synapse Analytics

Module B - Dec 1, 2023

The PREVIEW for V1.5 AI and Machine Learning on Microsoft Azure Specialization was made available
for partners

October 1, 2023
Azure Active Directory has been renamed Microsoft Entra ID

August 28, 2023

The Microsoft Cloud Partner Program has changed its name to the Microsoft AI Cloud Partner
Program effective immediately

Module B - July 5, 2023

V1.4 AI and ML on Microsoft Azure Specialization checklist is published. This checklist version is
required for audits July 5, 2023 – Jan 2, 2024

Updates to Control 1.1: Under the AI Powered App option, Azure AI Services have been called out. For
Azure AI Search the ability to price is added in the control option. For Azure AI Search advanced features,
vector search is now an optional choice for evidence

Azure Innovate has been added for this specialization’s program benefits – see the Partner FAQ

Module B - Jan 2, 2023

V1.3 AI and ML on Microsoft Azure Specialization audit checklist is published. This checklist version
is required Jan 2, 2023- June 30, 2023

Module B- Dec 5, 2022

The PREVIEW for V1.3 AI and Machine Learning on Microsoft Azure Specialization was made
available for partners. There are no new Module A or B Control updates
• FAQ updates include the clarification that a “No Pass” results when a partner fails or withdraws
from the audit. This status resets from “Audit Failed” within one week to “Not Enrolled,” for
partners to reapply.

Module B- Oct 3, 2022

Microsoft retired Gold Cloud partner competency, Solutions partner designation required. Gold &
Silver competencies are retired and replaced with Solutions Partner designations. For this specialization,
your organization must have an active Solutions Partner for Data & AI (Azure) designation.

2
Module A - July 1, 2022
Checklist updates published May 2, 2022 in preview for the Module A audit checklist are now
required. In Control 2.2, a new required Skilling Plan has been added to the checklist.

Module B - July 1, 2022

Checklist updates published May 2, 2022 in preview for the Module B workload, AI and Machine
Learning on Microsoft Azure specialization V1.0.1 had changes now required in V1.1 Changes
include:
• Solution Design Control 2.1 has added best practices for Responsible AI. This includes a “read
and review” acknowledgement of Microsoft’s RAI Standards and a new Impact Assessment
Template for any AI system being created. This is required July 1, 2022
Module B - May 2, 2022
1. Guidance for the definition of Proof of Concept and Pilots added to the FAQ
2. Preview updates to Module A were made available, these are required July 1, 2022

Module B - Feb 25, 2022

Clarification for Module B 1.1 Portfolio Assessment evidence

Jan 1, 2022
Guidance and FAQ Updates

3
Contents

AI and ML on MSFT Azure Specialization Program Overview .................................................................................... 5

How to apply ........................................................................................................................................................................... 5
NDAs for the audit ................................................................................................................................................................. 6
Payment terms and conditions ............................................................................................................................................ 6
Audit blueprint ....................................................................................................................................................................... 7
Audit roles ................................................................................................................................................................................ 7
Audit Process: High-level overview .................................................................................................................................... 8
Audit Process: Details ............................................................................................................................................................ 9
Audit preparation best practices and resources............................................................................................................... 9
Audit checklists ...................................................................................................................................................................... 11
Partner FAQ……………………………………………………………………………………………………………………………………..…24

4
AI and Machine Learning on Microsoft Azure Program Overview

This document defines the requirements to earn the AI and Machine Learning on Microsoft Azure
specialization. It also provides further requirements, guidelines, and audit checklists for the
associated audits required to earn this Azure specialization.

The AI and Machine Learning on Microsoft Azure specialization is designed for partners to demonstrate
their deep knowledge, extensive experience, and proven success in planning and deploying AI and Machine
Learning on Microsoft Azure cloud for their customers. Such partners empower their customers to deploy
Azure AI services and machine learning solutions, from the assessment phase to design, pilot,
implementation, and post-implementation phases to realize the full breadth of these transformative,
secure solutions at enterprise scale.

The AI and Machine Learning on Microsoft Azure specialization allows partners with an active Solution
Partner designation to further differentiate their organizations, demonstrate their capabilities, and
build stronger connections with customers. For this specialization, your organization must have an
active Solutions Partner for Digital & App Innovation (Azure) or Data & AI (Azure) designation.

Partners who meet the comprehensive requirements to earn an Azure specialization, receive a
customer-facing label they can display and a business profile in Microsoft AppSource partner gallery. In
AppSource, access to specific Microsoft go-to-market programs is prioritized in customer searches to help
drive new business. Partners can also generate a certified letter from Microsoft that verifies the Azure
specialization that they have earned. For these reasons, this opportunity is available only to partners that
meet additional, stringent requirements.

How to apply

Partners with the appropriate role and access permissions can apply. To do so, they sign into their
Partner Center account.

On the left pane, select Azure under the Specialization section. Toggle to the specialization that you
wish to apply for by using the drop-down menu at the top of the page.

5
Important: Take note of the audit effective dates and access the most current version from Partner Center.
(Audit checklists are updated twice a year). Partners audit on the version that is live on their audit date,
not the application date.
Partners may apply for the audit only after all other program requirements have been fully met. Partners
must complete the audit within thirty (30) calendar days of the audit application, and they must complete
it against the then-current program requirements.

NDAs for the audit

Auditors comply with requests from partners to sign a direct NDA. All ISSI auditors are under a
nondisclosure agreement (NDA) with Microsoft. If a partner would like an NDA to be signed directly
between ISSI and the partner organization for purposes of the audit, one can be provided by the partner
during the audit scheduling process to ISSI. ISSI will sign and return it.

Payment terms and conditions

Pricing schedule

Module B Audit: $2,000 USD

Module A+B Audits: $3,000 USD

A Gap Review Meeting is included with each Module audit.

Payment terms

The cost of the audit is payable in full to the audit company and must be settled before the
audit begins. Failure to pay will result in cancellation of the audit.

Program status term

When a partner meets all prerequisite requirements shown in Partner Center and Microsoft receives a
valid Pass Report from the third-party audit company, the partner will be awarded the AI and Machine
Learning on Microsoft Azure specialization for one (1) calendar year.

The status and the AI and Machine Learning on Microsoft Azure specialization label can be used only by
the organization (determined by Partner Center MPN PGA ID account) and any associated locations
(determined by MPN PLA ID) that met all requirements and passed the audit. Any subsidiary or affiliated
organizations represented by separate Partner Center accounts (MPN PGA ID) may not advertise the
status or display the associated label.

6
Audit blueprint

Audits are evidence-based. During the audit, partners will be expected to present evidence they have met
the specific requirements on the checklist. This involves providing the auditor with access to live
demonstrations, documents, and SME personnel to demonstrate compliance with checklist requirements.
The audit checklist will be updated to stay current with technology and market changes, and the audit is
conducted by an independent, third-party auditor.

The following is included in the audit blueprint:

1. Audit Roles
2. Audit Process: High level overview
3. Audit Process: Details
4. Audit Best practices and resources

Audit roles
Role of the auditor

The auditor reviews submitted evidence and objectively assesses whether the evidence provided by the
partner satisfies the audit checklist requirements.

The auditor selects and evaluates evidence, based on samples of the information available from live
systems. The appropriate use of such sampling is closely related to the confidence that can be placed in the
audit conclusions. All ISSI auditors are under a non-disclosure agreement (NDA) with Microsoft. Auditors
will also comply with requests from partners to sign a direct NDA.

Role of the partner

The partner must provide objective evidence that satisfies the auditor for all checklist items. It is the
responsibility of the partner to have reviewed all check-list items prior to the audit, to have collated
all necessary documentation and evidence, and to have ensured that the right subject matter experts
are available to discuss and show systems, as appropriate. All audit evidence must be reproducible
and verifiable.

Role of the Microsoft Partner Development Manager

For partners that have an assigned Microsoft Partner Development Manager (PDM), the PDM is responsible
for ensuring that the partner fully understands the requirements prior to applying for the audit. The PDM
may attend the optional consulting engagements that ISSI offers, but the PDM and other Microsoft FTEs
may not attend the audit.

7
Audit Process: High-level overview

Step Action Responsibility

1 Review: Specialization requirements in Partner Center. Review Partner

audit checklists in the specialization and begin to prepare needed
evidence with personnel for an evidence-based audit.
Recommended: Before you apply, review the specific audit checklist
thoroughly and confirm SME personnel.

2 Meet the prerequisites and apply for the audit: In the initial Partner
application phase, applications are submitted in two (2) stages:
1. Prerequisite requirements(see Partner Center for details)
2. Audit
Do not start the application process unless you are ready to
undertake the audit. Assess your firm’s ability to complete the
audit, including considerations for readiness, employee availability,
and holidays.
3 Validate: The partner meets all requirements prior to audit. Microsoft

4 Confirmed by Microsoft: Microsoft confirms to the third-party Microsoft

audit
company that the partner is eligible for audit.

5 Schedule with partner: The auditor will schedule within two(2) Auditor(with
business days. partner)

6 Conduct the audit: Within thirty (30) calendar days of the Auditor
approval for
audit.

7 Provide a Gap Report: If applicable, to the partner within two(2) Auditor

business days of the completed audit, listing any Open Action
Items. *
8 Acknowledge Gap Report receipt and schedule meeting: Partner
Within two (2) business days of receiving the Gap Report, the
partner acknowledges receipt of the report and schedules a Gap
Review Meeting. Partners can begin immediate remediation of
open items.
9 Complete the meeting: Within fifteen (15) calendar days of Auditor (with
receiving the Gap Report, the partner schedules and completes partner)
the Gap Review Meeting with the auditor to provide evidence
and address any Open Action Items. *
10 Issue Final Report: To the partner within five (5) business days. Auditor
Notify Microsoft of audit Pass or No Pass result.

11 Notify the partner: About program status within two (2) business Microsoft
days.

* These steps will be skipped if the partner has no Open Action Items after the audit.

8
Audit Process: Details

Microsoft uses an independent, third-party audit company, Information Security Systems International,
LLC (ISSI), to schedule and conduct Azure specialization audits. After the audit date has been
confirmed, ISSI will provide an agenda to the partner. The duration of an audit is four (4) hours for
Module B workloads and eight (8) hours for Module A+B audits combined, depending upon the scope
of the audit.

During the audit, the partner must provide access to the appropriate personnel who can discuss and
disclose evidence that demonstrates compliance with program requirements. We highly recommend that
subject matter experts for each section attend as well as a person who is familiar with the entire audit.
On the day of the audit, the partner must be prepared to provide the auditor with access to live
demonstrations, documents, and personnel, as necessary to demonstrate compliance with the
requirements. During the audit, the auditor will seek to verify that the partner’s evidence has addressed all
required audit checklist items satisfactorily.

A note on audit checklist effective dates: Partners are audited against the checklist items that are active on
the date of their remote audit, not the date they apply. Audits are updated twice annually. The partner
application or renewal date has no bearing on the version of the checklist that is used for the audit.
The audit can produce either of two (2) outcomes:

1. The partner passes the audit.

• The auditor will present a brief synopsis of the audit. This will include identifying
observed strengths and opportunities for improvement.
• The auditor will provide a Final Report to the partner.
• The auditor will notify Microsoft.

2. The partner does not satisfy all checklist items during the audit.

• The auditor will present a brief synopsis of the audit at the end of the day, including observed
strengths and Open Action Items, as outlined in the Gap Report, within two (2) business days.
• The partner will acknowledge receipt of the Gap Report within two (2) business days.
• The partner will move into the Gap Review phase and schedule their Gap Review Meeting
within fifteen (15) calendar days.

The Gap Review

If the partner does not, to the auditor’s satisfaction, provide evidence that meets the required scores across
all audit categories during the audit, the partner will move into a Gap Review. A Gap Review is part of the
audit and completes the process.

Within two (2) business days after the audit, the partner will receive a Gap Report, which details any Open
Action Items and the outstanding required evidence. It is suggested to begin remediation on any open
action items as soon as possible following the audit.

9
The partner then has two (2) business days to acknowledge receipt of the Gap Report and schedule a Gap
Review Meeting. The Gap Review Meeting is conducted with the auditor over the partner’s virtual
conference platform of choice. The meeting must take place within fifteen (15) calendar days of when the
Gap Report was sent, and it may last no longer than one (1) hour. During the Gap Review Meeting the
partner must present evidence that addresses any and all Open Action Items.

The Gap Review Meeting can produce either of two (2) outcomes:

1. The partner resolves all Open Action Items.

• The auditor confirms that the partner has provided the required evidence.
• The auditor provides a Final Report to the partner.
• The auditor notifies Microsoft about the outcome (subject to Auditor Terms and Conditions).
2. The partner does not resolve all Open Action Items.
• The auditor presents a brief synopsis of the audit, including missed items.
• The partner receives a Final Report that details the missed items.
• The auditor notifies Microsoft about the outcome (subject to Auditor Terms and Conditions).

If the partner is still unable to provide satisfactory evidence to the auditor during their Gap Review
Meeting, the partner will be deemed to have failed the audit. Partners that still want to earn this
Azure specialization will need to begin the application process again.

Completion of the audit

The audit process concludes when ISSI issues the Final Report after the audit or after the Gap Review.
Partners will be awarded a Pass or No Pass result upon completion of the audit process, including if they
withdraw from the audit process.

Audit preparation best practices and resources

Partners should ensure that the audit checklist has been thoroughly read in advance of the audit

• Partners should ensure that all partner stakeholders involved have a copy of the audit
checklist and that a stakeholder who knows the entire process is available for the duration of
the audit
• Partners should confirm that they have live access granted, and files and tools are readily
available during the audit exhibits

Stakeholder SME attendance in the audit

Stakeholders who can best address the relevant section should be available for the audit. However, please
make sure that a stakeholder who knows the entire process is available for the duration of the audit.

10
Auditors often probe for more information

The auditor probes for more information to ensure that mature and repeatable processes are in place with the partner
and that they are established, effective, and efficient.

The auditor is looking to see how a document was created, where it is located, and what source
materials were used to create the document. By probing for more information, the auditor evaluates
and validates that the partner is operating at an advanced level. This can only be done by questioning
during the audit. This approach is explained to the partner during the opening meeting.
Acceptable evidence: Excerpts, exhibit file formats and use of PowerPoints

PowerPoints are a common and accepted format for presenting a high-level overview of a partner’s systems.
However, please also be prepared to present live demonstrations from source files so that the auditor may
confirm that the systems in place are mature and effective. Excerpts can be used to communicate the high-
level overview but are not acceptable evidence, source documents must be presented.

Additional resources: Two optional audit preparation offers from the auditing firm*

To ensure objectivity, consulting auditors and auditors conducting the actual audits are different ISSI auditors.

1. Partners can participate in an optional, one (1)-hour, live Audit Process & Controls Overview session provided by
ISSI. This session provides a high-level overview of key aspects of the Azure Specialization audit process. The
session includes a discussion of the checklist requirements along with best practices to help partners prepare for
the audit. Partners work directly with ISSI to schedule this remote session (via online web conference). For more
information about this session, see Azure Specialization - Audit Process and Controls Overview

2. ISSI also provides optional extensive, in-depth consulting engagements to help partners prepare for their Azure
specialization audit. Partners work directly with ISSI to schedule this remote session (via online web conference).
For more information about this type of in-depth engagement, see Azure Specialization Consulting Offer
https://issi-inc.com/az-advspeconsulting/
* Please note that there is a cost associated with the consulting and audit preparations services. See Payment Terms and
Conditions.

Audit checklists

The AI and Machine Learning on Microsoft Azure specialization audit checklist contains two (2) modules,
Module A: Cloud Foundation and Module B: AI and Machine Learning on Microsoft Azure workload.

Module A, The Cloud Foundation module evaluates the use of a consistent methodology and process for
Azure adoption that is aligned with customers’ expected outcomes, spanning the entire cloud adoption
lifecycle. Module B, The AI and Machine Learning on Microsoft Azure module validates that the partner has
adopted robust processes to ensure customer success across all phases of deploying Azure AI services and
machine learning solutions, from the assessment phase to design, pilot, implementation, and post-
implementation phases.

11
Review the following audit checklist tables for more details about each control phase and to learn how the
partner will be evaluated for an audit. The same customers may be used for Module A & B. The estimated
length of both modules together is eight (8) hours.

12
Module A: Cloud Foundation
1. Strategy
2. Plan
3. Environment readiness and Azure landing zone
4. Governance
5. Manage

Module B: AI and Machine Learning on Microsoft Azure

1. Assess
2. Design& POC/Pilot
3. Deployment
4. Review and release for operations

To pass the audit, the partner must complete all audit checklist items.

Module A, Cloud Foundation is required for multiple Azure specializations. To complete Module A:
Cloud Foundation, the partner needs to pass all controls in Module A by providing the specified
evidence. Alternatively, the partner may present evidence of a previous pass result from Module A
or from another Azure specialization audit conducted on V2.0 or later. Partners who have passed
an Azure specialization audit before July 1, 2021 and for the Analytics on Microsoft Azure
specialization audit before Oct 1, 2021, have likely not passed the Module A audit and will need to
do so to qualify for the Module B workload audits.
Module B, AI and Machine Learning on Microsoft Azure workload. Each control has one (1) or
more requirements and required evidence the partner must provide for the auditor. Both the
requirements and the required evidence are defined in the following tables.

For some controls, a reference customer or customer evidence is the documentation requested. Unless
otherwise stated, the partner must show at least three (3) unique customers with deployments completed
within the last twelve (12) months. Please note some checklists call for four (4) customer examples. The
partner can use the same customer across audit checklist controls, or they can use a different customer. For
audit evidence relating to customer engagements, the partner can use a customer case study and reference it
multiple times. The same or different customers can be used for Modules A & B if they demonstrate
requirements.

13
Module A: Cloud Foundation

1.0 Strategy and Economics

The partner must have a defined approach for helping their customer evaluate and define a cloud adoption strategy
beyond an individual asset (app, VM, or data).

Requirement

1.1 Cloud Adoption Business Strategy

The partner must have a process that captures the data-driven business strategies being used to
guide customer decisions. The process should include, at minimum, the following:

• A strategy review that captures the customer’s business needs and the
problems the customer is trying to solve

• Personalized recommendations from the partner for the customers’

business strategies

Required evidence:
A Report, Presentation, or Document that captures strategic inputs and decisions for two (2) unique
customers, that demonstrates Cloud Adoption Strategy Evaluator assessment output, with projects
completed in the past twelve (12) months. These projects must be aligned with the above-described
process and highlight both customer Business and Financial outcomes.

For an example, see the Strategy and plan template in the Cloud Adoption Framework for Azure, or
the Cloud Adoption Strategy Evaluator.

2.0 Plan

The partner must have a consistent approach to planning for cloud adoption that is based on the strategy outlined in
the preceding section.

Requirement

2.1 Cloud Adoption Plan

The partner must have a process and approach for planning and tracking the completion of cloud
adoption projects. For an example of a cloud adoption plan, see the Azure DevOps Demo Generator
for the Cloud Adoption Framework.
Required evidence:
The partner must provide evidence of their capability with examples of two (2) unique
customers, with projects that were completed in the past twelve (12) months. Acceptable
evidence must include at least one (1) of the following:

• Cloud Adoption Plan Generator output or

• Azure DevOps backlog or
• Any other tools for project planning and tracking

14
2.2 Plan for Skilling
When customers adopt the cloud, their existing technical staff will need a variety of new skills to aid
in making technical decisions and to support the new cloud implementations. To ensure the long-
term success of the customer, the partner must document a skilling plan to prepare the customer’s
technical staff.

The Partner must document a list of key customer technical roles expected to require new
skills such as, but not limited to, IT Admins, IT Governance, IT Operations, and IT Security.
The documentation must include:
A description of the new skills the technical roles will need to achieve to successfully manage
the new environment.
Resources the customer can leverage when training their technical employees such as
Microsoft learning paths, technical certifications, or other comparable resources.

For guidance, review Microsoft docs Azure Cloud Adoption Framework How to build a skilling
readiness plan.

Required evidence:
The partner must provide a skilling plan for at least two (2) unique customer engagements
completed within the last 12 months. The two (2) skilling plans documentation can include a
customer-facing presentation, planning documents, post deployment documentation or similar
plan documentation.

15
3.0 Environment Readiness and Azure Landing Zone

The partner must be able to demonstrate that the following design areas are addressed through their approach
to landing zone implementation.

Requirement

3.1 Repeatable Deployment

The partner must demonstrate adherence to Azure landing zone design areas through a
repeatable deployment. The deployment should configure, at minimum, the following identity,
network, and resource organization attributes:

• Identity
o Adoption of identity management solutions, such as Microsoft Entra ID (formerly
Azure Active Directory) or equivalent

• Networking architecture design (topology)

o Define an Azure network topology - Cloud Adoption Framework | Microsoft Docs
o Application of hybrid architectures that use Azure ExpressRoute, VPN
Gateway, or equivalent services for connecting local datacenters to Azure

• Resource organization
o Implementation of tagging and naming standards during the project
The partner must demonstrate which of the following approaches they used when they
deployed Azure landing zones:

1. Start small and expand: Azure landing zone does not deploy governance
or operations configurations, which are addressed later in the
implementation.

2. Full Azure landing zone conceptual architecture: Azure landing zones implement
standard approach to the configuration of governance and operations tools prior to
implementation.

3. Alternative approach: If the partner follows a proprietary approach or a mixture of the

two (2) approaches above, the partner must clearly articulate their approach to
environment configuration.
Required evidence:
The partner must provide evidence of a repeatable deployment they used to create landing zones
aligned to the Azure landing zone conceptual architecture or equivalent complete architecture deployed
to two (2) unique customer environments using Bicep, ARM (AZURE Resource Manager) templates,
Terraform modules, or equivalent tools to automatically deploy the environment configuration.
Ifa customer deviates from specified architecture, the partner must demonstrate the customer
requirements to justify the deviation.

The provided template can be pulled directly from the implementation options, or it can be based on
the partner’s own IP (Intellectual Property). In either case, the script as evidence must demonstrate the
configuration of the identity, network, and resource organization, as described earlier.
4.0 Governance

16
 The partner must demonstrate their customer’s role in governing cloud-based solutions and the Azure tools
they use to facilitate any governance requirements their customer might have today or in the future.

Requirement

4.1 Governance Tooling

The partner must demonstrate the ability to deploy the required governance tools for two (2)
unique customer projects.

Required evidence:
The partner must demonstrate the use of Azure Policy or equivalent tool to provide controls to govern
the environment for two (2) unique customers with projects that were completed in the past twelve
(12) months.

5.0 Manage

The partner must demonstrate that they have set up their customer for operational success after the deployment is
completed. All partners have a role in setting up operations management, even if they do not provide long-term
managed services.

Requirement

5.1 Operations Management Tooling

The partner must demonstrate the use of Azure products or equivalent to help their customer and/or
managed service provider operate the environment after deployment.

Required evidence:
The partner must demonstrate the deployment of at least one (1) of the following Azure products or
third-party equivalents: Azure Monitor, Azure Automation, or Azure Backup/Site Recovery, for two (2)
unique customers with projects that were completed in the past twelve (12) months.

Module B: AI and Machine Learning on Microsoft Azure specialization workload

1.0 Assess
Partner must have a consistent approach for assessing customer requirements for AI solution.
Requirement

17
1.1 Portfolio Assessment
The partner must demonstrate how they assess current state and customer requirements to ensure
that adequate pre-deployment planning and sizing are performed. The Assessment must include:

• Requirements need analysis:

Partner must have a demonstrated approach for helping customers understand the
clear AI requirements in relation to their business goals. For example:
 Is the challenge suitable to use AI or rules to resolve? If the
challenge has enough features and weak certainty, is it suitable to
use AI to resolve?

• Business need identification: Partner must document the following:

 Current pain points, challenges, expected ROI, and end user needs
 Product fit and gaps, to identify which product will best serve the customer’s needs
 Data governance and compliance requirements
 Security requirements
 Budget requirements
 Ethics, responsible use, and other requirements around AI

• AZURE AI Solution Identification: The partner must document the following:

 Primary AI / ML use cases
 Current infrastructure or greenfield physical infrastructure
 Logical architecture and requirements
 Data storage needs from a business standpoint– volume, type,
location, current state vs. future state
 MLOps/AIOps process and key KPIs to measure (data drift, audit etc.)
 ML Model requirements and success metrics
 Solution performance requirements and data requirements.
 User roles, security requirements and how each will use and access the data
 Existing infrastructure/networking components that will connect to Azure

• Data needs: At the workload level, the partner must document which data needs
must be fulfilled to meet stated business requirements around data:
 Classification and risk of data involved – explain if any masking techniques was applied
 Identify data sources (on-premises, AWS, Google, etc.) and
destination (data storage on Azure)
 Identify who or what services should consume the data

• Security and Compliance needs: Partner must document the following customer needs:
 Identity and access management, role-based access control,
encryption, industry, and geography-centric compliance requirements,
if applicable.

In addition, the partner must demonstrate assessment capability across at least one (1) of the below
practice specialties in either Machine Learning or AI Powered Applications. All controls under each
specialty must be demonstrated in evidence.

• Machine Learning Lifecycle (competency in building custom machine learning).

The Machine Learning Lifecycle Assessment must include requirements for:

 Data drift, training, and retraining scenarios
 Inference requirements (real time and / or batch)
 Data sets available, data dictionary, data set profiles

Assessment and documentation for the required ML Lifecycle and ML Ops roles and processes:

18
 • AI-Powered Apps (competency in applying Azure AI services such as Azure Open AI
Service, Azure AI Vision, Azure AI Speech, Azure AI Language, - Azure AI Search or other
Azure AI services with applications in the Azure AI portfolio) or customizing existing
algorithms in cloud native apps.

The AI- Powered Apps Assessment must include:

• Success criteria for Azure AI services use cases (e.g., accuracy threshold) Data
requirements and data labelling requirements
• Performance, load, throughput, and transaction requirements
• Usage option (Container vs Azure host)
• Azure AI Search: Ability to appropriately price, size and scale solution to meet storage
capacity, query performance and throughput needs
• Azure AI Search: Experience with both Push and Pull data ingestion methods
• Azure AI Search: Usage of advanced search features such as: vector
search, scoring profiles for search relevancy, synonym mapping
and/or custom skill development

Required evidence:
The partner should provide relevant documents showing that the preceding items were reviewed for at
least three (3) unique customers, with projects that were completed within the last twelve (12)
months. The evidence must show that all above assessment details were considered for each customer.
Assessments may be done manually or through an industry-accepted assessment tool.

Accepted Documentation:
Any of the following can be used: An Assessment Report, an Assessment Checklist, Templates,
Questionnaires, Project Plan, Data Migration Assistant (DMA) Reports, or other Third-party
Tooling Reports.

19
2.0 Design and Proof of Concept (PoC)
Partner has robust methodologies for designing the workload.
Requirement
2.1 Solution Design
The partner must provide solution designs showing a consistent approach that addresses
customer requirements captured from the assessment phase. Solution design must show, where
applicable, the below customer requirements based on the assessment. Where the approach is
not applicable, the partner must state why not.
User Roles: User roles required to deploy the AI solution (developer, AI Engineers, data scientists,
DevOps, AIOps etc.) and establish role-based access
Data Source: All data sources and file types to be ingested
Ingestion Engine: The use of a data ingestion engine to extract, transform, load, and clean data.
Ingestion engines includes but are not limited to native products such as Azure Data Factory,
Informatica, Data Stage, and Azure Databricks
Data Storage: The storage type for the ingested data. Data storage can include but is not limited
to native products such as Azure Blob, Azure Data Lake, Azure Synapse Analytics (formerly Azure
Data Warehouse), and Azure Synapse
Encryption Method: Data encryption approach. Data encryption methodology can include but is
not limited to Transparent Data Encryption (TDE), masking and Azure Key Vault
Data preparation: Can include but is not limited to: Azure Synapse Analytics, Azure Databricks, Azure
MLCompute
Microsoft’s RAI Standard: Read and review the RAI Standard V2
Impact Assessment: Partner will conduct an impact assessment for any AI system being created
using the Impact Assessment Template and the Responsible AI Impact Assessment Guide.
Security: Appropriate service and index level security implementation (I.e., VNETs, private endpoints,
RBAC, security filtering, etc.)
Sizing and performance: Identify where design is meeting the requirements and considerations
identified in the assess phase
Monitoring: Performance monitoring and diagnostics
Cost control: Identify where design can extract financial insights from available data
Model / Algorithm/AI Service Selection: Design incorporates AI model, algorithm, service selection
including training method
Inferencing / Deployment: Approach and design (ACI AKS quotas, limits, managed endpoints
etc.) The AI considerations are incorporated into the design e.g., Transparency, explainability, bias,
disparity etc. Automation and Dev Ops: Design includes DevOps, ML Ops, CI/CD tools and
processes as required to meet the requirements
AI Architecture: Solution architecture highlights best practices and guidance for the selected
AI services

In addition, the partner must demonstrate design capability across at least one (1) of the
below practice specialties:

• Machine Learning Lifecycle (competency in building custom machine learning).

Design must show:
 ML Platform design including end to end MLOps process/ pipelines for scale
 ML scoring requirements and metrics
 Versioning and tracking, ML audit, governance, and policies
OR
• AI-Powered Apps Competency in applying Azure AI Services, (Azure Open AI Service,
Azure AI Vision, Azure AI Speech, Azure AI Language, Azure AI Content Safety),
Azure AI Search) or customizing existing algorithms in cloud-native apps.
• Design must show:
 Azure AI services transactions, pricing and cost considerations are factored
into the design
 AI Interface and interaction design is factored into the design
 Container deployment architecture is factored into the design
 Cloud native app design and development

20
  Azure AI Search: Data indexing and data tuning for search relevancy and
performance with optional vector search
 Azure AI Search: Search pipeline

2.1 Required evidence:

The partner must provide relevant solution design documents that address the points above, from at
least three (3) unique customers with Azure AI projects completed within the past twelve (12)
months.

Acceptable Documentation:
Partner must show design documents, which includes at minimum two (2) of the following:
 Project Plan
 Functional Specifications
 Architectural Diagram
 Automated Tooling Reports
 Physical and Logical diagrams
 The partner acknowledgement for review of the RAI Standard V2
as read
 A completed Customer Impact Assessment using the Impact Assessment
Template

2.2 Azure Well Architected Review of Workloads

The partner must demonstrate usage of the Azure Well-Architected Review on Azure AI workloads.
The Azure Well-Architected Review is designed to help partners evaluate your customers' workloads
against the latest set of industry best practices. It provides actionable guidance to design and
improve your customers' workloads.

The Review can be used to evaluate each workload against the pillars of the
Azure Well Architected Framework hat matter to that workload.

Required evidence:
Note the mandatory review for the operational excellence pillar. The AI Azure specialization
checklist has a requirement for the Well Architected Review for Operational Excellence Pillar,
which must be conducted upon Review and Release for Operations completion, as specified in
control 4.1.
The partner must provide exported results from the completed Microsoft Azure Well Architected
Review using the assessments in Well-Architected Review, conducted in the last twelve (12)
months, for three (3) workloads using Azure AI services indicating the customer’s name.

The three (3) workloads can come from one (1) or more customers.

21
 2.3 Proof of Concept or Pilot
Partner must provide evidence of a completed proof of concept (PoC) or pilot project. The
PoC or pilot project must validate the design decisions, review, and adjust the design as
appropriate before production rollout.

PoC or pilot project must document the purpose, customer pain points, project success criteria,
intended benefits of the project, and results for at least one (1) of the following Azure services:

1. Azure AI Services: (Azure Open AI Service, Azure AI Vision, Azure AI Speech, Azure AI
Language, Azure AI Search, Azure AI Content Safety)
2. Azure Machine Learning

Required evidence:
Partner must provide relevant documentation for one (1) customer with a completed proof of
concept (PoC) or pilot project within the past twelve (12) months that includes at least one (1)
of the four Azure AI Services listed above in 2.3.

Accepted Documentation:

PoC/Pilot architecture diagrams; Reference architectural design blueprints; Test plans and
results; Implementation documentation; other PoC documents; or a Monitoring tool report.

3.0 Deployment
Partner has robust methodologies for deploying the workload.
Requirement
3.1 Deployment
Partner must provide evidence of the partner’s capability to implement AI solutions deployed in
production environments, based on customer-approved designs. The deployment must include
at least one (1) of the following Azure products:
1. Azure AI Services
2. Azure Applied AI Services
3. Azure Machine Learning
4. Azure AI Search

Required evidence:
Partner must provide documentation for any of the above AI services for three (3) unique
customers with completed projects within the past twelve (12) months. Documentation
provided for each customer must cover the entire sequence of the project (from design
to production deployment) and must include at least two (2) of the following items:
• Signed SOWs for all projects
• Solution Design Documents for all projects
• Project Plan and Migration/deployment sequence
• Architecture Diagrams
• High-level Design (HLD) and Low-Level Design (LLD)
• As-built Documentation

22
4.0 Review and Release for Operations
Partner has robust methodologies for transitioning the workload.
Requirement
4.1 Service Validation and Testing
Partner must validate the deployment, including:
• Demonstrate the process and approach for testing and evaluating the
performance of all solutions against end user expectations and Azure best
practices.
o Demonstrate assessment where applicable against the Cloud Adoption
Framework best practices and the disciplines for Well Architected Framework
Operational Excellence Pillar:
 Application design
 Monitoring
 Application performance management
 Code deployment
 Infrastructure provisioning
 Testing

The partner must demonstrate the process and approach for evaluating and improving architectural
best practices to remediate issues with workloads that do not meet performance or cost
expectations.

Required evidence:
Documentation of testing and performance validation that addresses the above points for the
three (3) unique customers with projects completed in the last twelve (12) months. The
documentation must indicate that the implemented solution meets customer expectations with a
sign-off from the customer. These projects can be the same as the projects evidenced earlier in
Control 3.0, Deployments.

Exported results from a completed Microsoft Well Architected Review (using the Operational
Excellence pillar) assessments in Well-Architected Review conducted post-deployment, indicating
the customer’s name is the acceptable evidence for this section. The Well-Architected review must
have been for a project completed in the last twelve (12) months.

4.2 Post-deployment Documentation

Partner must provide documentation post-deployment to ensure customers are successful in using the
new service in Azure.
• Demonstrate how the partner documents, decisions, architectural
designs, and procedures were implemented.
• Demonstrate Standard Operating Procedures for business-as-usual
operations team which describe ‘how-to’ scenarios.

Required evidence:
Documentation showing the above points, for three (3) unique customers with Azure AI projects
completed within the last twelve (12) months.

These projects can be the same as the projects evidenced in Control 3.1

23
Azure Specializations Partner FAQ

Questions regarding the Azure Partner program specializations, the current checklists and pre-qualifications for
partners can usually be answered by visiting Microsoft Azure Partner Specializations

Questions on the audit checklists and program can be sent to the Azure Partner Specializations help alias
<mailto:[email protected]>

If you have questions that have not been answered , please go to Partner Center support to create a ticket
with our Frontline team.

24