9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine

Search - Podcasts Top Results Training

Guidelines for Search - Podcasts Top Results Training

What are Apple Podcasts?

The importance of your work as an Apple Podcasts Rater

Helpful tips & tricks in rating the results

How to assess the popularity of a Podcasts content

Researching Query Intent

Preview pages

Spell Correction

Adult Content

Rating Non-English Queries

Problem: Other

Unacceptable: Misleading

Mandatory Rating Comment

Relevance Ratings based on Query Intent

Queries pointing to a singular, navigational result

What are Channels on podcasts?

What are the types of Podcasts?

Rating Scale for Queries with a singular, navigational result

How to rate Episodes results when the query type is Podcast Show Navigational

What is a snippet?

PODCAST SHOW NAVIGATIONAL

Ratings general idea:

Query: [pardon my take]

Query: [my favorite murder]

Query: [serial]

Query: [hooked]

Query: [smartless]

PODCAST EPISODE NAVIGATIONAL

Query: [why is everyone moving to dallas]

Query: [dollup ronald reagan]

Query: [women building peace]

Query: [serial the alibi]

Query: [doctor death chris and jerry]

PODCAST CHANNEL

Query: [ted audio collective]

Query: [wondery]

Queries without singular, navigational results

Rating Scale for Queries without a singular, navigational result

PODCAST HOST

Query: [lex fridman]

Query: [georgia hardstark ]

Query: [deepak chopra]

Query: [joe rogan]

Query: [bailey sarian]

Query: [dax shepard]

Query: [adam grant]

Query: [karen kilgariff]

AMBIGUOUS

Query: [emma]

Query: [theres something about]

BROWSE

Query: [podcasts]

CATEGORY

Query: [science]

Query: [true crime]

Query: [art]

Query: [news]

PODCAST TOPIC/KEYWORD

Query: [game of thrones]

Query: [asmr]

Query: [mafia]

PROVIDER

https://baseline.apple.com/training/evaluations/1036/guidelines 1/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine

Query: [npr]

Query: [crooked media]

Query: [domino magazine]

PERSON

Query: [martin short]

Query: [anthony joshua]

Query: [alastair reynolds]

Update 9/01/2023
Added instructions regarding snippets

Update 08/11/23
Add an example to the How to rate episodes results when the query type is podcast navigational section

What are Apple Podcasts?

Apple Podcasts is home to the largest and most talked-about collection of shows across
all subjects. From globally recognized names to the most authentic independents, this is
a place where every voice matters. And that says it all.

podcasts.apple.com

https://baseline.apple.com/training/evaluations/1036/guidelines 2/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine

podcasts.apple.com

podcasts.apple.com

Apple Podcasts is the best app for finding, following, and listening to millions of the
world’s most popular podcasts. And you can quickly discover new shows through expert
curation and personalized recommendations.

The importance of your work as an Apple Podcasts

Rater
The data we receive from you in the form of high-quality relevance judgments will be used to build and improve
artificial intelligence systems such as search algorithms and machine-learned rankers that power the user
experience for Apple Podcasts users.

Our ultimate goal is to surprise and delight our customers by improving search quality and enhancing customer
satisfaction, and you play an essential role in this.

Ask yourself, “Why is this particular result returned for this query? Is this relevant?”. Then, stay curious and do

https://baseline.apple.com/training/evaluations/1036/guidelines 3/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine
a thorough search to understand the relationship between a query and the result.

Your attention to detail, research and language skills, and cultural knowledge of the market are all critical to the
success of our projects. Please keep in mind that your tasks will be spot-checked for quality and measured
against your peers. If your accuracy rate is consistently high enough, you may be qualified to work as an
auditor who gives feedback to your peers.

Helpful tips & tricks in rating the results

How to assess the popularity of a Podcasts content
To define how popular content is, you must research different platforms and not rely only on Apple Podcasts'
user signals. For example, Google Podcasts, Google Search, and Spotify are great resources, but you will not
have the user signals ( number of reviews and star avg) available; hence, you would need to look at how high
on the Search Results Page the intended content is being ranked. Usually, content closer to the top of the page
is popular on those platforms.
Many Podcasts are also video casts, and creators most of the time have their content on Youtube, making this
platform very useful for measuring a Podcast Show's ( number of subscribers in the YouTube channel) and an
episode's ( number of views) popularity.
Make sure to use reliable Podcasts platforms from your country to assess popularity - please provide the link to
it.

Remember: use the Apple Podcasts app as a starting point rather than a definitive popularity signal resource.

Researching Query Intent

We expect you to use search engines such as Bing, Google, etc., to help you understand the intent of each
query. For example, searching "[query] podcast" or "[query] apple podcasts" will help narrow down the intent.

Primary and Secondary intent:

The primary intent of a query is the most likely intent, i.e., the intent of most users who enter that query in
Apple Podcasts.
On the other hand, a secondary intent is less likely or would be a less popular intent than a primary one. A
secondary intent could be:
content relevant to a smaller group of users than for the primary intent
complimentary content; such as reviews or summaries of the intended show
lower quality/lesser-known content

Preview pages
Each result will have a link to an Apple Podcasts Preview page. This preview page will provide more details
about the result and allow you to listen to the Podcast.

Spell Correction
Our users misspell some queries. When this is the case, assume the query intent is for the correctly spelled
query.
If the query is [mark kilmek], the user meant to type [mark klimek]. Use your best judgment.

Adult Content
We assume that all adult or pornographic content is Off-Topic for non-adult and non-porn queries. For example,
any adult or porn content is Unacceptable: Off-Topic for the query [popular podcasts] because the user has not
expressly indicated that they want the content of this type. On the other hand, if the query is [erotica] or [porn],
any adult or porn content is relevant.

Rating Non-English Queries

Please note that Apple Podcasts users query for content in multiple languages.
All queries in all languages are valid search queries, and the results should be rated based on relevance and
quality, just like with any other type of query. Please utilize translation tools and conduct thorough side
searches to understand the likely user intent.

For example, suppose the user enters the Spanish language query [liderazgo hoy]. In that case, they're looking
for the "Liderazgo Hoy con Victor Hugo Manzanilla" Podcast, which would be rated Perfect. Results should be
in either the language of the query, market language, or in English (OK for all markets).

Note: If the result is in a different language than the query, with no apparent relevance to that market or direct
title match then the rating should be Unacceptable: Off-Topic.
For example, user enters the English language query [politics] and the result is German channel [RND

https://baseline.apple.com/training/evaluations/1036/guidelines 4/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine
RedaktionsNetzwerk], the rating would be Unacceptable: Off-Topic
For example, in the US market a result in Spanish would be relevant but considered secondary intent.

Problem: Other
A problem or technical issue with the task in BaseLine that makes it impossible to judge relevance. For
example, a piece of content with an unavailable show/episode/channel/provider name, an unavailable
thumbnail image, and no clickable preview page link, as seen in the screenshot below.

-
Note: “Problem: Other” rating should not be used if the content is unavailable in Apple Podcasts, but it is still
possible to determine relevance between input and output based on the result title, description or provider
name and by completing a side search on Google and/or on other platforms such as Spotify, Stitcher, TuneIn,
etc.

Exception: For cases where only the thumbnail image is available but no title/description and content is
unavailable, rate Problem: Other

Unacceptable: Misleading
This rating should be used for content that is likely to mislead users about its authorship and/or non-podcast
content. A thorough side search and checking the Apple Podcasts Preview page information for each result
will aid in spotting potential discrepancies and red flags.

Mandatory Rating Comment

Each rating must be explained in the comment box [even if “optional” is indicated]. Rating comments are
always mandatory. The comment should be concise and must only explain why the rating chosen is the
correct one in applying the guidelines. For a Podcast Navigational query, if the result is the intended podcast, it
gets the rating of Perfect. The comment can be, “This is the queried for Navigational podcast, rated Perfect.”

Relevance Ratings based on Query Intent

Queries pointing to a singular, navigational result
1. Podcast Navigational (users are looking for a specific podcast show, e.g. [this week in marvel], [fake
heiress])
2. Podcast Episode Navigational (users are looking for a specific podcast episode [episode 6 fake heiress])
3. Podcast Channel (users are looking for a specific podcast channel, e.g. [ted audio collective], [wondery
kids], [wondery], [audiochuck], [national geographic], [marvel])

What are Channels on podcasts?

A channel is a collection of shows offered by a Podcasts creator. You can listen to free shows and
channels and purchase a subscription to others to access exclusive content, new releases, ad-free episodes,
and more.

Please find below a couple of examples of podcast channel search results on an iOS device.

What are the types of Podcasts?

There are two types of Podcasts, Episodic and Serial. Please see the definition below:

https://baseline.apple.com/training/evaluations/1036/guidelines 5/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine
Episodic: This is the most common type of Podcast. The Podcast episodes are displayed from newest-to-
oldest. The most recently published episodes will be listed at the top of the results page. This format is
recommended for stand-alone episodes. A good example of an “episodic” Podcast is news and current affairs
types of shows.

Here is an example of the episodic format on the Apple Podcasts web preview page:

Podcast show: The Daily - episodes are arranged from the newest-to-oldest.

Example of Episodic type of Podcast

Serial: The Serial format arranges podcast episodes from oldest-to-newest nested within the assigned
season/year.

The newest season/year is listed at the top of the Apple Podcasts listing. However, there are cases where the
oldest/first season (Season 1) is listed at the top of the result page. This option supports narratives, storytelling,
thematic, and multiple seasons/years.

Here are a couple of examples of the serial format on the Apple Podcasts web preview page:

Podcast show: Dirty John displays the oldest season(Season 1, or the year of the first season, 2017) followed
by the first episode of the season at the top of the result page. Note: sometimes, the show releases two
episodes on the same day.

https://baseline.apple.com/training/evaluations/1036/guidelines 6/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine

Example of serial type of podcast

Podcast show: CounterClock displays the latest season followed by the first episode of the season at the top of
the result page. Note: sometimes, the show releases all episodes (or parts) of the season/year on the same
day in ascending numeric order. Please see the screenshot below:

Example of serial type of podcast

Rating Scale for Queries with a singular, navigational

result
Perfect: The singular piece or collection of content implied by the query. Reserved for the Navigational
result.
Excellent: A piece of content that clearly satisfies a likely primary intent of the query.

https://baseline.apple.com/training/evaluations/1036/guidelines 7/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine
Good: A piece of content that only partially satisfies the intent of the query but is still high-quality
or satisfies the intent of the query but is lower quality
or satisfies a secondary intent of the query.
Acceptable: A piece of content that is related to the query but may be of poor quality, not popular, or poorly
satisfies a secondary intent of the query.
Unacceptable: Off-Topic: A piece of content that does not satisfy any of the above conditions.
Unacceptable: Misleading - This rating should be used for content that is likely to mislead users about its
authorship intentionally and/or non-podcast content.
Problem: Other: A problem with the task in BaseLine; for example, the result image thumbnail and the result
show/episode title are missing, making it impossible to rate the result.

How to rate Episodes results when the query type is

Podcast Show Navigational

Episodic type of Podcasts

Rating:

Excellent:
All the recent or latest* episodes, except trailer episodes, from the intended show (Episodes in both video
and audio types)
* Sometimes the intended Podcast Show stop releasing new episodes. The latest released episodes are the
most recent ones and should be rated as Excellent.

Good:
“Old/dated” episodes from the intended show would receive a Good rating. Below are some examples of
what we should consider an “old” episode that are eligible for a rating of Good:
Daily release frequency: 1-2 weeks old episode
Semi-weekly frequency: 3 weeks old episode
Weekly release frequency: 1 month-old episode
Trailer episodes of the intended show

Acceptable:
Episodes from the secondary intent podcast show

Below is an example of a show that has stopped releasing new episodes

https://baseline.apple.com/training/evaluations/1036/guidelines 8/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine

Screenshot 2023-08-11 at 3.12.47 PM.png · 485 KB

Serial type of Podcasts

Rating:

Excellent:
Recent episodes, except trailer episodes, of the current season
Recent follow-up/bonus episodes from a past season
First episode of the season (S1E1 , S2E1, S3E1…)

Good:
All other episodes that are not from the current season
Not a recent follow-up/bonus episodes from old “cases”
Season’s trailer
Not the first episode of the season (S1E2, S3E4, S2E8...)

Acceptable:
Episodes from the secondary intent podcast show

What is a snippet?
A snippet is a line from an episode transcript that is shown in the podcast search results when the user
searches for content using keywords/tokens.
Matching tokens from the search query to tokens in the transcript snippets can indeed enhance the user
experience in some ways.

Below is an example of a snippet and how it is shown in baseline UI:

https://baseline.apple.com/training/evaluations/1036/guidelines 9/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine

Example of a snippet

The goal is to provide another avenue to establish relevance beyond title matching. Sometimes there will be a
full query ↔ snippet match and sometimes only partial match. Remember, snippets is to establish connection
between the query and the episode result. Episode's relevance should be predominant when choosing the
appropriate rating. Make sure to do a side research to understand the content of the episode result and its
relevancy to the query.

NOTE: You will see snippets returning for all query types, but it will be most helpful for queries without a
singular navigational intent, such as Topic/Keyword, Browser, Ambiguous, and Category.

PODCAST SHOW NAVIGATIONAL

Navigational queries imply a particular podcast that the user typically queries for by name, e.g., [man in the
window podcast], [pod save america], [armchair expert], [armchair expert with dax shepard].

Ratings general idea:

Perfect: The intended Podcast Show

Excellent:
Episodes of the intended Podcast show
Note: Episodes of the intended Podcast Show can receive Excellent or Good based on recency. Please
familiarize yourself with the type of podcasts and how that would affect the ratings
Future release episodes of the intended Podcast Show - This is not a trailer episode. Please refer to the
example - Query [smartless] later in this section.
Channel of the intended Podcast Show

Good:
Trailer episode of the intended Podcast Show
Secondary intent Podcast Show
High-quality podcasts sharing the same category and slightly similar topic
Podcast shows with near text match AND high popularity
High-quality podcast by the same channel/provider/creator with loose connection to category and/or topic.
Note: those Podcasts can receive Good - Acceptable rating based on quality and popularity.

Acceptable:
Episodes from the secondary intent Podcast Show
Channel of the secondary intent Podcast Show
Other podcast shows with a connection to the provider but on a different topic, or in a similar category but
different topic or low quality, or poorly satisfying a secondary intent such as content only relevant through
title match.

Please keep in mind that recency, relevance, quality, and popularity should be always taken into
consideration while selecting the rating. Please find below examples and further explanation on how to rate
Podcasts content by each rating. Note: both video and audio media types are relevant.

Query: [pardon my take]

Perfect - The intended podcast show, Pardon My Take

The podcast Pardon My Take by Barstool Sports. Pardon My Take (PMT) is a comedic sports podcast on
which “Big Cat & PFT Commenter deliver the loudest and most correct sports takes in the history of the
spoken word.” Category: Sports & Recreation. This is the navigational episodic show the user is searching
for, rate as Perfect.

Excellent - content likely to satisfy primary intent, for example, seasons and episodes - video and audio
media types - ( Please refer to How to rate episodes results when the query type is Podcast Navigational
section) from the intended show, including future releases episodes and the channel result containing the
intended podcast show.
Note: Episodes of the intended show can receive Excellent or Good based on recency. Please familiarize
yourself with the type of podcasts and how that would affect the ratings

The podcast Pardon My Take by Barstool Sports - is an episodic type of podcast ( please refer to “What are
the types of Podcasts?”), and all recent episodes from the intended show should be rated as Excellent.
The Channel Barstool Sports. This is a Podcasts Channel result that contains the intended show, and other
shows by Barstool Sports, which “began as a newspaper in 2003 and has since grown into one of the top
media brands and podcast networks in the country, with shows like Pardon My Take, Million Dollaz Worth of
Game, Spittin Chiclets, and so many more. Follow alongside your favorite personalities in sports, comedy,
and culture on Barstool Sports, with new content from their 60+ shows every single day of the week.” This
result is highly relevant to the query and likely to satisfy the primary intent of the user, and it should be rated
Excellent.

Good - content likely to satisfy a secondary intent, for example but not limited to, episodes - video and audio
media types - ( Please refer to How to rate episodes results when the query type is Podcast Navigational
section), trailer of the intended show and secondary intent Podcasts shows: high-quality podcasts sharing the
same category and slightly similar topic or high-quality podcast by the same provider/creator with loose
connection to category and/or topic, or near text match podcasts WITH high popularity.

The podcast Pardon My Take by Barstool Sports - is an episodic type of podcast ( please refer to “What are
the types of Podcasts?”), and dated episodes should be rated as Good at most.
Heartland Radio: Presented by The Pat McAfee Show by Barstool Sports. This is another comedy podcast
by Barstool Sports, but it's not on the topic of sports commentary. Likely to satisfy a secondary intent of most
users. Rate as Good.
The Podcast The Pat McAfee Show 2.0. This Podcasts Show will satisfy a secondary intent; therefore it
merits Good rating. This is a popular podcast that follows the same themes, styles, and concepts as Pardon
My Take. On "The Pat McAfee Show, Pat McAfee and his friends deliver one-of-a-kind opinions that won't be
heard anywhere else. Pat's a recently retired NFL All-Pro who has zero filters." Category: Sports &
Recreation.

https://baseline.apple.com/training/evaluations/1036/guidelines 10/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine

Acceptable - for example but not limited to, lower quality podcasts with a connection to the provider but on a
different topic, or in a similar category but different topic or low quality, or poorly satisfying a secondary intent
such as content only relevant through title match, Channel or episodes from the secondary intent Podcast
Shows.

Recent Episode from a secondary intent Podcast Show Pad McAfee Show 2.0: PMS 2.0 937 - Feel Good
Friday With The Match Recap, Mark Schlereth, Kendrick Perkins, & AJ Hawk - the result would poorly satisfy
the user intent; hence Acceptable rating.
Pod Yourself A Gun - A Sopranos Podcast. This is a comedy podcast, but not on the topic of sports.
Acceptable relevance at best.
Macrodosing: Arian Foster and PFT Commenter podcast by Barstool Sports. This is a podcast from the same
provider as Pardon My Take, but on a completely different topic and category. In this show “Arian Foster and
PFT Commenter explore conspiracies, conundrums, and the dark corners of the deep web”. Because the
intended podcast Pardon My Take and this result are very different but share the same provider, rate as
Acceptable.

Unacceptable: Off-Topic - result with no apparent relevance to the intent

Life Behind Bars with Noah Rothbaum & David Wondrich by The Daily Beast. “Life Behind Bars, which is
hosted by the Daily Beast’s Half Full editor Noah Rothbaum and its legendary columnist David Wondrich,
allows listeners to be flies on the wall as two of the world’s foremost drinks experts talk about cocktails and
spirits”. Category: Society & Culture. This podcast is unrelated to the query and the user’s intent and should
be rated Unacceptable: Off-Topic.

Unacceptable: Misleading - this rating should be used for content that is likely to mislead users about its
authorship
The Sports Betting Rundown by Pardon My Take. The provider name "Pardon My Take" is the name of the
highly popular Pardon My Take by Barstool Sports, and is very misleading to use as a provider name for a
low-quality sports commentary podcast. Rate as Unacceptable: Misleading.

Query: [my favorite murder]

Perfect - The intended podcast show, My Favorite Murder

The particular podcast show implied by the query: My Favorite Murder with Karen Kilgariff and Georgia
Hardstark. Lifelong fans of true crime stories Karen Kilgariff and Georgia Hardstark tell each other their
favorite tales of murder and hear hometown crime stories from friends and fans. Categories: Comedy & True
Crime. This is the navigational episodic show the user is searching for, rated as Perfect.

Excellent - content likely to satisfy primary intent, for example, seasons and episodes - video and audio
media types - ( Please refer to How to rate episodes results when the query type is Podcast Navigational
section) from the intended show, including future releases episodes and the channel result containing the
intended podcast show.
Note: Episodes of the intended show can receive Excellent or Good based on recency. Please familiarize
yourself with the type of podcasts and how that would affect the ratings

My Favorite Murder - is an episodic type of podcast ( please refer to “What are the types of Podcasts?”),
and allrecent episodes from the intended show should be rated as Excellent.
The Channel result Exactly Right . This is a Podcasts Channel result that contains the intended show, and
other shows from the makers of the hit podcast, My Favorite Murder. “The Exactly Right podcast network
provides a platform for bold, creative voices to bring to life provocative, entertaining, and relatable stories for
fans of true crime, comedy, and popular culture.” This result is highly relevant to the query and likely to satisfy
the primary intent of the user, and it should be rated Excellent.

Good - content likely to satisfy a secondary intent, for example but not limited to, episodes - video and audio
media types - ( Please refer to How to rate episodes results when the query type is Podcast Navigational
section), trailer of the intended show and secondary intent Podcasts shows: high-quality podcasts sharing the
same category and slightly similar topic or high-quality podcast by the same provider/creator with loose
connection to category and/or topic, or near text match podcasts WITH high popularity.

My Favorite Murder - is an episodic type of podcast ( please refer to “What are the types of Podcasts?”),
anddated episodes should be rated as Good at most.
The Last Podcast on the Left (LPOTL). LPOTL hosts Ben Kissel, Marcus Parks, and Henry Zebrowski
discuss everything from serial killers to unsolved cases to giant conspiracies. Categories listed on Wikipedia:
comedy; true crime; dark comedy; paranormal. This podcast would satisfy a secondary intent.
What The F Podcast - “Best Friends, Rebecca Lassiter and Tamara London, share real and entertaining
stories that will make you say What the F!? They discuss true crime, pop culture, current news events,
personal and viral stories.” Since this comedy podcast covers true crime, but with a distinct pop culture
angle, it’s likely to satisfy only a secondary intent of most users looking for My Favorite Murder. Rate as
Good.
Jensen and Holes: The Murder Squad. “Retired Cold Case Investigator Paul Holes and Investigative
Journalist Billy Jensen dig deep into unsolved murders, unidentified remains, and missing person cases.”
This podcast has no comedic elements, but it's from the same provider as "my favorite murder", and in the
true crime category, rate as Good for satisfying a likely secondary intent.

Acceptable - for example but not limited to, lower quality podcasts with a connection to the provider but on a
different topic, or in a similar category but different topic or low quality, or poorly satisfying a secondary intent
such as content only relevant through title match, Channel or episodes from the secondary intent Podcast
Shows.

The Shocking Details. This podcast has no comedic elements, but is on the topic of true crime. Likely to only
poorly satisfy a secondary intent of the query. Rate as Acceptable. Morbid: A True Crime Podcast is a
comedy podcast very similar to MFM, and would be a better result to return here.

Unacceptable: Off-Topic - result with no apparent relevance to the intent

The Podcast Entrepreneur Stories 4 Inspiration. This podcast doesn't relate to the query in terms of category,
topic, themes, or style. Category: Business. Note: Any episode of Entrepreneur Stories 4 Inspiration will also
get a rating of Unacceptable: Off-Topic

Query: [serial]
Perfect - the navigational result

Serial, the particular serial navigational podcast show implied by the query. Serial is an investigative
journalism podcast hosted by Sarah Koenig, narrating a nonfiction story over multiple episodes (S1
investigated the 1999 murder of Hae Min Lee, S2 focused on Sergeant Bowe Bergdahl, S3 explored cases
within the Justice Center Complex in the Cleveland area).

Excellent - content likely to satisfy primary intent, for example, seasons and episodes - video and audio
media types - ( Please refer to How to rate episodes results when the query type is Podcast Navigational
section) from the intended show, including future releases episodes and the channel result containing the
intended podcast show.

https://baseline.apple.com/training/evaluations/1036/guidelines 11/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine
Note: Episodes of the intended show can receive Excellent or Good based on recency. Please familiarize
yourself with the type of podcasts and how that would affect the ratings

Serial is a serial type of podcasts ( please refer to “What are the types of Podcasts?”). Below are possible
scenarios where the episode result can receive an Excellent rating ( please, familiarize yourself with How to
rate episodes results when the query type is Podcast Navigational section):
Episodes from the current season
Recent follow-up/bonus episodes from a past season
First episode of the season (S1E1 , S2E1, S3E1…)
Serial the Podcasts Channel result. This Podcasts Chanel contains the intended show, and other shows from
the makers of the hit podcast, Serial. This result is highly relevant to the query [serial], likely to satisfy the
user's primary intent, and it should be rated Excellent.

Exception: Below is an example of a show very similar to the intended

podcast. This is rare and the correct rating will be Excellent for the
Podcast show and Good for its Episodes ( please refer to “What are the
types of Podcasts?” as the rating can change based on recency)

S-Town - S-Town is also an investigative journalism Podcast hosted by Brian Reed and created by the
producers of Serial and This American Life. S-Town is very similar to Serial in its themes and topics and is
likely to satisfy the primary intent of the user searching for [serial]. Rate as Excellent.
S-Town - is a serial type of podcasts, ( please refer to “What are the types of Podcasts?”) very similar to the
intended podcast, Serial. Below are possible scenarios where the episode result can receive an Good rating
( please, familiarize yourself with How to rate episodes results when the query type is Podcast
Navigationalsection):
Episodes from the current season
Recent follow-up/bonus episodes from a past season
First episode of the season (S1E1 , S2E1, S3E1…)

Good - content likely to satisfy a secondary intent, for example but not limited to, episodes - video and audio
media types - ( Please refer to How to rate episodes results when the query type is Podcast Navigational
section), trailer of the intended show and secondary intent Podcasts shows: high-quality podcasts sharing the
same category and slightly similar topic or high-quality podcast by the same provider/creator with loose
connection to category and/or topic, or near text match podcasts WITH high popularity.

Full-length episode (October 2018) from the intended show, Serial: “S03 Episode 04: A Bird in Jail Is Worth
Two on the Street” - The result is neither an episode from the current season nor the first episode of a
season; the rating should be Good at most. Please, familiarize yourself with How to rate episodes results
when the query type is Podcast Navigational section, and the below scenarios for Good rating episodes
results:
All other episodes that are not from the current season
Not a recent follow-up/bonus episodes from old “cases”
Season’s trailer
Not the first episode of the season (S1E2, S3E4, S2E8...)
The Nightmare Next Door. Nightmare Next Door is a true crime, episodic type of show that peels back the
peaceful facade of small-town America and exposes the dark secrets hidden behind its white-washed picket
fences. Since this show is not an investigative journalism one like Serial, it’s likely to only satisfy a
secondary intent of the user and should be rated Good.
Wondery the Podcasts Channel result: This channel contains 111 podcast shows, including true crime
podcasts "The Vanished Podcast", and "Over My Dead Body". Likely to satisfy a secondary intent, rated
Good.
The Trojan Horse Affair - Trailer from the intended show, Serial. This is a trailer, not a full-length episode, and
hence likely to only satisfy a secondary intent of most users. Rated Good.

Acceptable - for example but not limited to, lower quality podcasts with a connection to the provider but on a
different topic, or in a similar category but different topic or low quality, or poorly satisfying a secondary intent
such as content only relevant through title match, Channel or episodes from the secondary intent Podcast
Shows.

The Nightmare Next Door is an episodic type of podcast and satisfies a secondary intent. Either recent or
dated episodes results should be rated as Acceptable at most.
Introducing: Nice White Parents - Trailer. Nice White Parents is a five-part series from the the makers of
Serial and The New York Times. Hosted by Chana Joffe-Walt. This trailer is likely to only poorly satisfy the
secondary intent of most users since it’s very different from Serial the investigative journalism crime podcast.
Rated Acceptable.

Unacceptable: Off-Topic - result with no apparent relevance to the intent

Mental Illness Happy Hour. This is a Health & Fitness podcast hosted by Paul Gilmartin, unrelated to Serial,
the intended podcast. Rate as Unacceptable: Off-Topic.

Query: [hooked]
Acceptable - for example but not limited to, lower quality podcasts with a connection to the provider but on a
different topic, or in a similar category but different topic or low quality, or poorly satisfying a secondary intent
such as content only relevant through title match, Channel or episodes from the secondary intent Podcast
Shows.

Fiction podcast "Hooked on Horror" by Hooked . The result is relevant through title match but is unpopular
and different from the intended Apple TV+ podcast, rate as Acceptable.

Query: [smartless]
Excellent - content likely to satisfy primary intent, for example, seasons and episodes - video and audio
media types - ( Please refer to How to rate episodes results when the query type is Podcast Navigational
section) from the intended show, including future releases episodes and the channel result containing the
intended podcast show. Note: Episodes of the intended show can receive Excellent or Good based on
recency. Please familiarize yourself with the type of podcasts and how that would affect the ratings

Searching for "smartness" on 01/11/2023 would return a future release episode, "Steven Spielberg" (the
episode will be release on 01/16/2023.) The result is very relevant to the query and should be rated as
Excellent. Please see picture below:

https://baseline.apple.com/training/evaluations/1036/guidelines 12/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine

PODCAST EPISODE NAVIGATIONAL

Navigational episode queries imply a very specific episode that the user typically queries for by name, i.e.
[serial the alibi]. The episode requested in the query receives a rating of Perfect.

Perfect: The intended Podcast episode

Excellent: channel of the episode, other episodes, and the show of the navigational episode would likely be
excellent.
Excellent ~ Acceptable: Other podcasts/channels on the same topic, category and/or themes, etc. ,
according to relevance, quality, and popularity.

Query: [why is everyone moving to dallas]

Perfect - the navigational episode

Why Is Everyone Moving to Dallas? by Freakonomics Radio + Stitcher -

https://podcasts.apple.com/us/podcast/491-why-is-everyone-moving-to-dallas/id354668519?
i=1000548433781. This is the intended, queried for the episode, rated Perfect. “When Stephen Dubner
learned that Dallas–Fort Worth will soon overtake Chicago as the third-biggest metro area in the U.S., he got
on a plane to find out why. Despite getting stood up by the mayor, nearly drowning on a highway, and eating
way too much barbecue, he came away impressed. (Part 1 of 2 — because even podcasts are bigger in
Texas.)”

Excellent - a piece of content that clearly satisfies a likely primary intent of the query, for example, the show
and any other episode or season of the podcast of the navigational episode

The Freakonomics Radio show https://podcasts.apple.com/us/podcast/freakonomics-radio/id354668519.

How Did a Hayfield Become One of America’s Hottest Cities? by Freakonomics Radio. “Frisco used to be just
another sleepy bedroom community outside of Dallas. Now it’s got corporate headquarters, billions of
investment dollars, and a bunch of Democrats in a place that used to be deep red. Is Frisco nothing more
than a suburb on steroids — or is it the future of the American city?”. This is part 2 of the intended podcast
episode “Why Is Everyone Moving to Dallas?” and is highly relevant to the query even though the connection
is less apparent than part 1. Rate as Excellent.

Good - a piece of content that only partially satisfies the intent of the query, but is still high-quality or satisfies
the intent of the query but is lower quality or has near text match WITH high-quality or satisfies a secondary
intent of the query

7 Reasons Why Everyone is Moving to Dallas Texas, an episode by Living In Dallas Texas. “Dallas Texas is a
great city to live, work and play. We will go over 7 reasons why a lot of people move to the Dallas area.“ This
is a lesser-known podcast episode by the provider Darius Holmes, related to the user’s intent. Likely to
satisfy a secondary intent, rate as Good.
The podcast show Living In Dallas Texas by Darius Holmes https://podcasts.apple.com/us/podcast/living-in-
dallas-texas/id1524098101. This is a lesser-known podcast show by the provider Darius Holmes, related to
the user’s intent. Likely to satisfy a secondary intent, rate as Good.

Acceptable - a piece of content that is related to the query (i.e. keyword/title matching), but may be of poor
quality, not popular, or poorly satisfies a secondary intent of the query

EPISODE 6: SPRAWL II - EXPLORING DALLAS AND NORTH TEXAS Living in the Sprawl: Southern
California's Most Adventurous Podcast https://podcasts.apple.com/us/podcast/episode-6-sprawl-ii-exploring-
dallas-and-north-texas/id1566972238?i=1000526727775. “On this week's episode, we take you on a trip
outside of Southern California to Dallas, Texas. Host Jon Steinberg shares his top ten list of things to do
when exploring Dallas and North Texas.” This episode about Dallas could poorly satisfy a secondary intent of
the query and should be rated Acceptable.

Unacceptable: Off-Topic - result with no apparent relevance to the intent

Chris and Jerry, an episode of Wondery https://podcasts.apple.com/us/podcast/chris-and-

jerry/id1421573955?i=1000419075761. This is a True Crime podcast episode, completely unrelated to the
query and the user intent. Rate as Unacceptable: Off-Topic.

Query: [dollup ronald reagan]

Perfect - the navigational episode

400 - Ronald Reagan w/ guest Patton Oswalt (Part 1) https://podcasts.apple.com/us/podcast/400-ronald-

reagan-w-guest-patton-oswalt-part-1/id643055307?i=1000453535509. This is part 1 of 2 of the navigational

https://baseline.apple.com/training/evaluations/1036/guidelines 13/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine
Episode.
400 - Ronald Reagan w/ guest Patton Oswalt (Part 2) https://podcasts.apple.com/us/podcast/400-ronald-
reagan-w-guest-patton-oswalt-part-2/id643055307?i=1000454358815. This is part 2 of the navigational
Episode. Note: since parts 1 and 2 are sharing the same Episode name, they are both rated Perfect. If Part 2
did not share the same episode title or the input was specifically for part 1, this result would be rated
Excellent.

Query: [women building peace]

Perfect - the navigational episode

Women Building Peace: Afghanistan https://podcasts.apple.com/us/podcast/women-building-peace-

afghanistan/id73802620?i=1000548714569.
Women building peace: Ethiopia https://podcasts.apple.com/us/podcast/women-building-peace-
ethiopia/id73802620?i=1000550199115. Usually the title of the episode isn't the same. These are matching
the query and are one of many 'women building peace' episodes and they are all eligible for perfect.

Query: [serial the alibi]

Perfect - the navigational episode

Serial S01 Episode 01: The Alibi https://podcasts.apple.com/us/podcast/s01-episode-01-the-

alibi/id917918570?i=1000319686008. This is the intended episode from the intended show.

Excellent - a piece of content which clearly satisfies a likely primary intent of the query, for example the show
and any other episode or season of the podcast of the navigational episode.

the show and any other episode or season of the podcast Serial

Unacceptable: Off-Topic - results with no apparent relevance to the intent

Up and Vanished - S1E3: The Alibi - https://podcasts.apple.com/us/podcast/s1e3-the-alibi/id1140596919?

i=1000374970705. This is an episode titled "The Alibi", but it's from the show Up and Vanished, not Serial.
Completely unrelated to the user intent. Note: The Podcast Up and Vanished will also get a rating of Off-
Topic

Query: [doctor death chris and jerry]

Perfect - the navigational episode

Chris and Jerry | 2, Dr. Death - https://podcasts.apple.com/hr/podcast/chris-and-jerry-2/id1421573955?

i=1000419075761. This is the intended episode, rated Perfect.

Excellent - a piece of content which clearly satisfies a likely primary intent of the query, for example the show
and any other episode or season of the podcast of the navigational episode.

Any other episode or season of the Podcast Dr. Death

Unacceptable: Off-Topic - results with no apparent relevance to the intent

V-Day Special of Let Me tell you a story, hosted by Chris and Jerry -
https://podcasts.apple.com/us/podcast/ep-4-v-day-special/id1450082856?i=1000429485167This Podcast
has a keyword match in "Chris and Jerry", but is completely unrelated to the user intent. Note: The Podcast
Let Me tell you a story by Chris & Jerry will also get a rating of Unacceptable: Off-Topic.

PODCAST CHANNEL
Navigational Podcast Channel queries imply a very specific Podcast Channel which the user typically
queries for by name, i.e. [ted audio collective], [wondery kids], [espn], [crooked media], [the new york times],
[national geographic].

Query: [ted audio collective]

https://baseline.apple.com/training/evaluations/1036/guidelines 14/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine

Perfect - the navigational channel requested by the user

The TED Audio Collective channel result → https://podcasts.apple.com/us/channel/ted-audio-

collective/id6442503658. This is the intended, navigational channel, rated Perfect.

Excellent - a piece of content that clearly satisfies the primary intent of the query. For example, the primary
intent of Shows and Episodes in the collective

WorkLife with Adam Grant (TED Talks) podcast show https://podcasts.apple.com/us/podcast/worklife-with-

adam-grant/id1346314086
Taken for Granted: Indra Nooyi Says It’s Time for Leaders to Care, an episode from WorkLife with Adam
Granthttps://podcasts.apple.com/us/podcast/taken-for-granted-indra-nooyi-says-its-time-for-
leaders/id1346314086?i=1000540467431

Good - a piece of content that only partially satisfies the intent of the query, but is still high-quality or satisfies
the intent of the query but is lower quality or satisfies a secondary intent of the query

TED en Español - https://podcasts.apple.com/us/podcast/ted-en-español/id1368017834. In the US storefront,

for the majority of listeners, a Spanish-language podcast will likely satisfy a secondary intent.

Acceptable - a piece of content that is related to the query (i.e. keyword/title matching), but may be of poor
quality, not popular, or poorly satisfies a secondary intent of the query

Unacceptable: Off-Topic: a piece of content that does not satisfy any of the above conditions

Verdict with Ted Cruz by Soundfront https://podcasts.apple.com/us/podcast/verdict-with-ted-

cruz/id1495601614. This podcast is completely unrelated to the query [ted audio collective] and should be
rated as Unacceptable: Off-Topic despite the keyword match on “ted”.

Problem: Other - a problem with the task in BaseLine, for example the result image thumbnail and the result
show/episode title are missing, making it impossible to rate the result.

Query: [wondery]
Perfect - the navigational channel requested by the user

The Wondery channel result →https://podcasts.apple.com/us/channel/wondery/id6442475667. This is the

intended, navigational channel, rated Perfect.

Good - a piece of content which only partially satisfies the intent of the query, but is still high-quality or satisfies
the intent of the query but is lower quality or satisfies a secondary intent of the query

The Wondery Kids channel result - https://podcasts.apple.com/us/channel/wondery-kids/id6442504168

Although The Wondery Kids Podcast Channel result is from the same provider as Wondery, this is geared
only towards kids and is only likely to satisfy a secondary intent and be rated Good.

Acceptable - a piece of content that is related to the query (i.e. keyword/title matching), but may be of poor
quality, not popular, or poorly satisfies a secondary intent of the query

USA Today channel result - https://podcasts.apple.com/us/channel/usa-today/id6442586336. This is not the

intended Channel but the resulted channel does have content that is provided by USA Today/Wondery, which
is relevant but likely to poorly satisfy a secondary intent and can be rated Acceptable.
KSL Podcasts channel result - https://podcasts.apple.com/us/channel/ksl-podcasts/id6442569421. This is
not the intended Channel but the resulted channel does have one show that is provided by KSL
Podcasts/Wondery, which is relevant but likely to poorly satisfy a secondary intent and can be rated
Acceptable.

Queries without singular, navigational results

In the Podcasts Search Evaluation, there are multiple query types without a singular navigational result
available. These query types include: Ambiguous, Browse, Category, Podcast Topic/Keyword, Provider,
Podcast Host, and Person.

Unlike Navigational queries, users are not searching for a singular specific Podcast, Podcast Episode, or
Channel. Instead, multiple pieces of content can satisfy the users’ primary intent and/or secondary intents.

https://baseline.apple.com/training/evaluations/1036/guidelines 15/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine

Rating Scale for Queries without a singular, navigational result

The highest applicable rating for non-navigational query results is Excellent.

Excellent: A piece of content which clearly satisfies a likely primary intent of the query.
Good: A piece of content which only partially satisfies the intent of the query, but is still high-quality
OR satisfies the intent of the query but is lower quality
OR satisfies a secondary intent of the query.
Acceptable: A piece of content that is related to the query, but may be of poor quality, not popular, or poorly
satisfies a secondary intent of the query.
Unacceptable: Off-Topic: A piece of content that does not satisfy any of the above conditions.
Unacceptable: Misleading - This rating should be used for content that is likely to intentionally mislead users
about its authorship and/or non-podcast content.
Problem: Other: A problem with the task in BaseLine, for example the result image thumbnail and the result
show/episode title are missing, making it impossible to rate the result.

PODCAST HOST
The highest applicable rating for this query type is Excellent.
For Podcast Host queries shows and episodes hosted by the intended person are eligible for Excellent.
Content that is likely to satisfy a secondary intent should be rated Good or Acceptable based on popularity and
quality.

Query: [lex fridman]

Excellent - for example episodes and shows by the intended host. Note: Podcast Shows that are still host by
the intended Podcast Host should also receive an Excellent rating.

#272 – Brett Johnson: US Most Wanted Cybercriminal, an episode of the Lex Fridman Podcast -
https://podcasts.apple.com/us/podcast/272-brett-johnson-us-most-wanted-cybercriminal/id1434243584?
i=1000555373059. This is an episode hosted by the intended podcast host, and will be rated Excellent.
The Lex Fridman Podcast by Lex Fridman https://podcasts.apple.com/us/podcast/lex-fridman-
podcast/id1434243584. “Conversations about science, technology, history, philosophy and the nature of
intelligence, consciousness, love, and power. Lex is an AI researcher at MIT and beyond.” Category:
Technology. This result is a podcast show hosted by the intended host, and will be rated Excellent.

Good - a piece of content which only partially satisfies the intent of the query, but is still high-quality, or satisfies
the intent of the query but is lower quality, or satisfies a secondary intent of the query. Note: Podcast Shows
hosted by the intended Podcast Host but not anymore should also receive a Good rating ( Please be mindful of
how long ago the host stopped contributing to the podcast show.)

An episode where Lex Fridman is the guest on Good For You podcast -
https://podcasts.apple.com/us/podcast/lex-fridman/id1485527143?i=1000494688953. “Lex and Whitney
discuss love and robots. Follow Lex Fridman everywhere: @lexfridman”.
Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas by Sean Carroll | Wondery -
https://podcasts.apple.com/us/podcast/sean-carrolls-mindscape-science-society-philosophy/id1406534739.
“Each week, Sean Carroll will host conversations with some of the most interesting thinkers in the world.
From neuroscientists and engineers to authors and television producers, Sean and his guests talk about the
biggest ideas in science, philosophy, culture and much more.” Category: Science. This podcast is very similar
to the intended podcast and likely to satisfy a secondary intent of the user, rated Good.

Acceptable - a piece of content that is related to the query (i.e. keyword/title matching), but may be of poor
quality, not popular, or poorly satisfies a secondary intent of the query

Huberman Lab podcast by Scicomm Media - https://podcasts.apple.com/us/podcast/huberman-

lab/id1545953110. Category: Health & Fitness. “The Huberman Lab Podcast discusses neuroscience: how
our brain and its connections with the organs of our body control our perceptions, our behaviors, and our
health.” The intended podcast by Lex Fridman is mainly focused on science, technology, and intelligence,
while the Huberman Lab podcast is on the topic of neuroscience, which is likely relevant to a much smaller
group of listeners. Likely to satisfy a distant secondary intent, rated Acceptable.
The Good For You comedy podcast with Whitney Cummings - https://podcasts.apple.com/us/podcast/good-
for-you/id1485527143. Lex Fridman was a guest on this show, on episode #50. Slightly relevant to the query,
rate as Acceptable.
Lex Fridman Podcast | 5 minute podcast summaries - https://podcasts.apple.com/us/podcast/lex-fridman-
podcast-5-minute-podcast-summaries/id1567129969. This is an unpopular summary podcast of the show
hosted by Lex Fridman, and the episode are one 5 mins long, likely to only poorly satisfy a secondary intent
of the user, rated Acceptable.
OwlTail Channel- with 5minute podcast summaries podcasts -
https://podcasts.apple.com/us/channel/5minute-podcast-summaries/id6442481394. This is a podcast
channel result containing 5 minute podcast show summarizing the Lex Fridman podcast. Acceptable
relevance.

Unacceptable: Off-Topic - result with no apparent relevance to the input

Anything Ghost Show by Lex Wahl - https://podcasts.apple.com/us/podcast/anything-ghost-

show/id301683725. Category: Religion & Spirituality. “Since 2006, Anything Ghost has been sharing people's
personal paranormal experiences.” Unrelated to the query and user intent, rate as Unacceptable: Off-Topic.
Episode Jennette McCurdy from The Good For You comedy podcast with Whitney Cummings show -
https://podcasts.apple.com/us/podcast/jennette-mccurdy/id1485527143?i=1000578669060. This episode is
unrelated to Lex Fridman, and it should be rated as Unacceptable: Off-Topic.

Query: [georgia hardstark ]

Excellent - for example episodes and shows by the intended host. Note: Podcast Shows that are still hosted
by the intended Podcast Host should also receive an Excellent rating.

My Favorite Murder with Karen Kilgariff and Georgia Hardstark . Category: True Crime. This result is a
podcast show currently being co-hosted by the intended host, and will be rated Excellent.

Good - a piece of content which only partially satisfies the intent of the query, but is still high-quality, or satisfies
the intent of the query but is lower quality, or satisfies a secondary intent of the query. Note: Podcast Shows
hosted by the intended Podcast Host but not anymore should also receive a Good rating ( Please be mindful of
how long ago the host stopped contributing to the podcast show.)

https://baseline.apple.com/training/evaluations/1036/guidelines 16/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine
Slumber Party With Alie and Georgia . Category: Comedy. This result is a podcast show that was co-hosted
by the intended host, and will be rated Good.

Query: [deepak chopra]

Acceptable - a piece of content that is related to the query (i.e. keyword/title matching), but may be of poor
quality, not popular, or poorly satisfies a secondary intent of the query

iHeartPodcasts Channel - https://podcasts.apple.com/us/channel/iheartpodcasts/id6442482239. Deepak

Chopra is a guest on the podcast show Dear Chelsea, episode "Meditation with Deepak Chopra", from the
iHeartPodcasts Channel. This Channel result is likely to only satisfy a distant secondary intent of most users,
but it is still slightly relevant. Rate as Acceptable.

Query: [joe rogan]

Good - a piece of content which only partially satisfies the intent of the query, but is still high-quality, or satisfies
the intent of the query but is lower quality, or satisfies a secondary intent of the query. Note: Podcast Shows
hosted by the intended Podcast Host but not anymore should also receive a Good rating ( Please be mindful of
how long ago the host stopped contributing to the podcast show.)

The Joe Rogan Experience Experience - https://podcasts.apple.com/us/podcast/the-joe-rogan-experience-

experience/id1441095668. This is a podcast that is a breakdown or review of the podcast The Joe Rogan
Experience by host Joe Rogan. Satisfies a secondary intent.

Query: [bailey sarian]

Good - a piece of content which only partially satisfies the intent of the query, but is still high-quality, or satisfies
the intent of the query but is lower quality, or satisfies a secondary intent of the query. Note: Podcast Shows
hosted by the intended Podcast Host but not anymore should also receive a Good rating ( Please be mindful of
how long ago the host stopped contributing to the podcast show.)

Audioboom Studios - https://podcasts.apple.com/us/channel/audioboom-studios/id6442683786. This is a

channel containing the podcast hosted by the intended host. Satisfies a secondary intent.

Query: [dax shepard]

Excellent - for example episodes and shows by the intended host. Note: Podcast Shows that are still hosted
by the intended Podcast Host should also receive an Excellent rating.

The Podcast Armchair Expert Umbrella with Dax Shepard - https://podcasts.apple.com/us/podcast/armchair-

expert-umbrella-with-dax-shepard/id1345682353
An episode from Armchair Expert Umbrella with Dax Shepard: “We are supported by... Sheryl
Sandberg”https://podcasts.apple.com/us/podcast/we-are-supported-by-sheryl-sandberg/id1345682353?
i=1000533962603

Good - a piece of content which only partially satisfies the intent of the query, but is still high-quality, or satisfies
the intent of the query but is lower quality, or satisfies a secondary intent of the query. Note: Podcast Shows
hosted by the intended Podcast Host but not anymore should also receive a Good rating ( Please be mindful of
how long ago the host stopped contributing to the podcast show.)

Dax Shepard and Kristen Bell: Meet the Filmmakers, from Events at the Apple Store -
https://podcasts.apple.com/us/podcast/dax-shepard-and-kristen-bell-meet-the-filmmakers/id549867516. "Join
comic talent Dax Shepard and actor Kristen Bell as they discuss their new film "Hit and Run", written and
directed by Shepard." This is an interview with Dax Shepard from 2012, and likely to only partially satisfy the
user intent.

Acceptable - a piece of content that is related to the query (i.e. keyword/title matching), but may be of poor
quality, not popular, or poorly satisfies a secondary intent of the query

The Podcast episode Loyal Dax Shepard Fans Are Selling Out Theaters, from School of
Podcastinghttps://podcasts.apple.com/us/podcast/loyal-dax-shepard-fans-are-selling-out-
theaters/id83653087?i=1000425928325 "Tryo Heinritz attended a live event from Dax Shepard, and David
Hooper gives us behind the scenes of hosting an event." Only slightly relevant to the query intent, rate as
Acceptable.

Query: [adam grant]

Excellent - for example episodes and shows by the intended host. Note: Podcast Shows that are still hosted
by the intended Podcast Host should also receive an Excellent rating.

WorkLife with Adam Grant https://podcasts.apple.com/us/podcast/worklife-with-adam-grant/id1346314086.

WorkLife is a podcast hosted by Adam Grant, rate as Excellent.

Good - a piece of content which only partially satisfies the intent of the query, but is still high-quality, or satisfies
the intent of the query but is lower quality, or satisfies a secondary intent of the query. Note: Podcast Shows
hosted by the intended Podcast Host but not anymore should also receive a Good rating ( Please be mindful of
how long ago the host stopped contributing to the podcast show.)

#22 Adam Grant: Givers, Takers, and the Resilient Mind https://podcasts.apple.com/us/podcast/22-adam-
grant-givers-takers-and-the-resilient-mind/id990149481?i=1000392517331. This is an episode from The
Knowledge Project hosted by Shane Parrish, where Adam Grant is a special guest. Rate as Good.

Query: [karen kilgariff]

Excellent - for example episodes and shows by the intended host. Note: Podcast Shows that are still hosted
by the intended Podcast Host should also receive an Excellent rating.

My Favorite Murder with Karen Kilgariff and Georgia Hardstark Exactly

Righthttps://podcasts.apple.com/us/podcast/my-favorite-murder-with-karen-kilgariff-and/id1074507850. A
podcast hosted by Karen Kilgariff, rated Excellent.
321 - Tenfold More Murder: Part 1, an episode of My Favorite Murder -
https://podcasts.apple.com/us/podcast/321-tenfold-more-murder-part-1/id1074507850?i=1000556500355.
Do You Need A Ride? Exactly Right https://podcasts.apple.com/us/podcast/do-you-need-a-ride/id869415596.
A podcast hosted by Karen Kilgariff, rated Excellent.

https://baseline.apple.com/training/evaluations/1036/guidelines 17/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine
Good - a piece of content which only partially satisfies the intent of the query, but is still high-quality, or satisfies
the intent of the query but is lower quality, or satisfies a secondary intent of the query. Note: Podcast Shows
hosted by the intended Podcast Host but not anymore should also receive a Good rating ( Please be mindful of
how long ago the host stopped contributing to the podcast show.)

Karen Kilgariff, an episode from the Never Not Funny: The Jimmy Pardo
Podcasthttps://podcasts.apple.com/us/podcast/karen-kilgariff/id142348028?i=1000487257073. “Never Not
Funny is an award-winning weekly podcast hosted by comedian and TV personality Jimmy Pardo and
producer Matt Belknap. Since 2006, the show has offered comedy fans the unique opportunity to eavesdrop
on freewheeling conversations between Pardo and his sometimes famous, always funny friends.” This is an
episode where Karen Kilgariff is a guest.

AMBIGUOUS
Ambiguous queries are simply too vague to identify a clear primary intent, and are often just a few letters long:
[an], [c], [bill]. The highest applicable rating for this query type is Excellent. Results for Ambiguous queries
need to be rated based on relevance, popularity, and quality.

Tip: For ambiguous queries, enter the query in the search field in Apple Podcasts to see suggested search
hints and most likely user intents.

Query: [emma]
This query is very ambiguous, and the user could be searching for a podcast host (Emma Chamberlain), a
topic (the book “Emma” by Jane Austen), or possibly a person (Emma Stone, Emma Thomson, etc.).

Excellent - a piece of content which clearly satisfies a likely primary intent of the query

The podcast Stupid Genius with Emma Chamberlain - https://podcasts.apple.com/us/podcast/stupid-genius-

with-emma-chamberlain/id1458568923 - A very popular Podcast matching the query. Note: Any episode of
Stupid Genius with Emma Chamberlain will also get a rating of Excellent.

Good - a piece of content that only partially satisfies the intent of the query, but is still high-quality or satisfies
the intent of the query but is lower quality or has near text match WITH high-quality or satisfies a secondary
intent of the query

The Podcast Emma by Jane Austen by Loyal Books - https://podcasts.apple.com/us/podcast/emma-by-jane-

austen/id729067901 - A not so popular podcast about Jane Austen's book 'Emma'. Note: Any episode of
Emma by Jane Austen will get a rating of Good as well.
The Podcast episode called 'Emma Stone' on Podtalk - https://podcasts.apple.com/us/podcast/emma-
stone/id1195190377?i=1000379945129- Podtalk is a popular Podcast, and this episode is relevant to the
query. However, note how Emma Stone is not actually in this episode, but rather interpreted by Carlie Craig.

Acceptable - a piece of content that is related to the query (i.e. keyword/title matching), but may be of poor
quality, not popular, or poorly satisfies a secondary intent of the query

The Podcast episode 'Solving the Climate Crisis, Preserving Biodiversity and the 6th Extinction with Emma
Marris' from Factually! with Adam Conover by Earwolf- https://podcasts.apple.com/us/podcast/solving-
climate-crisis-preserving-biodiversity-6th/id1463460577?i=1000444652474 - Emma Marris is not a
mainstream popular person.

Unacceptable: Off-Topic - result with no apparent relevance to the input

Laughter Permitted with Julie Foudy by ESPN, Julie Foudy - https://podcasts.apple.com/us/podcast/laughter-

permitted-with-julie-foudy/id1455251280. Unrelated to the query, does not satisfy a primary nor secondary
intents of the user. Rate as Unacceptable: Off-Topic.

Query: [theres something about]

Acceptable - a piece of content that is related to the query (i.e. keyword/title matching), but may be of poor
quality, not popular, or poorly satisfies a secondary intent of the query

There's Something About Mary Carey https://podcasts.apple.com/us/podcast/theres-something-about-mary-

carey/id1608835689. This result is relevant through keyword/title match but also not popular or high quality,
rate as Acceptable.

BROWSE
Examples of browse queries include: [popular podcasts], [new], [top shows], [free], [best].
The highest applicable rating for this query type is Excellent. Results need to be rated based on relevance,
popularity, and quality.

Tip: For browse and category queries, check the "Browse" and "Browse → All Categories" sections in the Apple
Podcasts app to see What's Hot, New & Noteworthy, and Curated Collections.

Query: [podcasts]
This is a browse query as every show in the Apple Podcasts catalogue qualifies as a Podcast.

Excellent - a piece of content which clearly satisfies a likely primary intent of the query

The Podcast The Clearing by Pineapple Street Media / Gimlet - https://podcasts.apple.com/us/podcast/the-

clearing/id1471432747 - This Podcast is a popular show, and often present in the Top Charts (All
Categories).Note: Any episode of The Clearing will also get a rating of Excellent.
The Podcast Man In The Window: The Golden State Killer by L.A. Times | Wondery -
https://podcasts.apple.com/us/podcast/man-in-the-window-the-golden-state-killer/id1465334342 - This
Podcast has been a frequent Top Charter (All Categories). Note: Any episode of Man In The Window will get
a rating of Excellent too.

https://baseline.apple.com/training/evaluations/1036/guidelines 18/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine

Good - a piece of content that only partially satisfies the intent of the query, but is still high-quality or satisfies
the intent of the query but is lower quality or has near text match WITH high-quality or satisfies a secondary
intent of the query

The Podcast ¿Qué Pasa, Midwest? by WNIN - https://podcasts.apple.com/us/podcast/qué-pasa-

midwest/id1281688535 - "¿Qué Pasa, Midwest? is a bilingual Podcast that creates a sense of community for
Midwestern Latinx who are missing an essential piece of their cultural identity". This result is quite specific
and more narrow in its target audience; a bilingual Podcast (when the query is not mentioning bilingual), and
it is focused on Midwest. Likely to satisfy a secondary intent. Note: Any episode of ¿Qué Pasa, Midwest? will
get a rating of Good too.

Acceptable - a piece of content that is related to the query (i.e. keyword/title matching), but may be of poor
quality, not popular, or poorly satisfies a secondary intent of the query

The Podcast Kid Tested, Mother Approved - https://podcasts.apple.com/us/podcast/kid-tested-mother-

approved/id1239120206 - An unpopular, and amateur Podcast about movies. Note: Any episode of Kid
Tested, Mother Approved will get a rating of Acceptable too.

CATEGORY

Category queries are: [arts], [fiction], [history], [true crime], [business], [comedy], [education], [games &
hobbies], [government], [health], [kids & family], [music], [news], [religion & spirituality], [science], [society &
culture], [sports], [technology], and [TV & film].
The highest applicable rating for this query type is Excellent. Results need to be rated based on relevance,
popularity, and quality.

Tip: For browse and category queries, check the "Browse" and "Browse → All Categories" sections in the Apple
Podcasts app to see What's Hot, New & Noteworthy, and Curated Collections.

Query: [science]
Excellent - for example high quality shows, episode, channels satisfying primary intent of the category

The podcast Making Sense with Sam Harris - https://podcasts.apple.com/us/podcast/making-sense-with-

sam-harris/id733163012 - An extremely popular Science & Medicine show. Note: Any episode of Making
Sense with Sam Harris will get a rating of Excellent too.
The podcast Zombified: A production of ASU and Zombie Apocalypse Medicine -
https://podcasts.apple.com/us/podcast/zombified-production-asu-zombie-apocalypse-
medicine/id1467459411 - A less popular, high-quality science Podcast. Note: Any episode of Zombified will
get a rating of Excellent too.

Good - for example lower quality and/or less relevant shows, episodes or channels of the intended category or
satisfying secondary intent, or has near text match WITH high-quality

The podcastTumble Science Podcast for Kids - https://podcasts.apple.com/us/podcast/tumble-science-

podcast-for-kids/id984771479 - For the query [science], a podcast for kids satisfies a secondary intent,
unless the user specifies that they're looking for content for children. Note: Any episode of Tumble Science
will get a rating of Good too.

Acceptable - a piece of content that is related to the query (i.e. keyword/title matching), but may be of poor
quality, not popular, or poorly satisfies a secondary intent of the query

The podcast Clinical PET Cast by Peter Faulhaber, MD - https://podcasts.apple.com/us/podcast/clinical-pet-

cast/id337947389 - This is a piece of content that is related to science, but is unpopular, and only relevant to
small group of users, namely radiologists and other medical professionals who want to know more about PET
imaging. Note: Any episode of Clinical PET Cast will get a rating of Acceptable too.

Unacceptable: Off-Topic - result with no apparent relevance to the input

The podcast Getting Curious with Jonathan Van Ness - https://podcasts.apple.com/us/podcast/getting-

curious-with-jonathan-van-ness/id1068563276". A weekly exploration of all the things Jonathan Van Ness
(Queer Eye, Gay of Thrones) is curious about." Category: Society & Culture. Note: Any episode of Getting
Curious will get a rating of Off-Topic too.

https://baseline.apple.com/training/evaluations/1036/guidelines 19/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine

Query: [true crime]

Excellent - for example high quality shows, episode, channels satisfying primary intent of the category

Dateline NBC - https://podcasts.apple.com/us/channel/dateline-nbc/id6442484494. The Dateline NBC

Podcast Channel includes true crime mysteries from The True Crime Original: "Listen to current and classic
episodes, plus original, in-depth investigative series told only as Dateline can". This result is popular and very
likely to satisfy a primary intent of the user and should be rated Excellent.
CBC True Crime - https://podcasts.apple.com/us/channel/cbc-true-crime/id6442568309. This Podcast
Channel is dedicated to true crime podcasts. It contains several popular podcasts that satisfy a primary
intent. This Channel would be eligible for Excellent.
Nikolas Cruz’s death penalty hearing; Kristin Smart murder trial delays — TCD Sidebar -
https://podcasts.apple.com/us/podcast/nikolas-cruzs-death-penalty-hearing-kristin-smart-
murder/id1451999167?i=1000574741888. This episode is from a high quality, popular podcast dedicated to
true crime and likely to satisfy a primary intent of the user and should be rated Excellent. Note: we can
determine popularity byreferencing to the podcast show page and by doing side research.

Good - for example lower quality and/or less relevant shows, episodes or channels of the intended category or
satisfying secondary intent, or has near text match WITH high-quality

The Channel Podshape True Crime - https://podcasts.apple.com/us/channel/podshape-true-

crime/id6442586726. This is an Australian true crime story channel, less relevant to the listeners in the US
storefront, and hence likely to satisfy a secondary intent of most users. Rated Good. (Note: if the storefront
was AU, this would likely satisfy primary intent and be given Excellent rating.).
ABC News - https://podcasts.apple.com/us/channel/abc-news/id6442507316. This Podcast Channel contains
popular true crime podcasts, but also contains other podcasts covering different topics. Since this Channel is
not exclusively for true crime, the best rating is Good.

Query: [art]
Acceptable - a piece of content that is related to the query (i.e. keyword/title matching), but may be of poor
quality, not popular, or poorly satisfies a secondary intent of the query

The Podcast Fresh Air - https://podcasts.apple.com/us/podcast/fresh-air/id214089682 The podcast "Fresh

Air" is in the Arts Category, but not in a ‘traditional’ sense. Has some relevance to the query, but it's not the
best result we could return.
The Podcast The Moth - https://podcasts.apple.com/us/podcast/the-moth/id275699983 The podcast "The
Moth | The Art and Craft of Storytelling" is in the Arts Category. Has some relevance to the query, but it's not
the best result we could return.

Query: [news]
Excellent - for example high quality shows, episode, channels satisfying primary intent of the category

The Podcast The Report - https://podcasts.apple.com/us/podcast/the-report/id1472798169 - This Podcast is

currently number 1 in the News Top Charts. Note: Any episode of The Report will get a rating of Excellent too
The Podcast The Daily by The New York Times - https://podcasts.apple.com/us/podcast/the-
daily/id1200361736- Very popular Podcast by highly regarded newspaper The New York Times. Note: Any
episode of The Daily will get a rating of Excellent too.

Good - for example lower quality and/or less relevant shows, episodes or channels of the intended category or
satisfying secondary intent, or has near text match WITH high-quality

The Podcast Wait Wait... Don't Tell Me! by NPR - https://podcasts.apple.com/us/podcast/wait-wait-dont-tell-

me/id121493804 - On the program, panelists and contestants are quizzed in humorous ways about that
week's news. Category: Comedy; but this is a news quizz (relevant to the query). Likely to satisfy a
secondary intent.Note: Any episode of Wait Wait... Don't Tell Me! will get a rating of Good too.
The Podcast English News - NHK WORLD RADIO JAPAN by NHK-
https://podcasts.apple.com/us/podcast/english-news-nhk-world-radio-japan/id141016660 - This is a news
Podcast, but about news in Japan. Note: Any episode of English News will get a rating of Good too.

Unacceptable: Off-Topic - result with no apparent relevance to the input

The Podcast Journeys of Faith with Paula Faris by ABC News-

https://podcasts.apple.com/us/podcast/journeys-of-faith-with-paula-faris/id1440648196- This is not a news
Podcast. Keyword match on "News" from ABC News. Note: Any episode of Journeys of Faith will get a rating
of Off-Topic too.

PODCAST TOPIC/KEYWORD
Podcast Topic/Keyword queries are slightly more specific than Ambiguous, Browse, and Category queries.
These queries do not seem to point to a particular show, episode, or podcast channel; instead, users are likely
wanting to surface popular Podcasts and Podcast episodes that cover the topics and/or keywords entered e.g.
[michelle obama book], [hunting], [scary], [nba], [cooking].

The highest applicable rating for this query type is Excellent. Results need to be rated based on relevance,
popularity, and quality.

Query: [game of thrones]

Excellent - for example high quality shows and episodes satisfying primary intent of the topic/keyword

The Podcast Game of Thrones The Podcast by Bald Move - https://podcasts.apple.com/us/podcast/game-of-

thrones-the-podcast/id508647267- Very popular Podcast dedicated to the TV Show. Note: Any episode of
Game of Thrones The Podcast will get a rating of Excellent too.
The Podcast Bend the Knee: A Song of Ice and Fire Podcast - https://podcasts.apple.com/us/podcast/bend-
the-knee-a-song-of-ice-and-fire-podcast/id1273613024 - A less popular Podcast dedicated to the TV Show.
Note: Any episode of Bend the Knee will get a rating of Excellent too.

https://baseline.apple.com/training/evaluations/1036/guidelines 20/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine

Good - a piece of content that only partially satisfies the intent of the query, but is still high-quality or satisfies
the intent of the query but is lower quality or or has near text match WITH high-quality or satisfies a secondary
intent of the query

The Podcast The Joffrey of Podcasts: Game of Thrones by Double P Media -

https://podcasts.apple.com/us/podcast/the-joffrey-of-podcasts-game-of-thrones/id676876079 - This is a
lesser known Podcast dedicated to a specific Game of Thrones character. Note: Any episode of The Joffrey
of Podcasts will get a rating of Good too.

Query: [asmr]
ASMR stands for autonomous sensory meridian response. It's a feeling of well-being combined with a tingling
sensation in the scalp and down the back of the neck, as experienced by some people in response to a specific
gentle stimulus, often a particular sound like whispering voices, paper tearing, and scalp massage.

Excellent - for example high quality shows and episodes satisfying primary intent of the topic/keyword

The Podcast Sweet Whispers Sensual ASMR Podcast - https://podcasts.apple.com/us/podcast/sweet-

whispers-sensual-asmr-podcast/id1036294870 - A popular Podcast dedicated to ASMR. Note: Any episode
of Sweet Whispers Sensual ASMR Podcast should get a rating of Excellent too.

Good - a piece of content that only partially satisfies the intent of the query, but is still high-quality or satisfies
the intent of the query but is lower quality or or has near text match WITH high-quality or satisfies a secondary
intent of the query

The Podcast Sleep Meditation Podcast - Relaxing Nature Sounds and Ambient Music for Sleep - ASMR
Sleep Triggers - https://podcasts.apple.com/us/podcast/sleep-meditation-podcast-relaxing-nature-sounds-
ambient/id1365312021 - This Podcast is not entirely dedicated to ASMR. It also offers some meditation, rain
sounds, etc. Likely to satisfy a secondary intent. Note: Any episode of Sleep Meditation Podcast should get a
rating of Good too.

Acceptable - a piece of content that is related to the query (i.e. keyword/title matching), but may be of poor
quality, not popular, or poorly satisfies a secondary intent of the query

An episode called ASMR by Positiviteeny! - https://podcasts.apple.com/us/podcast/asmr/id1431547509?

i=1000428457044- Hosts Travis, Amy and Erika discuss ASMR. Note: This is not a popular Podcast, the
show is rated Unacceptable: Off-Topic for this query and any podcast episode of Positiviteeny! not dedicated
to ASMRs should get a rating of Off-Topic.

Unacceptable: Off-Topic - result with no apparent relevance to the input

The Dutch language Podcast Nachtkastje https://podcasts.apple.com/us/podcast/nachtkastje/id1434872441

is not relevant to the majority of the US storefront listeners and should be rated as Unacceptable: Off-Topic.

Query: [mafia]

Good - a piece of content that only partially satisfies the intent of the query, but is still high-quality or satisfies
the intent of the query but is lower quality or or has near text match WITH high-quality or satisfies a secondary
intent of the query

The Audioboom Studios Podcast Channel - https://podcasts.apple.com/us/channel/audioboom-

studios/id6442683786. This Channel does not immediately have clear connection between query and
Channel result, but a Google Search on “Audioboom Studios podcast apple mafia” will surface the
connection for a matching podcast called “MAFIA” by Audiboom Studios. This satisfies a secondary intent
and should be rated Good.

PROVIDER
A query that refers to a Podcast provider by name; [npr], [ESPN], [crooked media], [the new york times]. The
highest applicable rating for this query type is Excellent.

The shows and episodes by the intended provider will be rated Excellent, but there are a few exceptions.

https://baseline.apple.com/training/evaluations/1036/guidelines 21/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine

Query: [npr]
Excellent - for example shows and episodes by the intended provider

The Podcast Fresh Air by NPR. A piece of content which very clearly satisfies the intent of the
query.https://podcasts.apple.com/us/podcast/fresh-air/id214089682.
Note: Any episode of Fresh Air should get a rating of Excellent.
The Podcast Planet Money by NPR -https://podcasts.apple.com/us/podcast/planet-money/id290783428.
Note: Any episode of Planet Money should get a rating of Excellent too.

Good - a piece of content which only partially satisfies the intent of the query, but is still high-quality, or satisfies
the intent of the query but is lower quality, or satisfies a secondary intent of the query

The Podcast Education Desk Podcast by NPR Illinois, 91.9 UIS -

https://podcasts.apple.com/us/podcast/education-desk-podcast-npr-illinois-91-9-uis/id1156694316. This
Podcast is likely only relevant to a smaller group of users, and therefore it satisfies a secondary intent of the
query.
Note: Any episode of Education Desk Podcast will get a rating of Good too.

Query: [crooked media]

Excellent - for example shows and episodes by the intended provider

The Podcast Pod Save America by Crooked Media - A piece of content which very clearly satisfies the intent
of the query https://podcasts.apple.com/us/podcast/pod-save-america/id1192761536.
Note: Any episode of Pod Save America should get a rating of Excellent too.
The Podcast With Friends Like These by Crooked Media - https://podcasts.apple.com/us/podcast/with-
friends-like-these/id1207507389.
Note: Any episode of With Friends Like These should get a rating of Excellent too.

Query: [domino magazine]

Excellent - for example shows and episodes by the intended provider

The Podcast Design Time . A piece of content which very clearly satisfies the intent of the query.
Note: Any episode of Design Time should get a rating of Excellent.

Good - a piece of content which only partially satisfies the intent of the query, but is still high-quality, or satisfies
the intent of the query but is lower quality, or satisfies a secondary intent of the query

The Podcast show How to Decorate is from a different provider with content similar to the intended
provider. It would satisfy a secondary intent.
Note: episodes from How to Decorate ( secondary intent show) would receive Acceptable rating at most.

PERSON
A query that refers to a person who does not host a Podcast of their own, nor has a strong association with
one; [martin short], [donald trump], [meryl streep].

The highest applicable rating for this query type is Excellent. Results need to be rated based on relevance,
popularity, and quality.

Query: [martin short]

Excellent - a piece of content which clearly satisfies a likely primary intent of the query

The Podcast episode Martin Short on Conan O'Brien Needs A Friend -

https://podcasts.apple.com/us/podcast/30-martin-short/id1438054347?i=1000441022213 This is an episode
featuring the queried for person: "Martin and Conan sit down to chat about the best time to be 30, the
production of Godspell that started it all, how Martin's Irish heritage shaped his comedy, doing Bette Davis in
front of Bette Davis, and which character Martin would like to be buried as. Plus, Conan and his team
deconstruct the family dynamic of the show". Note: The Podcast Conan O'Brien Needs A Friend will get a
rating of Good.

Good - a piece of content which only partially satisfies the intent of the query, but is still high-quality, or satisfies
the intent of the query but is lower quality, or satisfies a secondary intent of the query

Episode 50 | Why is Martin Short Funny? by Why Am I Laughing? -

https://podcasts.apple.com/us/podcast/episode-50-why-is-martin-short-funny/id1154539484?
i=1000394829620 - This is a Podcast in which Martin Short's comedic skills are discussed.

Unacceptable: Off-Topic - result with no apparent relevance to the input

The Podcast episode "SHORT: Farming chickens, pigs, and hunting... How homesteaders can stop buying
meat from the supermarket" by Austin Martin- https://podcasts.apple.com/us/podcast/short-farming-chickens-
pigs-hunting-how-homesteaders/id879310835?i=1000318468040 - this result episode is returned due to the
keyword matches on "short" and "martin", unrelated to the query intent. Note: The Podcast Homesteady will
get a rating of Off-Topic too.

Query: [anthony joshua]

Excellent - a piece of content which clearly satisfies a likely primary intent of the query

The Podcast episode Hour 2: Anthony Joshua on First Take - an episode on which the intended person is
interviewed - https://podcasts.apple.com/us/podcast/hour-2-anthony-joshua/id428989929?
i=1000439962278". Unified heavyweight champion Anthony Joshua joins the show." → The intended person
is interviewed on this returned episode.

Good - a piece of content which only partially satisfies the intent of the query, but is still high-quality, or satisfies

https://baseline.apple.com/training/evaluations/1036/guidelines 22/23
9/5/23, 10:20 AM Guidelines for Search - Podcasts Top Results Training — BaseLine
the intent of the query but is lower quality, or satisfies a secondary intent of the query

First Take by ESPN - https://podcasts.apple.com/us/podcast/first-take/id428989929 - This is the sports

Podcast First Take by ESPN, and Anthony Joshua was interviewed on one of the episodes of the show, see
the episode result above.
Tha Boxing Voice Podcast, by ThaBoxingvoice Radio - https://podcasts.apple.com/us/podcast/tha-boxing-
voice/id596622478. "Tha Boxing Voice Radio Show provides boxing discussion every Thursday & Sunday at
7pm EasternTime for the fans by the fans." There are several episodes on the topic of Anthony Joshua on
thisshow.

Acceptable - a piece of content that is related to the query (i.e. keyword/title matching), but may be of poor
quality, not popular, or poorly satisfies a secondary intent of the query

Episode Hour 2: NFC Least from First Take by ESPN - https://podcasts.apple.com/us/podcast/hour-2-nfc-

least/id428989929?i=1000579224030- This is a different episode from First Take by ESPN, where Anthony
Joshua is not mentioned or interviewed. However, the show does have an episode where Anthony Joshua is
appearing as a guest, see the show result above. Therefore other episodes from this show will be rated
Acceptable.

Query: [alastair reynolds]

Acceptable - a piece of content that is related to the query (i.e. keyword/title matching), but may be of poor
quality, not popular, or poorly satisfies a secondary intent of the query

The Podcast episode 68. Alastair Reynolds / Philip K. Dick Movies -

https://podcasts.apple.com/us/podcast/68-alastair-reynolds-philip-k-dick-movies/id395738416?
i=1000120252422 - This is an episode where the hosts only talk about the movies made from books by the
intended person, but is from 2012 which is making this less relevant and could satisfy a secondary intent of
the query. Rate as Acceptable. Note: As the show is not unpopular and relevant to sci-fi, it can also be rated
Acceptable.

Copyright © 2023 Apple Inc. All rights reserved. Apple Confidential.

https://baseline.apple.com/training/evaluations/1036/guidelines 23/23