Scribd
Upload
0 views

AIcrowd _ Multi-turn QA _ Challenges

The Meta CRAG-MM Challenge 2025 invites participants to build persona-rich NPCs and offers $20,000 in prizes. The challenge includes tasks such as single-source augmentation and multi-turn QA systems, focusing on generating answers from images and maintaining context in conversations. Participants can access resources and engage with the community via Discord and the official challenge website.

Uploaded by

Gobi
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
0 views

AIcrowd _ Multi-turn QA _ Challenges

The Meta CRAG-MM Challenge 2025 invites participants to build persona-rich NPCs and offers $20,000 in prizes. The challenge includes tasks such as single-source augmentation and multi-turn QA systems, focusing on generating answers from images and maintaining context in conversations. Participants can access resources and engage with the community via Discord and the official challenge website.

Uploaded by

Gobi
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 1

🙋 Build NPCs you’d talk to. Create persona-rich, memory-aware, and ready-to-act agents. $20,000 in prizes!

✨ ×
Search... Challenges Log In Sign Up
 Back to Meta CRAG - MM Challenge 2025

Warm-Up Round: Completed Round 1 16 days left Weight: 1.0

By Meta 1613 638 174 73 2 Share

Overview Leaderboard Discussion Insights Resources Submissions Rules Participate

New to the challenge? 🤔 Want to make your first submission?


⚙ Access the Starter-Kit here 📚 Check out the official CRAG-MM v0.1.1 Dataset here.
💬 Join the conversation on Discord – connect with other participants, get support, and stay
updated. Jump in and introduce yourself 👉 https://discord.gg/YWDQQa8byx

An MM-RAG QA system takes as input an image 𝐼 and a question 𝑄, and outputs an answer
𝐴; the answer is generated by MM-LLMs according to information retrieved from external
sources, combined with knowledge internalized in the model. A Multi-turn MM-RAG QA
system in addition takes questions and answers from previous turns as context to answer
new questions. The answer should provide useful information to answer the question,
without adding any hallucination.
TASK #1: SINGLE-SOURCE AUGMENTATION
In Task #1, we provide an image mock API to access information from an underlying
image-based mock KG. The mock KG is indexed by the image and stores structured data
associated with the image; answers to the questions may or may not exist in the mock KG.
The mock API takes an image as input and returns similar images from the mock KG along
with structured data associated with each image to support answer generation.
This task aims to test the answer generation capability of MM-RAG systems.

To know more about the Meta CRAG-MM challenge, please see:


https://www.aicrowd.com/challenges/meta-crag-mm-challenge-2025
📚 Explore the pre-release sample dataset now
💬 Join the conversation on Discord – connect with other participants, get support, and stay
updated.
👉 Jump in and introduce yourself: https://discord.gg/YWDQQa8byx
An MM-RAG QA system takes as input an image 𝐼 and a question 𝑄, and outputs an answer
𝐴; the answer is generated by MM-LLMs according to information retrieved from external
sources, combined with knowledge internalized in the model. A Multi-turn MM-RAG QA
system in addition takes questions and answers from previous turns as context to answer
new questions. The answer should provide useful information to answer the question,
without adding any hallucination.
TASK #1: SINGLE-SOURCE AUGMENTATION
In Task #1, we provide an image mock API to access information from an underlying
image-based mock KG. The mock KG is indexed by the image and stores structured data
associated with the image; answers to the questions may or may not exist in the mock KG.
The mock API takes an image as input and returns similar images from the mock KG along
with structured data associated with each image to support answer generation.
This task aims to test the answer generation capability of MM-RAG systems.

To know more about the Meta CRAG-MM challenge, please see:


https://www.aicrowd.com/challenges/meta-crag-mm-challenge-2025
An MM-RAG QA system takes as input an image 𝐼 and a question 𝑄, and outputs an answer
𝐴; the answer is generated by MM-LLMs according to information retrieved from external
sources, combined with knowledge internalized in the model. A Multi-turn MM-RAG QA
system in addition takes questions and answers from previous turns as context to answer
new questions. The answer should provide useful information to answer the question,
without adding any hallucination.
TASK #3: MULTI-TURN QA
Unlike Task #1 and Task #2, Task #3 evaluates the system’s ability to conduct multi-turn
conversations. Each conversation consists of 2–6 turns, with later questions may or may
not require the image for answering.
This task focuses on testing context understanding to ensure smooth multi-turn
interactions.
To know more about the Meta CRAG-MM challenge, please see:
https://www.aicrowd.com/challenges/meta-crag-mm-challenge-2025

PLATFORM CONNECT LEGAL


Challenges Blog Cookie consent
AIcrowd Research Forums Terms of Use
Organize a Challenge Contact Participation Terms
FAQ GitLab Privacy
Twitter Community Guidelines
Discord AI & Ethics
© 2025 AIcrowd. All rights reserved.

You might also like