INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,

PUNE

CONSUL
O
(YOUR PERSONAL
THERAPIST)
SUPERVISOR:
DR. PRIYANK
JAIN PRESENTED BY
BHUKYA RAMPRAKASH --
112215043
Mohd Zaid --
112215113
Aman Biswakarma --
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

INTRODUCTIO
N
2
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

INTRODUCTIO
Nyears, mental health issues have become a global
• In recent
concern, with stress, anxiety, and depression affecting millions
of individuals worldwide.
• Many individuals hesitate to seek help due to stigma, lack of
accessibility, or financial constraints. This results in prolonged
distress, affecting daily life, relationships, and productivity.
• Despite growing awareness, existing solutions remain
inadequate: long waiting times, high therapy costs, and social
barriers prevent people from getting the support they need.
The urgency to bridge this gap is greater than ever.
• Our project aims to redefine mental health support by
integrating AI-driven solutions that provide accessible,
personalized, and stigma-free assistance to those in need. 3
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

MOTIVATIO
N
4
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

MOTIVATION
• Rising mental health concerns – 1 in 5 people suffer; many
don’t get help.
• Barriers – Stigma, high costs, long waiting times.
• Technology as a solution – AI chatbots offer instant,
personalized support.

CONSULO’s unique approach:

• Combines text, voice, and video for better engagement.
• Encourages users to express, heal, and seek help.
• Mission – Bridging the treatment gap with AI-driven mental
health support.

5
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

MOTIVATION
• Statista's (A leading online
statistics portal) 2021
survey shows mental health
issues among Indian adults
by gender.
• As of October 2021, women
had the highest share of
mental health disorders in
India, amounting to 39
percent and 30 percent for
stress and anxiety health
disorder respectively.
Comparatively, 33 percent
of men had depression as FIGURE 1: SHARE OF MENTAL HEALTH DISORDERS AMONG INDIANS IN INDIA IN 2021,
compared to women with 31 BY GENDER
ACCORDING TO STATISTA (LEADING SURVEY STATISTICS)
percent during the same
time period.
6
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

LITERATURE
REVIEW

7
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

LITERATURE
REVIEW
Sr.
Title Methodology Used Research Review
No
The paper highlights SERMO as a promising AI-
This study involved a literature review and user discussions driven mental health tool, effectively integrating
to develop SERMO, an AI-powered chatbot designed for CBT-based interventions and emotion detection to
emotion regulation using Cognitive Behavioral Therapy support users in managing emotional distress, but
A Mental Health Chatbot for (CBT) principles. The chatbot employs Natural Language further clinical validation and AI refinement are
Regulating Emotions Processing (NLP) and a lexicon-based emotion detection necessary. Overall, SERMO demonstrates the
1 (SERMO) - Concept and algorithm to analyze German free-text input, identifying potential of AI in mental health support, but future
Usability Test emotions like fear, anger, sadness, and joy. SERMO follows improvements in machine learning-based emotion
the ABC model of CBT, where users describe Activating detection, personalization, and ethical safeguards
Events, Beliefs, and Consequences to regulate emotions will be key to wider adoption and clinical
through guided self-reflection[1]. integration.

The paper highlights MindBot as a significant

advancement in AI-driven mental health support. It
The study followed a structured approach to develop and
successfully integrates NLP and Large Language
evaluate MindBot, an AI-powered mental health chatbot.
Design and Evaluation of an Models (LLMs) to engage users in meaningful,
The process began with the data collection and Natural
AI-Powered Conversational empathetic conversations, helping them manage
Language Processing (NLP) techniques . For model
2 Agent for Personalized Mental
selection and training, NLTK’s Analyzer was employed for
their emotional states. However, the system has
Health Support and some limitations, including potential
sentiment classification, while GPT-3.5-turbo was
Intervention (MindBot) misinterpretation of complex emotions, privacy
integrated for dynamic response generation, ensuring
8
concerns regarding sensitive mental health data,
contextual and meaningful conversations[4].
and challenges in scalability and multilingual
support.
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

LITERATURE
REVIEW
Sr.
Title Methodology Used Research Review
No

The paper highlights MOODIFY AI-driven mental

The methodology of MOODIFY involves a structured
health application tailored for young adults
approach integrating facial emotion detection, after
facing emotional distress. Key strengths include
which Spotify API recommendations provide mood-
MOODIFY: Tailored, Personal and accurate emotion detection, an adaptive chatbot,
enhancing songs and podcasts. The AI chatbot,
Multifaceted AI Assistant for and a well-received anonymous chat feature.
3 Young Adult Mental Health
developed with TensorFlow Lite, engages in
However, the study acknowledges limitations
empathetic conversations, offering coping strategies
Issues. such as the need for enhanced NLP accuracy in
based on validated psychological tests such as the
chatbot interactions, ethical concerns regarding
Perceived Stress Scale (PSS) and General Health
data privacy, and potential biases in emotion
Questionnaire (GHQ).
recognition algorithms.

This study involves developing an AI-powered

chatbot using the Rasa framework and integrating a The study demonstrates that sentiment analysis
Long Short-Term Memory (LSTM) model for improves chatbot functionality by allowing it to
sentiment analysis. A dataset of 1.6 million posts detect emotions and provide tailored responses,
Integrating Sentiment Analysis from X (formerly Twitter) was collected and increasing the potential for real-time mental
4 to Enhance Mental Health preprocessed using Natural Language Processing health support. However, challenges remain,
Support Chatbots (NLP) techniques. The LSTM model was evaluated including misinterpretation of complex user
based on training accuracy (76%) and validation inputs, limited contextual understanding, and the
accuracy (76.9%), ensuring reliable sentiment need for improved accuracy in sentiment
detection. classification.
9
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

RESEARCH
GAP
The
The majority of the existing literature studies on AI-powered mental health
m: using NLP, sentiment analysis, and emotion detection to provide
chatbots
personalized support and therapy recommendations. However, these solutions
face limitations, including inaccurate emotional interpretation, lack of
personalization, ethical concerns, and poor context retention.

Us
• Our approach addresses the above limitations by utilizing the YOLOv11 model
: to capture facial emotions accurately, enabling a more personalized AI therapist
that enhances user engagement and satisfaction compared to traditional
chatbots.
• Additionally, our system integrates a voice assistant with personalized chat
responses tailored to the user’s detected emotions, making the interaction
more human-like and emotionally intuitive.

10
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

PROBLEM
STATEMEN
T
11
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

PROBLEM
STATEMENT
• Mental health issues are a growing global crisis,
affecting 1 in 5 people. Many individuals face barriers in
seeking help, including the high cost of therapy, stigma,
and long waiting times.

• Traditional mental health support is often limited,

inaccessible, and unavailable during moments of
immediate need, leaving people without proper
assistance when they need it most.

• Despite the urgency, there is a significant gap in

accessible, affordable, and personalized mental health 12
support.
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

SOLUTION
APPROACH

13
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

SOLUTION
APPROACH
CONSULO is an innovative AI-powered mental health therapist that
bridges the gap in accessible, affordable, and personalized support.
By integrating advanced technologies, CONSULO offers a
comprehensive mental health solution that is available anytime,
anywhere. It combines several key features to provide users with
immediate, empathetic support:

• GenAI Chatbot: Provides empathetic, text-based support

where users can freely express their feelings and receive
understanding and encouragement.
• Emotion Recognition: Uses video to analyze and detect
user emotions in real-time, tailoring responses without the
need for text or prompts.
14
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

SOLUTION
• VoiceAPPROACH
Assistance: Allows users to interact with CONSULO using
voice commands, enabling hands-free conversations and
immediate emotional support.
• Professional Help: Connects users to licensed therapists and
emergency services if professional intervention is needed, offering
direct access to support when necessary.
• Rehab Community (Talkbox): Offers a safe, anonymous space
for users to share their experiences and engage with others facing
similar challenges, helping reduce isolation.
This combination of features ensures that CONSULO provides a
personalized, stigma-free, and affordable mental health care
experience, empowering users to seek support, express their
emotions, and heal in a way that is both accessible and effective.
15
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

IMPLEMENTATIO
N AND
ARCHITECTURE
16
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

IMPLEMENTATIO
N
Frontend Implementation (React.js, WebRTC, OpenCV)-
• Emotion Detection Panel: React component with camera
access, real-time emotion analysis (%), DL model output
• Chat Interface: Dynamic message panel with text, video input
& voice recording. Use WebRTC/OpenCV for taking these input.

Backend Implementation (Node.js)-

• API: RESTful endpoints (sessions, preferences, history).
• WebSocket: Real-time chat Connection.
• Security: JWT(role-based access), rate limiting.
• Community: Provide a reddit/discord like community for
patients to share experiences, using Socket.io.
17
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

IMPLEMENTATIO
N
Database (MongoDB) -
• User Schema: Authentication, preferences, profile.
• Emotion Records: Tracks emotional states.

AI Integration (Flask, YOLO, Transformers) -

• Emotion Recognition: Detects facial expressions in video frames, then
YOLO model find the current emotion of the user.
• LLM Service: Generate contextual response according to user
request.

Emergency Protocol (NLP, Twilio)-

• Emergency protocol: NLP-based crisis detection, pattern
classification, severity classifier, hierarchical response, automated
alerts.
18
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

Techsta
ck

19
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

ARCHITECTUR
E

20
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

BENEFIT
S
21
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

BENEFITS
AI-Driven Emotional Support: Crisis Detection &
Real-time chatbot, multimodal Professional Access: NLP-
interaction (chat, voice, video), based distress detection,
and emotion-aware responses emergency alerts, and direct
for personalized care. therapist booking for critical
cases.

Community & Safe Venting: Secure & User-Friendly

Talkbox, Mini Blog for peer Experience: JWT
support, private space for users authentication, role-based
to express emotions freely. access, caching, and a soothing,
empathetic UI for user comfort.

22
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

CONCLUSION

23
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

CONCLUSIO
N
• Mental health issues are widespread, often unnoticed, and
worsened by stigma and limited access to help.
• The need for immediate, personalized, and stigma-free
support is greater than ever.
• CONSULO bridges this gap with AI-driven emotional
intelligence and real-time crisis intervention.
• Provides a safe space for users to vent, seek help, and feel
heard without judgment.
• Combines technology with empathy, ensuring no one faces
mental health struggles alone.

24
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

FUTURE SCOPE

25
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

FUTURE
SCOPE

• Multilingual Support: Expanding to multiple languages

for global accessibility, ensuring effective mental health
support across diverse regions.

• Advanced Voice Assistant: Enhancing AI to be fully

voice-responsive, like Alexa, enabling hands-free interaction
for seamless user experience.

26
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

REFERENCES

27
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

REFERENCE
S
1 K. Denecke, S. Vaaheesan and A. Arulnathan, "A Mental Health Chatbot for Regulating Emotions
. (SERMO) - Concept and Usability Test," in IEEE Transactions on Emerging Topics in Computing,
vol. 9, no. 3, pp. 1170-1182, 1 July-Sept. 2021, doi: 10.1109/TETC.2020.2974478

2 S. Rahman, S. Sarker, A. K. M. N. Haque, M. M. Uttsha, M. F. Islam and S. Deb, "AI-Driven Stroke

. Rehabilitation Systems and Assessment: A Systematic Review," in IEEE Transactions on Neural
Systems and Rehabilitation Engineering, vol. 31, pp. 192-207, 2023, doi:
10.1109/TNSRE.2022.3219085.
3
.
M. Pantic and I. Patras, "Dynamics of facial expression: recognition of facial actions and their
temporal segments from face profile image sequences," in IEEE Transactions on Systems, Man,
and Cybernetics, Part B (Cybernetics), vol. 36, no. 2, pp. 433-449, April 2006, doi:
4 10.1109/TSMCB.2005.859075.
.

S. M. Kambare, K. Jain, I. Kale, V. Kumbhare, S. Lohote and S. Lonare, "Design and Evaluation of
an AI-Powered Conversational Agent for Personalized Mental Health Support and Intervention
(MindBot)," 2024 International Conference on Sustainable Communication Networks and
Application (ICSCNA), Theni, India, 2024, pp. 1394-1402, doi:
10.1109/ICSCNA63714.2024.10863855.
28
 INDIAN INSTITUTE OF INFORMATION TECHNOLOGY,
CONSULO PUNE

THANK YOU

29