SKINCARE RECOMMENDATION SYSTEM

USING COMPUTER VISION

Vikram Kumar K Emuri Bhavanesh Dr. J. Cruz Antony
Department of Computer Science and Department of Computer Science and
Department of Computer Science and Engineering Engineering
Engineering Sathyabama Institute of Science and Sathyabama Institute of Science and
Sathyabama Institute of Science and Technology Technology
Technology Chennai, India Chennai, India
Chennai, India

Abstract— This analysis highlights the limitations of

traditional skincare, which relies on subjective methods and This research investigates the use of computer vision (CV)
fails to deliver truly customized solutions. It proposes in personalised facial skin care, and its focus is on several
computer vision (CV) as a transformative force, capable of
CV algorithms including landmark localization, object
objectively analyzing skin images through AI algorithms. By
identifying individual concerns and offering data-driven
detection and image segmentation. Hence it examines
product recommendations, CV paves the way for their potentiality for melanoma detection and classifying
personalized skincare regimes and enhanced outcomes. This skin conditions thus creating a novel business model that
shift signifies a move away from one-size-fits-all approaches employs CV for acne recognition, product suggestions as
and opens doors to endless possibilities for individual- well as skin typing. It is evident from positive results
specific skincare in the future. attained in skin type classification that this could be a
transformative system to specify convenience and efficacy
of recommendations on skincare products[1].
INTRODUCTION
Using Neural Collaborative Filtering (NCF) with implicit
Classic methods haven’t provided great custom skincare rating based on sentiment analysis from user reviews. The
suggestions because they depend on generalized self- findings show that NCF outperforms explicit rating and
diagnoses and rudimentary methods of skin analysis. matrix factorization methods based on implicit rating.
Computer vision (CV), however, is an option — it can Implicit ratings are obtained through carrying out
analyze skin images using image recognition algorithms sentiment analysis on data mined from Female Daily
and machine learning systems. Those techniques make it website while root mean squared error is used to
possible to identify skin issues, evaluate the health of your train/evaluate the model. In conclusion, the paper suggests
skin and make recommendations for suitable beauty that exploring other attributes could further improve the
products to get the results you want with your skincare. performance of the classifier thereby making a significant
contribution to skincare recommendation systems by
Conventional skincare involves asking people about their referencing related literature[2].
preferred products through questionnaires or personal
interviews. The responses are subjective, likely to be
wrong and may not accurately reflect the nuances of This paper examines the integration of machine learning
particular skin conditions. CV (and AI in general) is and deep learning algorithms to develop personalized
intended to have an objective approach. skincare product recommendation platform. The platform
relies on YOLOv4’s object recognition algorithm to
Through careful data collection on the skin, CV-driven identify facial features, and provides customized advice
advice systems can create custom skincare products for for users’ skin problems. In addition, this research also
each customer. The system considers skin type, concerns, looks at other common skin issues such as wrinkles, spots
and lifestyles (e.g., exposure to pollutants) through a and acne vulgaris by examining how various components
recommendation tool that chooses products for particular
of skincare products work in solving these. The main
skin issues aimed at enhancing general skin health. This
objective is to give an insight of what your skin needs
individualised method makes it possible to avoid such
mistakes as choosing incompatible skincare products with hence enabling you to purchase a suitable product that
potential side effects. will not harm it[3].

Personalized skincare has taken another step with the Moreover, the paper presents a technique that combines
incorporation of CV into the skincare recommendation deep neural networks with user reviews in order to
system. CV-powered systems help people objectively provide personalized recommendations for skin care
analyze their skin, prescribe personal product products. Additionally, it surveys existing systems for
recommendations, and continuously monitor their skin for recommending skincare items online based on text
a healthy radiant look. With regards to CV technology analysis efficiency through user opinions. It surpasses
advancement, there will be endless opportunities for traditional matrix factorization approaches often
customized skincare and the world of unique skincare mentioned in recommender systems like collaborative
regimes for each individual. filtering, neural collaborative filtering and hybrid
LITERATURE SURVEY recommender systems based on deep learning.Exploring

979-8-3503-1702-2/23/$31.00 ©2023 IEEE

sentiment analysis, attention networks, and topic people’s skin types by evaluating extracted features from
modelling, the survey provides insights into context-aware their pictures and then offer skin care recommendations
recommendation systems for skincare products[4]. based on individual characteristics. This will only work
when we apply OpenCV image processing methods like
SYSTEM ARCHITECTURE thresholding or color segmentation and this way we are
able to detect skin flaws such as acne, wrinkles and hyper-
pigmentation. In addition, it enhances the systems ability
through use of specialized classifiers or pre-trained
models to determine and quantify their existence hence
making it more effective in addressing specific skincare
problems. It also allows textures analysis algorithms
available in both OpenCV and TensorFlow libraries to be
applied thus analyzing aspects of skin texture like
smoothness as well as pore size. Therefore, our system
will provide detailed feedback regarding perfect texture
diagnosis for better appearance meaning that users can
enjoy ultimate experience in terms of all aspects regarding
their skins [5].

PROPOSED SYSTEM

In a world where your selfie takes you to the road of

healthy skin, imagine it. Here we have our Smart Digital
Skincare Recommendation System that uses advanced
computer vision algorithms like FaceNet to bring life out
of selfie snapshots.

This proposal introduces a CV-based system for Step - 3 User Input Processing:
recommending skincare products using TensorFlow and
OpenCV for face recognition and skin analysis, along In our User Input Processing module, we would
with NLP for understanding user queries about skin incorporate an analysis like NLP (Natural Language
problems. Processing) with some sentiment analysis techniques such
that we can understand and interpret user input text in
Step 1- Face Recognition: regard to the concern they have about their skin, what
exactly they want to improve. We perform NLP analysis
Data is collected either by taking a picture or by carefully parsing the user-provided text for the
capturing a selfie. This module comes into place occurrence of keywords and specific key phrases which
converting your image into the correct form. For this can denote what kind of problem or goal the user is
purpose, it employs either OpenCV’s black magic or expecting. The system identifies patterns and discerns
TensorFlow’s brawn, acting as if they were sheriffs who relevant data from the user's input, to arrive at sound
had caught you in their sights. Then it extracts key judgment on the needs and preferences of every
features like landmarks and points of interest needed for individual, all pertaining to skincare. Again, this algorithm
analyzing your skin.[5] can further determine the mood and attitude of the user
towards his skin, ranging from frustration and feeling
uncomfortable to satisfaction and delight. More narrowly,
Step - 2 Skin Analysis: to understand the user's sentiment than just assuming that
all users are sympathetic towards the journey of this very
In our proposed skin analysis module, skin personal skincare. So, it influences what the system is
analysis is designed to use a blend of TensorFlow and recommending. A system of that kind, which fuses NLP
OpenCV techniques to give an in-depth look at the state of and sentimental analysis, makes an effort to provide
users’ skin. Initially, we plan on using TensorFlow personalized solutions reflecting specific user concerns
employing the Convolutional Neural Networks (CNNs) to and preference, as well as emotional context in a drive to
develop a model that can be used with labelled datasets make personal skincare better.
that have different types of skins such as; oily, dry and
normal. Consequently, the system will be able to identify
Step - 4 Product Recommendation: with a description of skin type and individual
recommendations of products that give choices to take
For these and other reasons, our product decisions in an informed way about healthier glowing
recommendation module relies on a mix of approaches skin.
aimed at matching users to personalized skincare
solutions. First, we deploy ingredient matching techniques Our system comes with a rich library filled with articles,
to map the user's skin type, identified blemishes, desired videos, and quizzes based on skin condition, routine, and
goals, and input keywords against a rich knowledge base ingredient function. It guides you not only around the
of skincare ingredients known for effective problem- System Recommendations but also in making your skin a
solving. That is, in effect, an intelligent match to relate master of its destiny.
attributes of the user to profiled attributes of ingredients in
the items offered to recommend a product with a formula This smart digital system of recommendations is only the
adaptation in skincare needs. We later apply product tip of an iceberg—imagine a future when computer vision
filtering mechanisms upon the matched ingredients, skin will reveal secrets toward really personalized and data-
type, and user preferences available offerings in skincare. driven skincare. Instant feedback on what your routine is
It is through this process that we hope to provide the best- doing: hyper-local product recommendations based on any
curated assortment of products customized to best meet variety of environmental factors, powered by real
our customers' unique needs in skin care. We dermatologists or an AI consultation—the sky's the limit.
subsequently rank the recommended products by real-time We believe that CV has the key to entering a world where
user reviews, ratings, and brand preferences. This everyone can get their healthiest and radiant skin.
objective further values the recommendations themselves
by integrating insights from peer experiences with direct This clarifies a bit further that the individual uses of
user input to not only apply ingredient matching and Tensorflow and OpenCV were mainly in facial feature
filtering of products but also strategies for the detection and personalized tracking. The skin analysis
prioritization of suggestions to the user so that they can techniques are developed in more detail, with an emphasis
be, if possible, fully personalized fitting the said products on texture and color as tools used in addressing
themselves. individualized needs. The fact that all this comes in the
form of a link to personalized product recommendation,
Step - 5 Output & Feedback: virtual try-on, and real-time feedback loop does, if
anything else, create more emphasis on the dynamism and
Propose a list of products and provide adaptability of the system today. Lastly, the thesis
justification about their appropriateness for addressing highlights empowerment of users through knowledge
specific concerns of the user.Provide space for the user to about their own capacities and a vision for opening a
give feedback on how appropriate they found the future with the power of CV.
recommendations and what they may have bought instead.
This feedback will be very helpful for your system to
learn how to improve its performance. RESULTS AND DISCUSSION

Additional Considerations: The skincare recommendation system harmonizes

OpenCV for image processing, TensorFlow for deep
It represents not only the basic features but also learning, NLP for text understanding, and machine
comprises some of the features that are important learning for facial training to deliver precise
requirements. By protecting user data, privacy 1recommendations tailored to individual needs.
enforcement is strict, and transparency in choice is given
for our product recommendations by clearly stating Machine learning algorithms, coupled with facial training
reasons one by one for each suggestion. We develop techniques, augment the system's ability to recognize
through a learning approach by persistently updating the facial features accurately, enhancing user identification
system with new data and keeping the updated user and personalization alongside the robust capabilities of
feedback in its place to bring continuous improvement in OpenCV, TensorFlow, and NLP.
accuracy and relevancy. These try to deliver personalized
solutions in skincare by also building user trust, The research on skincare recommendation systems
understanding, and satisfaction. harnessed TensorFlow, OpenCV, and NLP to provide
personalized solutions. Integration of TensorFlow and
Benefits: OpenCV facilitated robust skin analysis, including skin
type classification and blemish detection. NLP parsed user
The system of a skincare recommendation unites input for skincare concerns and goals, enabling tailored
all participants in a general updated, individualized recommendations. By matching user data with suitable
approach to skincare needs and tastes. The system avoids skincare ingredients and products, our system generated
a very highly subjective criticism as well as certain effective skincare suggestions. Challenges include data
prejudices in terms of skincare recommendations via an privacy, lighting variations, and expanding analysis
efficient and objective analysis technique. In simple capabilities. Our study demonstrates the potential of
access, the system provides immediate analytical results computer vision and NLP in transforming skincare
recommendations, paving the way for personalized and
data-driven solutions. Ongoing refinement is crucial to
address challenges and enhance system efficacy.

Fig. Skin type detection

REFERENCES

1. Chan, Hung-Tse, et al. "Smart Facial Skincare Products

Using Computer Vision Technologies." 2021 Asia-
Pacific Signal and Information Processing Association
Annual Summit and Conference (APSIPA ASC). IEEE,
2021.

2. Qalbyassalam, Chaira, Reza Fuad Rachmadi, and Arief

Kurniawan. "Skincare Recommender System Using
Neural Collaborative Filtering with Implicit Rating."
2022 International Conference on Computer
Engineering, Network, and Intelligent Multimedia
(CENIM). IEEE, 2022.

3. Li, Hsiao-Hui, et al. "Based on machine learning for

personalized skin care products recommendation
engine." 2020 International Symposium on Computer,
Consumer and Control (IS3C). IEEE, 2020.

4. Adebo, Adelola. A Natural Language Processing

Approach to a Skincare Recommendation Engine. Diss.
Dublin, National College of Ireland, 2020.

5. Kumbhar, P. Y., et al. "Real time face detection and

tracking using OpenCV." International journal for
research in emerging science and technology 4.4
(2017): 39-43.

6. L. Yuan, Z. Qu, Y. Zhao, H. Zhang and Q. Nian, "A

convolutional neural network based on TensorFlow for
face recognition," 2017 IEEE 2nd Advanced
Information Technology, Electronic and Automation
Control Conference (IAEAC), Chongqing, China, 2017,
pp. 525-529, doi: 10.1109/IAEAC.2017.8054070.