Automated Nutrient Level Determination Using

Machine Learning

Shivanand S. Gornale1(B) , A. C. Nuthan2 , and C Sumitha3

1 Department of Computer Science, Rani Chennamma University, Belgavi, Karnataka, India
[email protected]
2 Department of Electronics and Communication Engineering, GMIT, Bharathinagara, India
3 Department of Electrical and Electronics Engineering, GMIT, Bharathinagara, India

Abstract. In present scenario, people look health issues as non-trivial. The pri-
mary agenda of life is to have better nutrient food. The calories and nutrition intake
as proved harmful worldwide, as it has led to many disease’s however dieticians
have mistaken that the standard intake of number of calories is essential to main-
tain the right balance of nutrition and calorie context in the human body. The
common people (literate or illiterate) may not able to analyze and decide about
nutrient levels in packed food items and even unable to identify the freshness of
the food. This system is an aid in such scenarios where if the ingredient list is
given, then the system gives a suggestion regarding the nutrition level based on
the input. Since the system is to be used by the common (non-technical) people
the input to the system is the image of the nutrient list (available at rear end of
packaged food items).So our system involves both image processing and machine
learning (for automated suggestions based on the input ingredient list).

Keywords: Nutrient Level Determination · Food Freshness · Python · Machine

Learning

1 Introduction

Now a day’s most of the people look health issues as non-trivial. The primary agenda of
life is to have better nutrient food. Our system is an aid in such scenarios where if the
ingredient list is given, then the system gives a suggestion regarding the nutrition level
based on the input.
Since the systems is to be used by the common (non-technical) people the input
to the system is the image of the nutrient list (available at rear end of packaged food
items).So our system involves both image processing and machine learning (for auto-
mated suggestions based on the input ingredient list).The system has a privilege to detect
the freshness of certain food items.
This system helps to dietitians for maintain their weight and also it helps to control
the health condition of patients. The system designed here provides users with convenient
and intelligent mechanisms that allow them to track their food intake and monitor their
calorie count.

© The Author(s) 2023

R. Manza et al. (Eds.): ACVAIT 2022, AISR 176, pp. 701–710, 2023.
https://doi.org/10.2991/978-94-6463-196-8_53
702 S. S. Gornale et al.

Calculating food calorie and nutrition level in the intake on a daily basis is mere
impossible as it involves dieticians, doctors etc. To treat the patients who suffer from
obesity, overweightness, or other food-related health problems there should be correct
diagnosis which is challenging issue.
Shervin shirmohammadi et al. [1] gives personal software instrument which can cal-
culate intake calorie and nutrient in any camera equipped device. Image segmentation
and preprocessing identifies ROI (region of interest) i.e. food portions from the overall.
Amount of each food portion in image is calculated. Level of nutritional facts is calcu-
lated using the mass of each portion from its measured volume and matching it against
existing nutritional fact tables. As a continuation of the work, in [2] cloud based food
calorie measurement system is implemented which is suitable for android devices. Its
implementation is analyzed in detail about its functionality on Android system utilizing
cloud computing infrastructure. The algorithm is able to withstand huge test and training
dataset.
In the paper [3], as against the conventional methods, a novel method is proposed.
Vision Based Measurement (VBM) [1] has gained an appreciation as it is simple to use.
The algorithm deployed are transferred to the smartphone for real-time implementation.
In the paper [4], the problem caused by the uncertainty of freshness of fruits and
vegetables can be overcome by this new and, creative freshness sensor without any chem-
ical treatment which could damage the nutritional value of samples. Additionally, this
novel device could be considered as an innovative project which could be a solution for
microorganism caused respiratory diseases and allergy like diseases by detecting con-
tamination before microorganism colonies become visible with naked eye by measuring
ion concentration changes occurred at samples.
As one more method paper [5] also proposes a method for measuring food nutritional
value; this method can be used as a personal assistive application on smart phones. They
system analyzes food and fruit images using a set of images processing and analysis
procedures, and makes use of a database of images to extract the information needed
like color, shape and size properties from the images. It then uses an SVM to the food
portion types. Experimental results shows that our image processing technique works
better than others in segmenting and extracting the food portions.
In another method [6] uses smart phone to analyse images has many potential appli-
cations for people, especially patients. But, the variability of the food objects makes
it very difficult to adapt the algorithms with real data. Existing methods can process,
analyse and texture features. We have improved the functionality and flexibility of the
recognition system by adding shape and size features as well as color and texture. Fur-
thermore, by increasing the number of images in the database the recognition rate can
be increased.

2 Objectives

• A person is considered obese when the Body Mass Index is higher than or equal to
30kg. Recent studies have shown that obese people are more likely to have serious
health conditions such as hypertension, heart attack, high cholesterol, breast and colon
cancer etc.
 Automated Nutrient Level Determination Using Machine Learning 703

• High calorie intake as proved harmful worldwide, as it has let too many diseases.
However, dieticians have mistaken that a standard intake of number of calories is
essential to maintain the right balance of calorie content in the human body
• As per the guidelines of FDA, it has 32 different names of salts,31 different names of
sugar,17 different names of fats and 17 different names of allergens based on these,
help to decode the ingredient list.
• How it means, with the help of differently mentioned names of sugar, salt, fats and
allergens, it measures what type of ingredient is present in the given ingredient list.
• Scenarios which arrive even to common person during the purchase are

• Even though ingredient list is displayed, the common people may not able to
analyze and decide about nutrient levels in packaged food items.
• May not be able to identify the freshness of the food.

• This system should be able

• To identify the nutrition level from the input image ingredient list
• Freshness of the unpacked food item
• Best suitable suggestions based on identified nutrient level

• Proposed Methodology & Implementation (Fig. 1):

The work handles two of food items.

1. Packed Food.

Fig. 1. Proposed Methodology for packed food items

704 S. S. Gornale et al.

2. Unpacked Fresh vegetable/Fruits.

Capture the food of the image using smart phone. Captured image is preprocessed
and is converted into text using character recognition. The text converted image will
measures the nutrition and calories by using mathematical calculations which acts like a
dataset for machine learning. Using machine learning algorithm a suggestion is given to
specific people according to their diet concern. The final outcome of the food measures
the nutrition and calories. Same methodology is applied to check the freshness of food.
Let us consider snicker as an example, by capturing ingredient list (like sugar, choco-
late, milk fat & so on) by the camera. Such list will convert image to text with the help
of image processing (Figs. 2, 3, 4 and 5).
Then the system will measure the nutrition and calorie from the image captured.
Then the people will come to know about the nutrition and calorie of the snickers.
Those people who dieting they receive guidance from the dietician. What and how
much to be consumes for their health. If the snickers calories and nutrients are matched
with dietician guidance, then it is preferable to consume snickers, If it is not matched
with dietician guidance, then it is not preferable to consume.
The technical details block will explain how technically it works for both nutrition
and calorie measurement as well as for freshness identification of food (vegetables,
fruits, meat, etc.).
First we capture the image using smart phone it automatically select ingredient
list from the packed food for measurements and capture the food image for freshness
identification.
The image to text conversion uses the OCR (optical character recognition) tool to
extract the text from image (ingredient list). It helps to analyze the text in the image
that what image we upload and converts into text. For freshness identification no need

Fig. 2. Example w.r.t packed food

 Automated Nutrient Level Determination Using Machine Learning 705

Fig. 3. Technical procedure w.r.t. packed food

Fig. 4. Flowchart of handling packed food

to convert from image to text but for measuring nutrition and calorie needs to convert
from image to text.
The mathematical calculation block will measure the nutrition and calorie in the
packed food and freshness identification of the food. Finally it goes to the machine
learning block it gives final result to the users.
706 S. S. Gornale et al.

Fig. 5. Block diagram w.r.t. unpacked food

This block will identify the freshness of the food items. The purpose of this,people
don’t might be know if an food gone bad before buying it and don’t know how much foods
contaminated by microorganisms this leads to health issue.To overcome this problem
freshness identification technology is developed.
The freshness identification technology is done by freshness sensor which identifies
the food freshness as helps to know the hidden details present in foods like fruits,
vegetables, meat, etc.
Figure 6 shows flowchart of handling the unpacked food.
• First it will choose the colors of the food.
• On the basis of green, red and yellow we are going to implement the freshness
identification of the food.
• Based on the green color we decide that the fruit is not ripe, based on the red or yellow
color we decide that fruit is ripe.

3 Machine learning
Machine learning deals with the automation of programming where the computer pro-
gram modifies itself based on the new dataset. At first it should be basically trained
using test dataset. It is classified into Supervised learning and unsupervised learning.
Data samples are categorized into four types for classification: false negatives (FN), true
negatives (TN), true positives (TP), and false positives (FP).
Here lasso regression method is used to check the freshness based on features like
color texture, shape etc. The algorithm uses both variable selection and regularization
along with soft thresholding. It selects only a subset of the provided covariates for use
in the final
model. It is mathematical defined by,
N−1 N i=1 f(xi, yi, a, b).
To check the efficiency of classification, RMSE is evaluated.
 
• RMSE = N1 ni=1 xi2

n
x2
• RMSE= ni=1 ni2 x100%
i=1 i

Here xi and ni are the error and reference, N is length of the analyzed data which is
125 in this work.
 Automated Nutrient Level Determination Using Machine Learning 707

Fig. 6. Flowchart of handling unpacked food

4 Result
To check the freshness the dataset from Kaggle is used. The recognition rate is summa-
rized in the Table.1. About 125 experiments were conducted with the variation types,
colours and other parameters. The recognition rate is about 70% which can further be
increased.
Since the system is to be used by the common (non-technical) people the input to the
system is the image of the nutrient list (available at rear end of packaged food items).The
system has a privilege to detect the freshness of certain food items. Figure 7.and Fig. 8
shows the one of the example results of how the system works by handling both packed
and unpacked food.
The RMSE for the designed classification is 0.307 which is within the thumb rule
of 0.2 to 0.4.
708 S. S. Gornale et al.

Fig.7. Results of handling packed food

Table 1. Summary of Food Freshness

FOOD ITEMS RECOGNITION RATE (%)

USING USING USING SIZE USING USING ALL USING ALL
COLOUR TEXTURE FEATURES SHAPE FEATURES FEATURES
FEATURES FEATURES FEATURES (10FOLD
CROSS-
VALIDATION)
1 apple 60 85 31 23 98 59
2 orange 65 79 41 71 96 70
3 tomato 71 70 48 45 90 65
4 carrot 75 80 69 65 100 78
5 onion 46 80 32 23 90 54
6 bean 77 80 77 65 99 80
Total average 66 79 50 49 96 68

Fig. 8. Results of handling unpacked food

 Automated Nutrient Level Determination Using Machine Learning 709

5 Conclusion

• The proposed system can identify the nutrition level from the input image ingredient
list
• Based on nutrient level in the ingredient list the system is able to give appropriate
suggestions based on identified nutrient level.
• Freshness of the unpacked food item can also be checked
• Regression models were implemented to classify. Lasso regression showed better
results in terms of lesser MSE.
• In future the work can be extended to clustering so that system is more robust even
for the non-identified objects which make the system more suitable for real time
applications.

References
1. Bharat K.., Broder A., “A technique for measuring the relative size and overlap of public Web
search engines”, Computer Networks, 30(1–7), pp. 107–117, 1998
2. P. Pouladzadeh, S. Shirmohammadi, and R. Almaghrabi, “Measuring Calorie and Nutrition
from Food Image”, IEEE Transactions on Instrumentation and Measurement, DOI: https://
doi.org/10.1109/TIM.2014.2303533, February 25, 2014.
3. P. Pouladzadeh, G. Villalobos, R. Almaghrabi, and S. Shirmohammadi, “A Novel SVM Based
Food Recognition Method for Calorie Measurement Applications” , Proc. International Work-
shop on Interactive Ambient Intelligence Multimedia Environments, , p. 495 – 498, in Proc.
IEEE International Conference on Multimedia and Expo, Melbourne, Australia, 2012.
4. Wenyan Jia et al., “A Food Portion Size Measurement System for Image-Based Dietary
Assessment,” Bioengineering Conference, IEEE, pp. 3–5, April 2009.
5. R. Almaghrabi, G. Villalobos, P. Pouladzadeh, and S. Shirmohammadi, “A Novel Method
for Measuring Nutrition Intake Based on Food Image,” in Proc. IEEE International
Instrumentation and Measurement Technology Conference, Graz, Austria, p. 366 – 370, 2012
6. Food Volume to Weight Conversions: http://www.aqua-calc.com/page/density-table
7. Health Canada Nutrient Values. : http://www.hcsc.gc.ca/fnan/nutrition/fiche-nutridata/nut
rient_value-valeurs_nutritives-tc-tmeng.php, November, 2011
8. S. Avancha and A. Baxi, “Privacy in Mobile Technology for Personal Healthcare”, ACM
Computing Surveys, Vol. 45, No. 1, , pp.1–54C., November 2012.
9. Wang and Z. Ma, “Design of wireless power transfer device for UAV,” in Proc. IEEE Int.
Conf. Robot. Automat. (ICRA), , pp. 2449–2454, Aug. 2016
10. H. Hoashi, T. Joutou, and K. Yanai, “Image recognition of 85 food categories by feature
fusion” in Proc. 2010 IEEE Int. Symp. on Multimedia (ISM), pp. 296–301, 2010
11. F. Takeda et al., “Dish extraction method with neural network for food intake measuring system
on medical use,” Computational Intelligence for Meas. Syst. and Applications, pp. 56–59,
Jul. 2003.
12. R. M Haralick, K Shanmugan, and I Dinstein, “Textural features for image classification,”
IEEE Transactions on Systems, Man, and Cybernetics, vol. 3, pp. 610 - 621, 1973.
13. World Health Organization Statistics 2012: http://www.who.int/gho/publications/world_hea
lth_statistics/2012/ en/index.html, 2012
14. J. Baxter, “Food recognition using ingredient-level features,” http://jaybaxter.net/6869_food_
project.pdf.
710 S. S. Gornale et al.

15. Nashat, A. A., & Hassan, N. M. “Automatic Segmentation and Classification of Olive Fruits
Batches Based on Discrete Wavelet Transform and Visual Perceptual Texture Features” Inter-
national Journal of Wavelets, Multiresolution and Information Processing, 16(1), 185-193,
2018
16. Pan, C., Yan W., “Salient Object Detection Based on Perception Saturation. Multimedia Tools
and Applications”, DOI: https://doi.org/10.1007/s11042-020-08866-x), 2020.
17. Davey, M. W., Stals, E., Ngoh-Newilah, G., Tomekpe, K., Lusty, C., Markham, R. “Sampling
Strategies and Variability in Fruit Pulp Micronutrient Contents of West and Central African
Bananas and Plantains (Musa Species)”. Journal of Agricultural and Food Chemistry, 55(7),
2633–2644. 2007.

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-
NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/),
which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any
medium or format, as long as you give appropriate credit to the original author(s) and the source,
provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter’s Creative
Commons license, unless indicated otherwise in a credit line to the material. If material is not
included in the chapter’s Creative Commons license and your intended use is not permitted by
statutory regulation or exceeds the permitted use, you will need to obtain permission directly from
the copyright holder.