Nat. Volatiles & Essent.

Oils, 2021; 8(5): 4486 – 4494

Waste Segregation using CNN & IoT

1VRajesh, 2K Raghava Rao, 3P Devendra, 4E Venkatesh Babu, 5B Venkatesh, 6 L S P Sairam
Nadipalli, 7Sk Hasane Ahammad, 8 T Penchala Naidu
1,2,3,4, 5,,6,7,
Department of ECE, Koneru Lakshmaiah Education Foundation, Guntur, India-522502
Mail id: [email protected]

Abstract:
Due to rapid urbanization and ever-increasing population, India requires a sophisticated waste management system. Only
one-fifth of the generated waste is treated and the rest is dumped in landfill sites. Segregation of household waste reduces a lot of
complexity in waste treatment plants. Usually, segregation of household waste into dry and wet categories is done manually by the
workers collecting the waste. This paper describes a waste management system to segregate waste into dry waste and wet waste
through automation. The automation process saves a lot of time and effort and makes it easier for waste treatment plants. The
system includes a camera module, which captures the image of the waste based on input from a sensor. The image, that is stored on
Raspberry Pi, is then classified used the deep learning classification model. The proposed model is a miniaturized waste management
system uses IoT and Deep Learning which can be implemented in the waste treatment plants or large sized community garbage
containers on a large scale. The project consists of sensors, Raspberry Pi board, Deep Learning model, model training tool—Lobe, IoT
dashboard, camera module, deep learning libraries which include lobe, TensorFlow Lite and open CV and servo motor. The CNN
learning model uses ResNet and MobileNet algorithms for speed, accuracy and compatibility. The model, trained using a total of
12000 image samples, is efficient in segregating waste into dry waste and wet waste.
Keywords: CNN, Deep Learning, IoT, Lobe, MobileNet, Raspberry Pi, ResNet, TensorFlow Lite

I. Introduction
According to Down to Earth magazine, around 380 million people live in 7935 towns and cities in
India. And, they generate around 62 million tons of waste. Of which, only 43 million tons is collected and 11.9
million tons is treated. Remaining 31 million tons of untreated waste is dumped into landfills. Indian
government came up with an initiative towards a Clean India called Swachh Bharat. Now Swachh Bharat 2.0
which focuses on collection of waste to 100% with an investment of 1.41L crores. With the huge population
producing household garbage every day, there's a strong need for segregation of waste into dry and wet
categories. The segregation is essential because proper waste management not only reduces waste but also
reduces the emission of greenhouse gases like methane, carbon dioxide from the landfill sites.

Fig 1.1 - Wet waste and Dry waste categories

The wet waste category includes vegetable peels, leftover curries, chicken bones, fish bones, fruits, flowers,
and fallen dry leaves, and etc. While plastic, rubber, glass, metal, cardboard, paper, cloth come under the
category of dry waste. Wet waste is packed together bundled using paper for proper composting. Dry waste
will be sent for separation of material into recyclable and reusable categories. In this project, we have used
around 12000 image samples of dry and wet wastes to build an efficient model for classification. Raspberry

4486
Nat. Volatiles & Essent. Oils, 2021; 8(5): 4486 – 4494

Pi 3 and IoT devices are used to automate the process of segregation at waste treatment plants and municipal
garbage bins. Raspberry Pi 3 is the most widely used variant by the student and research community, since
the introduction of Raspberry Pi boards. Raspberry Pi is a low cost, SoC computer designed to easily interface
with IoT and embedded devices. It plugs into computer monitor, TV or Laptop for its display. Usage of Deep
learning along with IoT results in a smart and fully automated waste management system. We have used
convolutional neural networks to create an efficient model for image classification. This model which is in
TensorFlow lite format is better suited to run in embedded devices like Raspberry Pi 3, with it being a light
weight learning model along with better computing power.

Fig 1.2 – Train dataset composition Fig 1.3 – Test dataset composition

II. Literature Review:

The waste management system involves various types of waste like household, medical, electronic waste,
and others. Earlier research works proposed the methods that are done by using IoT devices and sensors.
Some papers proposed the segregation of waste into biodegradable and non-biodegradable categories, but
the paper, which is a recyclable material, will be thrown into landfills along with other biodegradable waste.
The intervals of the garbage collection and “distance between the treatment plant also plays a major role”
[1]. Conventional methods, where human workforce is involved in the manual segregation of the waste can
cause diseases and also such a method is time consuming and requires a lot of effort [15]. There were some
models proposed that trash containers will be screened and the waste is classified and stores the data on
webserver for future analysis [7]. Although some papers presented a faster algorithm like R-CNN , Raspberry
Pi cannot efficiently process these algorithms which require high computational power, which raspberry pi
cannot provide[5]. So, we propose to use new generation tool like Microsoft Lobe, which can automate the
training of the learning model and builds a model in accordance with our dataset. It also features the option
to export the learning model in many formats and can be used like any other pre-trained model. TensorFlow
lite format available in this application allows the usage of faster and simpler model to be used in mobile &
small devices like Raspberry Pi.

III. Methodology:
The proposed waste management system is designed using model training tool – Lobe and a
microcomputer to interface the deep learning model with IoT sensors and devices – Raspberry Pi 3. The IoT
sensors used in this project are IR sensor, Ultrasonic sensor along with devices like Pi Camera and Servo
motors. The prerequisites are
1. Build an efficient learning model
2. Install the necessary python libraries

4487
Nat. Volatiles & Essent. Oils, 2021; 8(5): 4486 – 4494

The learning model is built using an application called Lobe, which is designed to train a model from a given
data set.
Lobe:
Lobe is a deep learning tool designed by Microsoft. It automates model training by extracting the features
from the image or video samples from input data. This app is specially designed to train models that involve
image analysis, image processing and video analysis. The model is trained from the given data and can be
exported from the application. This exported model can be used as any other pre-trained model. Lobe uses
ResNet (residual neural networks) and MobileNet (mobile neural networks) to train a model. Lobe simplifies
the method of training a model. It requires simply three steps: Import the collected data set, Train the model
and Understand the results and improve the model through feedback. Finally, export the model into any
desired format that is suits with the device, compatibility and computing power.
Installation of python libraries is done by few simple Linux commands through the terminal of Raspberry Pi.
These python libraries allow the python code to interact with IoT devices and sensors connected to raspberry
pi. The libraries are used by camera to capture the images and store them in the target folder, to import the
code and input image from the source folder, and to use and interact with the learning model. The libraries
used are Lobe, TensorFlow Lite, Pillow and Open CV.
Additionally. Python programming language is used to write a code to interface with IoT devices and deep
learning model with Raspberry Pi board. The design broadly comprises of three modules. The modules
process the input to either send output to the next module or to perform its own functions. The modules
are:
1. Camera module: The camera module comprises of a Pi camera and IR sensor to detect the presence of
an object. The camera captures image of the object when the output of the IR sensor is positive. The
camera uses open CV library to capture the image and store it using a pre-determined name on the local
storage. Open CV is an open source library. It contains programming functions and computer vision
tools, which are designed to solve computer vision problems.
2. Deep learning model: The deep learning model uses CNN algorithm, which classifies the image sample
stored on Raspberry Pi board. The model is exported from the Lobe application in the format of
TensorFlow Lite. The python code uses the Lobe, Pillow and TensorFlow lite libraries to get the input
image and analyse it by using the learning model to classify the image. The output is predicted by the
model after analysing its features and parameters. TensorFlow lite is a set of tools, more compatible
with Android, embedded and IoT devices. It performs with higher speed than TensorFlow and can be
exported as smaller and simpler model. This library is used to interface with TensorFlow-lite files
3. IoT module: This module composes of IoT devices and sensors. Ultrasonic sensor is used to measure the
depth and analyse if there is space available in the bin to dispose the waste into it. Servo motor is used
to dispose the waste into appropriate bin.

4488
 Nat. Volatiles & Essent. Oils, 2021; 8(5): 4486 – 4494

Fig 3.1 – Design

CNN algorithm:
The learning model used ResNet and MobileNet algorithms, which are a type of convolutional neural
networks. Convolutional Neural Networks (CNNs) are deep neural networks that are most commonly used
for Image classification and analysis and also for Videos samples. CNN casts multiple layers on the image to
extract features and parameters and uses these features to analyse the input image. CNN is more suitable
for image classification, as it can reduce the parameters, dimensions and features while retaining the
important and necessary features. This increases the speed and reduces the complexity. The layers used by
CNN to understand the patterns and features are math layers, rectified linear unit layer, and fully connected
layer.
1. Residual Networks: Also known as ResNets are a type of deep convolutional neural networks. The
architecture of residual network is in Figure 1. ResNet50 is popular convolutional neural network and
a variant of ResNet model, that is 50 layers deep.

Fig 3.1 - ResNet

2. Mobile Networks: Mobile Networks are streamlined and are of convolutional neural networks architecture
that uses depth-separable convolutions, designed to perform well on mobile devices with low computational
power like embedded devices, IoT devices, Android devices, and etc. to build light weight deep neural
networks. Its structure is similar to the Inverted residual block. MobileNetV2 is the model used in this project,
initially contains fully convoluted layers with 32 filters, followed by 19 residual bottleneck layers.

4489
 Nat. Volatiles & Essent. Oils, 2021; 8(5): 4486 – 4494

Figure 3.2 – MobileNetV2

IV. Experimental Setup:

The proposed waste management system can be broadly divided into three modules. Prerequisites are
Raspberry Pi board, Pi-camera, Lobe application, Ultrasonic sensor, IR sensor, Servo motor.
Before implementation of the model, there are libraries that need to be installed and requirements that need
to be fulfilled. This includes training deep learning models and preparing Python code to implement our
project.
 Preparing the Deep Learning Model:
The model is trained using Lobe software application, which was created by Microsoft Corp. to make it easier
for training a model by automating the training part based on the input samples or dataset. Lobe trains the
model using CNN algorithm which uses ResNet50 and MobileNet for Accuracy and Speed, respectively. This
application was created to easily train models which involve image analysis, image classification, video
analysis, and video surveillance.
This application can be used by importing the folder which contains the input samples with their classes being
used as labels. The labels used are 'DRY' and 'WET' for our project. The model is then trained by the
application. And then after training, the model can be tested by giving live inputs and providing feedback to
improve its accuracy.
The model is then exported from the application as a TensorFlow Lite file, which is more suitable for Android,
embedded and IoT devices.

 Installation of Python Libraries:

The code is written in Python programming language, which easily compliments the usage of functions and
libraries of Deep Learning. The libraries required are:
 Open CV:
The open CV library is used to work with the camera module and store the image in the allotted folder. The
captured image is named with a name as provided in the python code.
 TensorFlow Lite:
This library allows Python code to apply the model, Tflite file, to the input image sample. And a Json file
supports this model file.
 Lobe :
This library is used to implement the DL model which is exported from Lobe application. This library is used
to import the model and input image from the raspberry pi folder which consists of the input image sample.

4490
Nat. Volatiles & Essent. Oils, 2021; 8(5): 4486 – 4494

Fig 4.1 - Model Architecture

The components used are IR sensor, Pi Camera, Raspberry Pi 3 board, Ultrasonic sensor and Servo motor and
software includes Lobe Application, Raspberry Pi OS, Python and its libraries. The model exported from the
Lobe application is placed in a folder on Raspberry Pi and the path is mentioned in the python code.
Working of the proposed model starts with the detection of the object thrown into the bin by an IR sensor.
The IR sensor, then sends a value, ‘1' for presence of object and '0' for absence, to the python code. When
the object is detected, the camera captures the image and names it 'img' as mentioned in the code, with an
extension of ‘.png’.
The image, after pre-processing and dimensionality reduction, is stored in the local folder of the Raspberry
Pi as specified in the Python code. After the image is captured, the deep learning pre-trained model is
imported and used to classify the image and decide the category of the object.
The result is obtained from the classification model, which is either 'Dry' or 'Wet'. The ultrasonic sensor
attached to the top of the bin, calculates the free space available in the bin. This sensor works by sensing out
high frequency sound wave beyond the range of human hearing. The transducer sends and receives the
ultrasonic sound and then it calculates the distance using the time taken by the ultrasonic sound to return
back and its speed. If the bin is vacant enough, the output from the deep learning model is used as input to
the servo motor.
The servo motor is set in such a way that, when the object is classified as wet waste, the top tilts 45 degrees
towards the ‘Wet waste’ bin. Otherwise, it tilts 90 degrees towards the ‘Dry waste’ bin to dispose the waste
into the appropriate bin. If the bin is already full, it sends a message to the user or administration to clear
the bin. The administrator is notified by the text message sent through Bluetooth or Internet connection.
V. Results:
The time interval to empty the bins is set to 8 hours and the bins can also be emptied whenever the depth
or vacant space calculated by ultrasonic sensor reaches the threshold level. The data is plotted with time vs
depth of the bin to analyze and understand the frequency and interval at which the collection of data is ideal.

4491
Nat. Volatiles & Essent. Oils, 2021; 8(5): 4486 – 4494

.
Fig 5.1 – Time vs Depth of the bin
As observed from the Fig 5.1, the default depth of bin is 35cm. Whenever the bin is emptied, the ultrasonic
sensor reads a value of 35. Each bar in this graph represents the vacant space available in that hour.
Whenever the value is less than our threshold value of 5, the user is alerted to clear the bin. So, the depth in
next hour is a value of 35.
The following are input images along with their outputs. The object in the input image is waste object that is
classified by our classification model and result can be observed in the output terminal of the python IDE.
The output is either ‘Dry’ or ‘Wet’ based on the classification of the input image.

Fig 5.2 – Pickle

4492
Nat. Volatiles & Essent. Oils, 2021; 8(5): 4486 – 4494

Fig 5.3 – Polythene cover

Fig 5.4 – Banana peel

Fig 5.5 – Cardboard

4493
Nat. Volatiles & Essent. Oils, 2021; 8(5): 4486 – 4494

Fig 5.6 – Plastic bottle

The above images are the screenshots of the input images along with the category they are classified into.
Objects such as Polythene cover (Fig 5.3), Cardboard (Fig 5.5) and Plastic bottle (Fig 5.6) are classified as ‘Dry’
waste, whereas Banana peels (Fig 5.2) and Pickle (Fig 5.4) are classified as ‘Wet’ waste. These results prove
that the learning model is working efficiently and classification is accurate.

VI. Conclusion & Future Work:

The use of AI automates the work and processes involved to present an efficient way of doing things.
Although people fear that the use of AI is going to affect their jobs, the are some issues that require
immediate action and efficient methods to tackle them. Waste management has become a colossal issue,
that needs automation of the process along with swift and efficient methods to tackle it. Here, AI can
contribute by reducing time and effort through automation. The proposed system can automate the process
of segregation and separate waste into Dry and Wet classes.
This system can be efficiently used on a large scale in waste treatment plants and large garbage bins to treat
the waste. The automation of model training by lobe application can be further applied in classifying
medical waste, Electronic waste, Industrial waste, etc. Through the use of IoT and AI, the segregation and
treatment of waste becomes through and efficient, leading us towards a safer, cleaner and greener
environment.
VII. References:
1. Ramakrishnan Raman, “Artificial Intelligence Based Smart Waste Management System”, Patent
Application Publication, India, Application No. 202121038040 A.
2. Shamini N, P. Mohamed Fathimal,“Smart Garbage Segregation & Management System Using Internet
of Things(IoT) & Machine Learning(ML)” ,D.O.I:
https://doi.org/10.1109/ICIICT1.2019.8741443.
3. S. Vinoth Kumar, T. Senthil Kumaran , A. Krishna Kumar and Mahantesh Mathapati , “Smart Garbage
Monitoring and Clearance System using Internet of Things” - 2017 IEEE International Conference
4. Dev, Boudhayan , "Automatic Waste Segregation Using Image Processing and Machine Learning”.
International Journal for Research pp. 2617–2618, 10.22214/ijraset.2018.5428. Accessed 7 Oct. 2019.
5. “An Intelligent System for Waste Materials Segregation Using IoT and Deep Learning” , V R
Azhaguramyaa , J Janet, 2021 J. Phys.: Conf. Ser. 1916 012028.
6. Farzana Shaikh , Farzana Shaikh , "Waste Profiling and Analysis using Machine Learning", IEEE , DOI :
https://doi.org/10.1109/ICIRCA48905.2020.9183035 .
7. Wang Dandong ,Yu Yifan , "Design of municipal solid waste intelligent supervision Platform based on big
data" , IEEE Xplore , DOI : https://doi.org/10.1109/MLBDBI51377.2020.00055
8. Dr. Vilas Warudkar , Dr. Sapna Mathur , " Machine Learning Based Automatic Waste Management for
E-Environment” , Patent Application Publication , India , Application no. 202021053077 A
9. “Automatic Waste Segregator for Future World” , International Journal of Pharmaceutical Research ,vol
. 12, no. 02, 2 Oct. 2020, 10.31838/ijpr/2020.12.02.231. Accessed 5 Mar. 2021.

4494
Nat. Volatiles & Essent. Oils, 2021; 8(5): 4486 – 4494

10. Md. Wahidur Rahman, Rahabul Islam , "Intelligent waste management system using deep learning with
IoT" , ScienceDirect.
11. Website : https://www.tensorflow.org/tutorials/image/image_recognition.
12. Website : https://www.sciencedirect.com/science/article/pii/S1110062118301375.
13. Mark Sandler Andrew Howard “MobileNetV2: Inverted Residuals and Linear Bottlenecks”,
https://arxiv.org/pdf/1801.04381.pdf.
14. Dr. P. Veena, Dr J Jerusalin Carol, Samrudhi Rajendra Kaware, Sesha Bhargavi Velagaleti , “Artificial
Intelligence Based Autonomous Waste Management for Ε-Environment Application Νο.202141019996
A, 2021, Patent Application Publication , India.

4495