prediction = model.

predict(x_one_hot)
next_index = np.argmax(prediction)
next_char = index_to_char[next_index]
generated_text += next_char

print("Generated Text:")
print(generated_text)
output:
1/1 [==============================] - 1s 517ms/step
1/1 [==============================] - 0s 75ms/step
1/1 [==============================] - 0s 101ms/step
1/1 [==============================] - 0s 93ms/step
1/1 [==============================] - 0s 132ms/step
1/1 [==============================] - 0s 143ms/step
1/1 [==============================] - 0s 140ms/step

\Application to text Image

1. Introduction When people listen to or read a narrative, they quickly create pictures in their
mind to visualize the content. Many cognitive functions, such as memorization, reasoning ability,
and thinking, rely on visual mental imaging or “seeing with the mind’s eye”
[1]. Developing a technology that recognizes the connection between vision and words and can
produce pictures that represent the meaning of written descriptions is a big step toward user
intellectual ability. Image-processing techniques and applications of computer vision (CV) have
grown immensely in recent years from advances made possible by artificial intelligence and deep
learning’s success. One of these growing fields is text-to-image generation.
The term text-to-image (T2I) is the generation of visually realistic pictures from text inputs.
T2I generation is the reverse process of image captioning, also known as image-to-text (I2T)
generation [2–4], which is the generation of textual description from an input image. In T2I
generation, the model takes an input in the form of human written description and produces a
RGB image that matches the description. T2I generation has been an important field of study due
to its tremendous capability in multiple areas. Photo-searching, photo-editing, art generation,
captioning, portrait drawing, industrial design, and image manipulation are some common
applications of creating photo-realistic images from text. The evolution of generative adversarial
networks (GANs) has demonstrated exceptional performance in image synthesis, image super-
resolution, data augmentation, and image-to-image conversion. GANs are deep learning-based
convolution neural networks
(CNNs) [5,6]. It consists of two neural networks: one for generating data and the other for
classifying real/fake data. GANs are based on game theory for learning generative models. Its
major purpose is to train a generator (G) to generate samples and a discriminator (D) to discern
between true and false data. For generating better-quality realistic image, we performed text
encoding using recurrent neural networks (RNN), and convolutional layers were used for image
decoding. We developed recurrent convolution GAN (RC-GAN), a simple an effective
framework for appealing to image synthesis from human written textual descriptions. The model
was trained on the Oxford-102 Flowers Dataset and ensures the identity of the synthesized
pictures. The key contributions of this research include the following: • Building a deep learning
model RC-GAN for generating more realistic images. • Generating more realistic images from
given textual descriptions. • Improving the inception score and PSNR value of images generated
from text. The following is how the rest of the paper is arranged: In Section 2, related work is
described. The dataset and its preprocessing are discussed in Section 3. Section 4 explains the
details of the research methodology and dataset used in this paper. The experimental details and
results are discussed in Section 5. Finally, the paper is concluded in Section 6. 2. Related Work
GANs were first introduced by Goodfellow
[7] in 2014, but Reed et al. [
8] was the first to use them for text-to-image generation in 2016. Salimans et al.
[9] proposed training stabilizing techniques for previously untrainable models and achieved
better results on the MNIST, CIFAR-10, and SVHN datasets. The attention-based recurrent
neural network was developed by Zia et al.
[10]. In their model, word-to-pixel dependencies were learned by an attention-based auto-
encoder and pixel-to-pixel dependencies were learned by an autoregressive-based decoder. Liu et
al.
[11] offered a diverse conditional image synthesis model and performed large-scale experiments
for different conditional generation tasks. Gao et al.
[12] proposed an effective approach known as lightweight dynamic conditional GAN (LD-
CGAN), which disentangled the text attributes and provided image features by capturing multi-
scale features. Dong et al.
[13] trained a model for generating images from text in an unsupervised manner. Berrahal et al.
[14] focused on the development of textto-image conversion applications. They used deep
fusion GAN (DF-GAN) for generating human face images from textual descriptions. The cross-
domain feature fusion GAN (CFGAN) was proposed by Zhang et al.
[15] for converting textual descriptions into images with more semantic detail. In general, the
existing methods of text-to-image generation use wide-ranging parameters and heavy
computations for generating high-resolution images, which result in unstable and high-cost
training.
(CNNs) [5,6]. It consists of two neural networks: one for generating data and the other for
classifying real/fake data. GANs are based on game theory for learning generative models. Its
major purpose is to train a generator (G) to generate samples and a discriminator (D) to discern
between true and false data. For generating better-quality realistic image, we performed text
encoding using recurrent neural networks (RNN), and convolutional layers were used for image
decoding. We developed recurrent convolution GAN (RC-GAN), a simple an effective
framework for appealing to image synthesis from human written textual descriptions. The model
was trained on the Oxford-102 Flowers Dataset and ensures the identity of the synthesized
pictures. The key contributions of this research include the following: • Building a deep learning
model RC-GAN for generating more realistic images. • Generating more realistic images from
given textual descriptions. • Improving the inception score and PSNR value of images generated
from text. The following is how the rest of the paper is arranged: In Section 2, related work is
described. The dataset and its preprocessing are discussed in Section 3. Section 4 explains the

details of the research methodology and dataset used in this paper. The experimental details and
results are discussed in Section 5. Finally, the paper is concluded in Section 6. 2. Related Work
GANs were first introduced by Goodfellow [7] in 2014, but Reed et al. [8] was the first to use
them for text-to-image generation in 2016. Salimans et al. [9] proposed training stabilizing
techniques for previously untrainable models and achieved better results on the MNIST, CIFAR-
10, and SVHN datasets. The attention-based recurrent neural network was developed by Zia et
al. [10]. In their model, word-to-pixel dependencies were learned by an attention-based auto-
encoder and pixel-to-pixel dependencies were learned by an autoregressive-based decoder. Liu et
al. [11] offered a diverse conditional image synthesis model and performed large-scale
experiments for different conditional generation tasks. Gao et al. [12] proposed an effective
approach known as lightweight dynamic conditional GAN (LD-CGAN), which disentangled the
text attributes and provided image features by capturing multi-scale features. Dong et al. [13]
trained a model for generating images from text in an unsupervised manner. Berrahal et al. [14]
focused on the development of textto-image conversion applications. They used deep fusion
GAN (DF-GAN) for generating human face images from textual descriptions. The cross-domain
feature fusion GAN (CFGAN) was proposed by Zhang et al. [15] for converting textual
descriptions into images with more semantic detail. In general, the existing methods of text-to-
image generation use wide-ranging parameters and heavy computations for generating high-
resolution images, which result in unstable and high-cost training.

This section describes the training details of deep learning-based generative models. Conditional
GANs were used with recurrent neural networks (RNNs) and convolutional neural networks
(CNNs) for generating meaningful images from a textual description. The dataset used consisted
of images of flowers and their relevant textual descriptions. For generating plausible images
from text using a GAN, preprocessing of textual data and image resizing was performed. We
took textual descriptions from the dataset, preprocessed these caption sentences, and created a
list of their vocabulary. Then, these captions were stored with their respective ids in the list. The
images were loaded and resized to a fixed dimension. These data were then given as input to our
proposed model. RNN was used for capturing the contextual information of text sequences by
defining the relationship between words at altered time stamps. Text-to-image mapping was
performed using an RNN and a CNN. The CNN recognized useful characteristics from the
images without the need for human intervention. An input sequence was given to the RNN,
which converted the textual descriptions into word embeddings with a size of 256. These word
embeddings were concatenated with a 512-dimensional noise vector. To train our model, we
took a batch size of 64 with gated-feedback 128 and fed the input noise and text input to a
generator. The architecture of the proposed model is presented in Figure 1. Eng. Proc. 2022, 20,
16 3 of 6 images were loaded for resizing to the same dimensions. All training images and
testing images were resized to a resolution of 128 × 128.

For training purposes, the images were converted into arrays, and both the vocabulary and
images were loaded onto the model. 4. Proposed Methodology This section describes the training
details of deep learning-based generative models. Conditional GANs were used with recurrent
neural networks (RNNs) and convolutional neural networks (CNNs) for generating meaningful
images from a textual description. The dataset used consisted of images of flowers and their
relevant textual descriptions. For generating plausible images from text using a GAN,
preprocessing of textual data and image resizing was performed. We took textual descriptions
from the dataset, preprocessed these caption sentences, and created a list of their vocabulary.
Then, these captions were stored with their respective ids in the list.

The images were loaded and resized to a fixed dimension. These data were then given as input
to our proposed model. RNN was used for capturing the contextual information of text sequences
by defining the relationship between words at altered time stamps. Text-to-image mapping was
performed using an RNN and a CNN. The CNN recognized useful characteristics from the
images without the need for human intervention. An input sequence was given to the RNN,
which converted the textual descriptions into word embeddings with a size of 256. These word
embeddings were concatenated with a 512-dimensional noise vector. To train our model, we
took a batch size of 64 with gated-feedback 128 and fed the input noise and text input to a
generator. The architecture of the proposed model is presented
Figure 1. Architecture of the proposed method, which can generate images from text
descriptions. Semantic information from the textual description was used as input in the
generator model, which converts characteristic information to pixels and generates the images.
This generated image was used as input in the discriminator along with real/wrong textual
descriptions and real sample images from the dataset. A sequence of distinct (picture and text)
pairings are then provided as input to the model to meet the goals of the discriminator: input
pairs of real images and real textual descriptions, wrong images and mismatched textual
descriptions, and generated images and real textual descriptions. The real photo and real text
combinations are provided so that the model can determine if a particular image and text
combination align. An incorrect picture and real text description indicates that the image does
not match the caption. The discriminator is trained to identify real and generated images. At the
start of training, the discriminator was good at classification of real/wrong images. Loss was
calculated to improve the weight and to provide training feedback to the generator and
discriminator model. As soon as the training proceeded, the generator produced more realistic
images and it fooled the discriminator when distinguishing between real and generated and
images

Video Recommendation System

Video recommendation systems are a fundamental component of many popular streaming

platforms, such as YouTube and Netflix. These systems are tasked with providing users with
engaging and personalized content recommendations. They come in various flavors, each offering
unique approaches to the task of suggesting personalized content to users. Three prominent types
of recommendation systems are content-based, collaborative filtering, and the innovative two-
tower architecture.

Content-Based Recommendation Systems

Content-based recommendation systems operate on the premise of suggesting items to users based
on the content attributes of those items and a user’s past preferences. These systems focus on
features and characteristics associated with items, such as text descriptions, genres, keywords, or
metadata.

The recommendations generated are aligned with the user’s historical interactions and
preferences. Content-based systems excel in providing recommendations that are closely related
to the user’s demonstrated interests. For example, a content-based movie recommendation system
might suggest films with similar genres or themes to those the user has previously enjoyed.

Collaborative Filtering Recommendation Systems

Collaborative filtering recommendation systems, on the other hand, rely on the collective
behavior and preferences of a large user base to make suggestions. This approach assumes that
users who have exhibited similar preferences in the past will continue to do so in the future.
Collaborative filtering can be further categorized into two subtypes: user-based and item-based.
User-based collaborative filtering recommends items to a user based on the preferences of users
who are similar to them. Item-based collaborative filtering suggests items similar to those the user
has shown interest in, based on the behavior of other users. These systems are effective at
suggesting items that are trending or popular among users with similar preferences.

Two-Tower Architecture

The Two-Tower architecture is a cutting-edge recommendation system design that leverages

neural networks to enhance recommendation quality. In this architecture, two separate “towers”
are used to encode user and item (content) information independently.

The user tower processes user data, such as profiles and historical interactions, while the item
tower encodes item features like metadata and content descriptors. By separately encoding user
and content information, the Two-Tower architecture excels in delivering highly personalized
recommendations. It is particularly adept at addressing challenges like the cold start problem,
where it must recommend to new users or new items with limited interaction data. This
architecture is highly efficient, scalable, and capable of fine-tuning recommendations based on
nuanced user preferences.

two tower architecture

Exploring Two-Tower Neural Networks for Enhanced Retrieval

In the realm of retrieval systems, Two-Tower Neural Networks (NNs) hold a special significance.
Our retrieval approach, grounded in machine learning, harnesses the power of the Word2Vec
algorithm to create embeddings for both users and media/authors based on their unique identifiers.

The Two Towers model expands upon the Word2Vec algorithm, permitting the incorporation of
diverse user or media/author characteristics. This adaptation also facilitates concurrent learning
across multiple objectives, enhancing its utility for multi-objective retrieval tasks. Notably, this
model retains the scalability and real-time capabilities inherent in Word2Vec, making it an
excellent choice for candidate sourcing algorithms.

Here’s a high-level overview of how Two-Tower retrieval operates in conjunction with a schema:
1. The Two Tower model comprises two distinct neural networks — one for users and one for
items.

2. Each neural network exclusively processes features pertinent to its respective entity and
generates an embedding.

3. The primary objective is to predict engagement events (e.g., user likes on a post) by measuring
the similarity between user and item embeddings.

4. Following training, user embeddings are optimized to closely match embeddings of relevant
items, enabling the use of nearby item embeddings for ranking purposes.

learning using four popular machine learning algorithms namely, Random Forest Classifier,
KNN, Decision Tree Classifier, and Naive Bayes classifier. We will directly jump into
implementation step-by-step.

classification using machine learning and machine learning image classification. However,
the work demonstrated here will help serve research purposes if one desires to compare their
CNN image classifier model with some machine learning algorithms.

Learning Objective:

• Provide a step-by-step guide to implementing image classification algorithms using

popular machine learning algorithms like Random Forest, KNN, Decision Tree, and
Naive Bayes.

• Demonstrate the limitations of traditional machine learning algorithms for image

classification algorithms tasks and highlight the need for deep learning approaches.

• Showcase how to test the trained models on custom input images and evaluate their
performance.

This article was published as a part of the Data Science Blogathon.

Table of contents

Dataset Acquisition
Source: cs.toronto

The dataset utilized in this blog is the CIFAR-10 dataset, which is a Keras dataset that can be
easily downloaded using the following code. The dataset includes ten classes: airplane,
automobile, bird, cat, deer, dog, frog, horse, ship, and truck, indicating that we will be
addressing a multi-class classification problem.

First, let’s import the required packages as follows:

from tensorflow import keras

import matplotlib.pyplot as plt
from sklearn.metrics import accuracy_score,confusion_matrix,classification_report
import numpy as np
import cv2
The dataset can be loaded using the code below:

(x_train, y_train), (x_test, y_test) = keras.datasets.cifar10.load_data()