Leveraging Federated Learning and Edge Computing

for Recommendation Systems within Cloud

Computing Networks
Yaqian Qi 1* Yuan Feng 1
Quantitative Methods and Modeling Interdisciplinary Data Science
Baruch Collegue, CUNY Duke University
55 Lexington Ave, New York, NY 10010 North Carolina USA
*
Corresponding author: [email protected] [email protected]

Hanzhe Li 3
Xiangxiang Wang 2
Computer Engineering
Computer Science New York University
University of Texas at Arlington New York USA
Arlington, Texas [email protected]
[email protected]
Jingxiao Tian 4
Electrical and Computer Engineering
San Diego State University
San Diego, US
[email protected]

I. INTRODUCTION
Abstract—To enable large-scale and efficient deployment of Linhua Technology, the world's leading edge computing
artificial intelligence (AI), the combination of AI and edge solution provider, and to Star Technology (referred to as
computing has spawned Edge Intelligence, which leverages the "Nebula Clustar") have entered into a cooperation to deal with
computing and communication capabilities of end devices and the data delay and privacy protection issues in centralized
edge servers to process data closer to where it is generated. A key machine learning training, and work together to create an all-
technology for edge intelligence is the privacy-protecting machine in-one machine for edge federated learning. This product uses
learning paradigm known as Federated Learning (FL), which MECS-7211 of Linhua Technology as the edge computing
enables data owners to train models without having to transfer server, and the FPGA privacy computing acceleration card of
raw data to third-party servers. However, FL networks are Nebulstar, fully integrates the advantages of high-performance
expected to involve thousands of heterogeneous distributed privacy computing and edge computing, and effectively meets
devices. As a result, communication efficiency remains a key the business requirements of intensive edge computing
bottleneck. To reduce node failures and device exits, a scenarios. Such as distributed machine learning, gene
Hierarchical Federated Learning (HFL) framework is proposed, sequencing, financial business, medical, video processing,
where a designated cluster leader supports the data owner network security and so on.
through intermediate model aggregation. Therefore, based on the
improvement of edge server resource utilization, this paper can At present, with the rapid development of the Internet of
effectively make up for the limitation of cache capacity. In order Things and the popularity of 5G network, a large number of
to mitigate the impact of soft clicks on the quality of user terminal devices are connected to the network to generate
experience (QoE), the authors model the user QoE as a massive data. Traditional data calculation and analysis are
comprehensive system cost. To solve the formulaic problem, the based on cloud computing, but with the rapid increase of data,
authors propose a decentralized caching algorithm with the process of transmitting from application terminals to cloud
federated deep reinforcement learning (DRL) and federated computing centers will cause delay and data leakage. How to
learning (FL), where multiple agents learn and make decisions process data in a timely and efficient manner has become a
independently. major challenge for cloud computing centers. Edge Computing,
a new computing model for computing at the edge of the
Keywords-Deep learning; Federal learning; Edge computing;
Cloud computing; Intelligent recommendation
network, emerged. At the edge of the network near people,
things or data sources, edge intelligent services are provided
nearby to respond with more efficient network services to meet
the increasing needs of many businesses such as the Internet of
Things, the Internet of vehicles, industrial control, intelligent
manufacturing, and large video.
On the other hand, the introduction of edge computing
technology has reduced the network burden of cloud centers,
but it has also caused security problems, and the localization of
data is easy to hinder the interaction between data, coupled
with the continuous tightening of data security and application
specifications in recent years, such as GDPR data privacy and
data protection issues have been highly valued. The centralized
calculation of data adopted by traditional machine learning Figure 1. Simplified federated learning architecture diagram
algorithms cannot cope with the requirements of data
specifications, which limits the development of artificial The simplified illustration of a Federated Learning
intelligence. architecture is now displayed, showing a more straightforward
view of the concept with a few edge devices connected to a
In this context, Federated machine learning (Federated central server. This design emphasizes the privacy-centric
machine Learning) should be born in time to provide a solution approach of Federated Learning, where only model updates are
to the security problem of edge computing. Federated learning communicated to the server, ensuring that raw data remains on
is a machine learning framework that only exchanges the the local devices.
parameters, weights and gradients of the model after encryption,
without moving the original data out of the local area or At the same time, although federated learning avoids the
moving the encrypted original data into a centralized location, direct exchange of sensitive data between participants, the
which can help multiple organizations to meet the requirements exchange of plaintext parameters still brings the risk of privacy
of user privacy protection, data security and government disclosure. Many existing privacy protection schemes used in
regulations. Therefore, in order to improve caching benefits, federated learning can ensure the security of transmission
this paper proposes an edge cache and recommendation system, parameters, but many privacy protection schemes may bring a
which forms an edge cache system supporting recommendation lot of extra computing and communication costs, and it is
in mobile edge cloud computing network. The proposed system difficult to balance the relationship between privacy protection
supports both direct hit and soft hit. The factors that affect the and efficiency.
user's QoE are modeled as the comprehensive system cost B. Recommendation system based on federated learning
(including similarity cost, delay cost and cache hit cost). The
authors further formulates the cache replacement problem as a Similar to the architecture design in the general federated
multi-agent Markov decision process (MDP) to minimize the learning domain, the architecture used in the research of
expected long-term system cost (reflecting the user QoE). federated recommendation systems can be divided into client-
server architecture and decentralized architecture, as shown in
II. RELATED WORK Figure 2.
A. The combination of edge computing and federated (1) The training process of client-server architecture is as
learning follows:
The combination of edge computing and federated learning  The server initializes the model parameters and sends
is designed to take advantage of edge computing's advantages the model parameters to each client.
in data processing speed and response time, while protecting
user data privacy through the distributed model training  The client uses the local data and the latest model
approach of federated learning. This combination has shown its parameters received from the server for training, and
potential in several areas, especially in recommendation sends the intermediate parameters to the server;
systems, which are able to provide personalized  The server aggregates the intermediate parameters,
recommendations while protecting user privacy. updates the global model, and sends the model back to
Federated Learning (FL) is a distributed machine learning the client.
framework that allows users to train models using local data Repeat steps (2) and (3) until the model converges.
sets. For participating nodes, they need to have sufficient
computing power, network bandwidth and storage capacity. Features: This type of architecture can use computing
But in federal edge learning (FEL), edge device resources are resources on the server side and reduce the computing pressure
relatively limited. These devices need to operate for a long time on the client side, but it is prone to single point of failure. At
and are capable of constantly updating their models. They the same time, for the curious server, it may infer the privacy
participate in the training process only when they are free and information of the client according to the intermediate
can withdraw from the process at any time. As a result, the parameters uploaded by the client, thus revealing the privacy of
overall system needs to be more flexible, scalable, and support the client.
low-overhead long-term model training and updating
requirements to meet the constraints and requirements of edge
devices.
 D. Advanced Technologies and Challenges in Privacy
Protection for Federated Learning and Edge Computing
The integration of federated learning (FL) with edge
computing has heralded a new era in privacy-preserving
artificial intelligence (AI), allowing data processing closer to
the source and minimizing privacy risks. However, deploying
these technologies at scale involves navigating a complex
landscape of advanced privacy-preserving techniques and
inherent challenges.
Figure 2. Federated Learning intelligent recommendation architecture design
 Advanced Technologies Overview
(2) The training process of decentralized architecture is as
follows:
Secure Multi-Party Computation (SMPC): SMPC enables
The server initializes the model parameters and then sends
parties to jointly compute a function over their inputs while
the model parameters to each client. keeping those inputs private. Its application in FL can protect
The client uses local data to train the model and update the data during aggregation, but it demands significant
local model parameters. computational and communication resources, posing a
challenge for edge devices.
The client selects some other clients, sends the local
intermediate parameters, receives the intermediate parameters Homomorphic Encryption (HE): HE allows computations
of other clients, and updates the local model; to be performed on encrypted data, providing results that, when
decrypted, match the results of operations performed on the
Repeat steps (2) and (3) until the model converges. plaintext. While HE offers strong privacy guarantees, its high
Features: computational overhead limits its practicality for resource-
constrained edge devices.
 Anonymity. In the process of model training, the client
can anonymously send intermediate parameters to Differential Privacy (DP): DP introduces randomness to the
other clients, so as to solve the problem of privacy data or the model outputs, ensuring that individual data
disclosure between clients. contributions are masked. Implementing DP in FL helps
mitigate the risk of information leakage but balancing privacy
 Save server resources. The server only needs to with data utility remains a critical challenge.
initialize the model parameters and distribute the
model parameters to each client, and does not need to  Practical Applications
participate in the update of the model. Several industries are pioneering the application of these
 High availability. There is no single point of failure, technologies, from healthcare, where patient data sensitivity is
that is, the failure of a single component of the server paramount, to finance and smart manufacturing, where
will not cause the entire federated learning system to operational efficiency and data privacy must coexist. These
stop training. applications demonstrate the potential of FL and edge
computing to revolutionize privacy-preserving data analysis
The similarities between the two: the original data of the and decision-making.
client does not leave the local, and a shared model is obtained
by sending intermediate parameters through the  Inherent Challenges
communication between the server and the client or the c Computational and Communication Costs: The primary
C. Federalization of the recommendation system bottleneck is the high cost of implementing privacy-preserving
algorithms on edge devices, which often have limited
The federalization of recommendation models has some computational power and energy resources.
commonality, and the training framework of a federalized
recommendation model is usually suitable for other models Technology Integration and Compatibility: Combining
with the same training mode. However, considering the different privacy-preserving technologies in a cohesive manner
different levels of privacy protection in different scenarios and that aligns with the decentralized nature of FL and edge
the different privacy issues that may be brought by different computing requires innovative architectural solutions.
model parameters, there are certain differences in the
Regulatory Compliance and Standards: Navigating the
federalization process of different recommendation models.
global landscape of data protection regulations, such as GDPR,
The federalization of models can be divided into three
poses challenges for the deployment of FL and edge computing
categories: the federalization of recommendation algorithms
solutions across borders.
based on collaborative filtering, the federalization of
recommendation algorithms based on deep learning and the  Future Research Directions
federalization of recommendation algorithms based on meta-
learning. Addressing these challenges calls for ongoing research into
optimizing privacy-preserving algorithms for efficiency,
developing lightweight versions suitable for edge devices, and
creating flexible frameworks that can adapt to various
regulatory environments. Moreover, exploring AI and machine (6)
learning techniques to enhance the efficiency of these
Thus, in a PFL considering model pruning, the optimization
technologies presents a promising avenue for future work.
objective is defined as:
By advancing our understanding and capabilities in these
areas, we can unlock the full potential of federated learning and
edge computing as cornerstones of a privacy-preserving digital
future. (7)

(8)
III. SYSTEM DESCRIPTION AND FUNCTION DEFINITION
Represents the model parameter matrix for all device
A. Personalized federated learning
combinations.
Consider a decentralized federated learning system with N
edge devices. Each device trains a personalized model x; € C. The process of proposing the method
Rd(d is the dimension of model xi). Thus, the local objective To minimize this, we propose a method that combines
function of device i is defined as follows: adaptive model pruning and neighbor selection, as shown in
Figure 1. The proposed method mainly includes two steps:
(1)
Where fi(xi) is the local loss function of device i, hi(xi)
represents the regularization function, and in represents the
regularization factor. In this article, we use model
regularization to facilitate collaboration between each device
and its neighbors. For device i, its regularization function can
be formalized as:

ℎ� (�� ) = \\ �� − �� \\2
1
\��\ �∈��
(2)
Figure 3. A method of joint model pruning and neighbor selection is
proposed
Si indicates the neighbor set of device i, and ||·|| indicates
L2 normal pattern. IV. EXPERIMENTAL EVALUATION
B. Combined with model pruning PFL This paper introduces our novel algorithm, DPMN, and
In this paper, we use adaptive model pruning to improve the validates its effectiveness through experiments conducted on
four widely recognized datasets:
following is the model formula function: �� ∈ 0,1 � If
communication efficiency of equipment. For device i, the
A. Experimental data
�� (k)=1, it means that the KTH neuron of the local model xi is  Fashion-MNIST: Comprising 70,000 grayscale images
preserved. If �� (k)=0, the KTH neuron of xi is discarded. Thus, distributed across 10 categories, each category contains

after pruning the model can be pressed �� = �� °�� .In addition,

7,000 images. Specifically, the dataset is split into a
training set with 6,000 images per category and a test
due to model pruning, the regularization function of device i set with 1,000 images per category. We employed the
can be redefined as: LeNet5 model for training on this dataset.
 CIFAR-10: This dataset includes 60,000 color images,
categorized into 10 classes with each class having
(3) 6,000 images. It is divided into a training set with
5,000 images per class and a test set with 1,000 images
per class. The LeNet5 model was also used for training
(4) on CIFAR-10.

��∗� = �� °�� represents the overlap between the device and

Is a commonly used negative exponential function that  CEMNIST: Featuring a collection of 731,668 training
samples and 82,587 test samples, the AlexNet model
the device's model mask. was utilized for training on this dataset.

In this article, we aim to minimize the loss function on the  IMAGENET: This extensive dataset consists of
local data set: 1,281,267 training samples, 50,000 validation samples,
and 100,000 test samples, with each sample being a
color image of size 224x224x3. Training on
(5) IMAGENET was conducted using the VGG16 model.
And regularization function:
 By employing these datasets, which vary significantly in D. Experimental conclusion
size, complexity, and image type (ranging from grayscale to The experimental evaluation of our novel algorithm,
color images), we were able to thoroughly assess the DPMN (Decentralized Personalized Model with Neighbor
performance and versatility of the DPMN algorithm across selection), underscores its significant potential in enhancing
different visual recognition tasks. federated learning applications within the domain of
B. Experimental verification recommendation systems, especially when integrated with edge
computing strategies in cloud computing networks. The
 DFL: Traditional decentralized training method. experiments conducted across diverse datasets — ranging from
 DPMP: A personalized model training method using Fashion-MNIST and CIFAR-10 to CEMNIST and
model pruning IMAGENET — demonstrate the versatility and efficiency of
DPMN in handling various visual recognition tasks, which are
 DPCG: Neighbor selection method based on Euclidean pivotal in the context of personalized recommendation systems.
distance for personalized model training
The comparison between DPMN and other decentralized
 Dpmn-r: Variant of DPMN, where neighbors are training methods, including traditional decentralized learning
randomly selected (DFL), personalized model training with model pruning
C. Experimental result (DPMP), and neighbor selection based on Euclidean distance
(DPCG), highlights the superiority of DPMN in balancing
model quality with communication efficiency. Particularly,
when the threshold of cosine similarity is set to 0.2, DPMN
exhibits optimal performance, achieving a significant reduction
in bandwidth resource consumption by an average of 45.4%.
This efficiency is critical in cloud computing environments
where bandwidth is a valuable resource.
Moreover, our findings reveal that the device selection
strategy proposed by DPMN leads to higher model accuracy
compared to random neighbor selection methods (Dpmn-r
variant). This outcome is crucial for recommendation systems,
where the accuracy of personalized recommendations directly
impacts user satisfaction and engagement.
In conclusion, the application of federated learning
leveraging edge computing, as embodied by our DPMN
algorithm, presents a promising approach to improving
Figure 4. δImpact on DPMN performance recommendation systems within cloud computing networks.
By efficiently managing communication overhead and

δthreshold of cosine similarity is set to 0.2, DPMN has the best

It can be seen from the experimental results that when the enhancing model accuracy through personalized and context-
aware learning, DPMN sets a new benchmark for the
effect and can achieve the balance between model quality and development of more responsive, efficient, and user-centric
communication efficiency. recommendation systems in the era of cloud and edge
computing. This research not only validates the effectiveness
of DPMN but also opens avenues for further exploration into
optimizing federated learning frameworks for complex, data-
driven applications in cloud computing ecosystems.
V. CONCLUSIONS
In this comprehensive study, we introduced the
Decentralized Personalized Model with Neighbor selection
(DPMN) algorithm and provided a thorough experimental
Figure 5. Accuracy vs. number of devices evaluation across a variety of datasets to demonstrate its
applicability and effectiveness in federated learning
From this group of experiments, it can be seen that in the environments, especially within the context of recommendation
same number of training rounds, our DPMN can significantly systems integrated with edge computing in cloud computing
reduce the bandwidth resource consumption of model training, networks. Our findings not only showcase the potential of
saving an average of 45.4% traffic overhead. When different DPMN in addressing the challenges of decentralized training
number of devices participate in model training, the proposed methods but also highlight the synergy between federated
device selection strategy can achieve higher model accuracy learning and edge computing as a transformative approach for
than the random neighbor selection method. privacy-preserving artificial intelligence (AI) applications.
A. Conclusions on the Advantages of Cloud Computing and
Deep Reinforcement Learning [1] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas,
“Communication-efficient learning of deep networks from decentralized
Our research underscores the substantial benefits of data,” in Artificial intelligence and statistics. PMLR, 2017, pp. 1273–
leveraging cloud computing and deep reinforcement learning in 1282.
enhancing the performance and efficiency of federated learning [2] Liu, B., Zhao, X., Hu, H., Lin, Q., & Huang, J. (2023). Detection of
systems. The integration of cloud computing provides a Esophageal Cancer Lesions Based on CBAM Faster R-CNN. Journal of
scalable and flexible infrastructure capable of handling the Theory and Practice of Engineering Science, 3(12), 36–42.
https://doi.org/10.53469/jtpes.2023.03(12).06
immense computational demands of federated learning,
[3] C. T Dinh, N. Tran, and T. D. Nguyen, “Personalized federated learning
particularly in the training and deployment of complex AI with moreau envelopes,” Advances in Neural Information Processing
models across distributed networks. Furthermore, the Systems, vol. 33, 2020.
application of deep reinforcement learning enables adaptive [4] Liu, Bo, et al. "Integration and Performance Analysis of Artificial
and intelligent decision-making processes within federated Intelligence and Computer Vision Based on Deep Learning
learning frameworks, optimizing resource allocation, and Algorithms." arXiv preprint arXiv:2312.12872 (2023).
improving the overall system efficiency and model accuracy. [5] A. Li, J. Sun, P. Li, Y. Pu, H. Li, and Y. Chen, “Hermes: an efficient
federated learning framework for heterogeneous mobile clients,” in
B. Future Prospects Proceedings of the 27th Annual International Conference on Mobile
Computing and Networking, 2021, pp. 420–437.
Looking ahead, the continued evolution of cloud computing
[6] A. Ghosh, J. Chung, D. Yin, and K. Ramchandran, “An efficient
technologies and advancements in deep reinforcement learning framework for clustered federated learning,” Advances in Neural
algorithms hold significant promise for further enhancing Information Processing Systems, vol. 33, 2020.
federated learning systems. The potential for developing more [7] Yu, L., Liu, B., Lin, Q., Zhao, X., & Che, C. (2024). Semantic Similarity
sophisticated and efficient model training and aggregation Matching for Patent Documents Using Ensemble BERT-related Model
methods, coupled with the optimization of resource utilization and Novel Text Processing Method. arXiv preprint arXiv:2401.06782.
in cloud environments, could lead to even greater [8] Y. Huang, L. Chu, Z. Zhou, L. Wang, J. Liu, J. Pei, and Y. Zhang,
improvements in AI model performance and system scalability. “Personalized cross-silo federated learning on non-iid data,” in
Additionally, the exploration of novel privacy-preserving Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35,
no. 9, 2021, pp. 7865–7873.
techniques and security measures will be crucial in addressing
[9] V. Zantedeschi, A. Bellet, and M. Tommasi, “Fully decentralized joint
the evolving challenges of data protection and regulatory learning of personalized models and collaboration graphs,” in
compliance in federated learning applications. International Conference on Artificial Intelligence and Statistics. PMLR,
2020, pp. 864–874.
C. Advantages of Federated Learning and Edge Computing
[10] He, Zheng & Shen, Xinyu & Zhou, Yanlin & Wang, Yong. (2024).
for Privacy Protection Application of K-means clustering based on artificial intelligence in
The integration of federated learning with edge computing gene statistics of biological information engineering.
10.13140/RG.2.2.11207.47527.
emerges as a particularly compelling solution for privacy
protection in AI applications. By enabling model training to [11] S. Wang, T. Tuor, T. Salonidis, K. K. Leung, C. Makaya, T. He, and K.
Chan, “Adaptive federated learning in resource constrained edge
occur directly on edge devices without the need to transfer raw computing systems,” IEEE Journal on Selected Areas in
data to centralized servers, this approach significantly mitigates Communications, vol. 37, no. 6, pp. 1205–1221, 2019.
privacy risks and data leakage concerns. Moreover, the [12] Kulkarni, Viraj, Milind Kulkarni, and Aniruddha Pant. “Survey of
decentralized nature of federated learning, combined with the Personalization Techniques for Federated Learning.” ArXiv:2003.08673
computational capabilities of edge computing, facilitates real- [Cs, Stat], March 19, 2020. http://arxiv.org/abs/2003.08673
time, personalized AI applications across various domains, [13] Tan, Alysa Ziying, Han Yu, Lizhen Cui, and Qiang Yang. “Towards
from healthcare and finance to smart cities and IoT, while Personalized Federated Learning.” IEEE Transactions on Neural
Networks and Learning Systems, 2022, 1–
upholding stringent privacy and data security standards. 17. https://doi.org/10.1109/TNNLS.2022.3160699
D. Concluding Remarks [14] McMahan, Brendan, Eider Moore, Daniel Ramage, Seth Hampson, and
Blaise Aguera y Arcas. “Communication-Efficient Learning of Deep
The convergence of federated learning, edge computing, Networks from Decentralized Data.” In Artificial Intelligence and
and cloud computing represents a powerful paradigm shift in Statistics, 1273–82. PMLR,
the development and deployment of AI systems. Our research 2017. http://proceedings.mlr.press/v54/mcmahan17a.html.
on the DPMN algorithm illustrates the efficacy of this [15] K.Tan and W. Li, "Imaging and Parameter Estimating for Fast Moving
integrated approach in enhancing recommendation systems, Targets in Airborne SAR," in IEEE Transactions on Computational
Imaging, vol. 3, no. 1, pp. 126-140, March 2017, doi:
with broader implications for various AI-driven applications. 10.1109/TCI.2016.2634421.
As we continue to navigate the complexities of data privacy, [16] K. Tan and W. Li, "A novel moving parameter estimation approach
security, and the ever-growing demand for intelligent, offast moving targets based on phase extraction," 2015 IEEE
personalized services, the insights gained from this study will International Conference on Image Processing (ICIP), Quebec City, QC,
undoubtedly contribute to the advancement of more secure, Canada, 2015, pp. 2075-2079, doi: 10.1109/ICIP.2015.7351166.
efficient, and user-centric AI solutions in the cloud and edge
computing era.
REFERENCES