0% found this document useful (0 votes)

34 views

Deep Chain

This article proposes a new distributed, secure and fair deep learning framework called DeepChain that uses blockchain technology. DeepChain provides an incentive mechanism to ensure participants behave correctly during model training. It also guarantees data privacy for each participant and provides auditability of the entire training process.

Uploaded by

PabloGomezPerez

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

34 views

Deep Chain

Uploaded by

PabloGomezPerez

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 18

This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TDSC.2019.2952332, IEEE
Transactions on Dependable and Secure Computing
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 1

DeepChain: Auditable and Privacy-Preserving

Deep Learning with Blockchain-based Incentive
Jiasi Weng, Jian Weng, Member, IEEE, Jilian Zhang, Ming Li, Yue Zhang, Weiqi Luo

Abstract—Deep learning can achieve higher accuracy than traditional machine learning algorithms in a variety of machine learning
tasks. Recently, privacy-preserving deep learning has drawn tremendous attention from information security community, in which
neither training data nor the training model is expected to be exposed. Federated learning is a popular learning mechanism, where
multiple parties upload local gradients to a server and the server updates model parameters with the collected gradients. However,
there are many security problems neglected in federated learning, for example, the participants may behave incorrectly in gradient
collecting or parameter updating, and the server may be malicious as well. In this paper, we present a distributed, secure, and fair deep
learning framework named DeepChain to solve these problems. DeepChain provides a value-driven incentive mechanism based on
Blockchain to force the participants to behave correctly. Meanwhile, DeepChain guarantees data privacy for each participant and
provides auditability for the whole training process. We implement a DeepChain prototype and conduct experiments on a real dataset
for different settings, and the results show that our DeepChain is promising.

Index Terms—Deep learning, Privacy-preserving training, Blockchain, Incentive

1 I NTRODUCTION

R ECENT advances in deep learning based on artificial

neural networks have witnessed unprecedented accu-
racy in various tasks, e.g., speech recognition [1], image
gradients to the parameter server. Upon receipt of the gra-
dients from all the parties, the parameter server aggregates
those gradients and updates the learning model parameters
recognition [2], drug discovery [3] and gene analysis for accordingly, after which each of the parties downloads the
cancer research [4], [5]. In order to achieve even higher updated parameters from the server and continues to train
accuracy, huge amount of data must be fed to deep learning her model on the same local data again with the down-
models, incurring excessively high computational overhead loaded parameters. This training process repeats until the
[6], [7]. This problem, however, can be solved by employing training errors are smaller than pre-specified thresholds.
distributed deep learning technique that has been inves- This federated learning framework, however, cannot
tigated extensively in recent years. Unfortunately, privacy protect the privacy of the training data, even the training
issue worsens in the context of distributed deep learning, data is divided and stored separately. For example, some re-
as compared to conventional standalone deep learning sce- searchers show that the intermediate gradients can be used
nario. to infer important information about the training data [17],
Privacy-preserving deep learning thus arises to deal with [18]. Shokri et. al [11] applied differential privacy technique
privacy concerns in deep learning, and various models have by adding noises in the gradients to upload, achieving a
been around in the past few years [8], [9], [10], [11], [12], [13], trade-off between data privacy and training accuracy. Hitaj
[14], [15], [16]. Among these existing work, federated learning et. al [19] pointed out that Shokri’s work failed to protect
is the widely adopted system context. Federated learning, data privacy and demonstrated that a curious parameter
also known as collaborative learning, distributed learning, is server can learn private data through GAN (Generative Ad-
essentially the combination of deep learning and distribut- versarial Network) learning. Orekondy et. al [20] exploited
ed computation, where there is a server, called parameter the intermediate gradients to launch linkability attack on
server, maintaining a deep learning model to train and training data, since the gradients contain sufficient data
multiple parties that take part in the distributed training features.
process. First, the training data is partitioned and stored at Phong et. al [16] proposed to use homomorphic en-
each of the parties. Then, each party trains a deep learning cryption technique to protect training data privacy from
model (the same one as maintained at the parameter server) curious parameter server. The drawback of their scheme is
on her local data individually, and uploads intermediate that they assumed the collaborative participants are honest
but not curious, hence their scheme may fail in scenario
• J. S. Weng, J. Weng, J. L. Zhang, M. Li, Y. Zhang and W. Q. Luo
where some participants are curious. To prevent curious
are with the College of Information Science and Technology in Jinan participants, Bonawitz et. al [14] employed a secret sharing
University, and Guangdong/Guangzhou Key Laboratory of Data Security and symmetric encryption mechanism to ensure confiden-
and Privacy Preserving, Guangzhou 510632, China. tiality of the gradients of participants. They assumed that
E-mail addresses: [email protected] (J. S. Weng), cryp-
[email protected] (J. Weng), [email protected](J. L. Zhang), (1) participants and parameter server cannot collude at all,
[email protected] (M. Li), [email protected] (Y. Zhang), and (2) the aggregated gradients in plain text reveal nothing
[email protected] (W. Q. Luo). about the participants’ local data. The second assumption,
Jian Weng is the corresponding author.
unfortunately, is no longer valid since membership inference

attack on aggregated location data is now available [21].

Despite extensive research is underway on distributed
deep learning, there are two serious problems that receive
less attention so far. The first one is that existing work gen-
erally considered privacy threats from curious parameter
server, neglecting the fact that there exist other security
threats from dishonest behaviors in gradient collecting and
parameter update that may disrupt the collaborative train-
ing process. For example, the parameter server may drop
gradients of some parties deliberately, or wrongly update
model parameters on purpose. Recently, Bagdasaryan et.
al [22] demonstrated the existence of this problem that
Fig. 1. The left corresponds to traditional distributed deep training frame-
dishonest parties can poison the collaborative model by
work, while the right is our DeepChain. Here, Trading Contract and
replacing the updating model with its exquisitely designed Processing Contract are smart contract in DeepChain, together guiding
one. Therefore, it is crucial for distributed deep learning the secure training process, while Tx refers to transaction.
framework to guarantee not only confidentiality of gradi-
ents, but also auditability of the correctness of gradient
collecting and parameter update. trusted parties through launching transactions, while local
The second problem is that in existing schemes those training and parameter update are performed by workers
parties are assumed to have enough local data for train- (an entity in DeepChain that will be defined shortly) who
ing and are willing to cooperate in the first place, which are incented to process the transactions. Through trans-
are not always true in real applications. For example, in action processing and incentive mechanism, DeepChain
healthcare applications, companies or research institutes are achieves collaborative training. Meanwhile, by using cryp-
usually facing the difficulty in collecting enough personal tographic techniques we ensure data confidentiality and
medical data, due to privacy regulations such as HIPAA auditability of the collaborative training process as well. To
[23], people’s unwillingness to share and malicious attacks summarize, in this paper we made the following contribu-
like identifying inference attacks against HCUPnet [24]. As tions:
a consequence, lack of training data will result in poor
deep learning models in general [25]. On the other hand, in • We propose DeepChain, a collaborative training
business applications some companies may be reluctant to framework with an incentive mechanism that en-
participate in collaborative training, because they are very courages parties to jointly participate in deep learn-
concerned about possible disclosure of their valuable data ing model training and share the obtained local gra-
during distributed training [11]. Obviously, it is vital to dients.
ensure data privacy and bring in some incentive mechanism • DeepChain preserves the privacy of local gradients
for distributed deep learning, so that more parties can and guarantees auditability of the training process.
actively involved in collaborating training. By employing incentive mechanism and transactions,
Traditional incentive mechanisms can be categorized participants are pushed to behave honestly, partic-
into three types: reputation-based [26], tit-for-tat [27] and ularly in gradient collecting and parameter update,
payment-based mechanism [28], [29]. Usually, these mech- thus maintaining fairness during collaboration train-
anisms, except for tit-for-tat, need a trusted centralized ing.
authority to audit participant behaviors and arbitrate their • We implement DeepChain prototype and evaluate
payoff. Unfortunately, they fail to provide public auditabil- its performance in terms of cipher size, throughput,
ity and decision fairness. Although there is no trusted cen- training accuracy and training time. We believe that
tralized party in tit-for-tat, it is not suitable for our setting, DeepChain can benefit AI and machine learning
because a party’s contribution is not symmetric to that of her communities, for example, it can audit collaborative
counterparts. It is worth noting that Blockchain, originated training process and the trained model, which rep-
from decentralized currencysystem, enables distrustful n- resents the learned knowledge. Well-trained models
odes to share a common transaction ledger without the need can be used for paid services when the model-based
of a trusted third party, by employing a consensus protocol pricing market is mature. In addition, making the
and financial incentives. This motivates us to introduce a best use of this learned knowledge by combining
payment-based incentive mechanism that guarantees public transfer learning technique can improve both the
authority and fairness. learning efficiency and accuracy.
In this paper, we propose DeepChain, a secure and
decentralized framework based on Blockchain-based incen- The rest of the paper is organized as follows. In Section 2,
tive mechanism and cryptographic primitives for privacy- we give a brief introduction of Blockchain and deep learning
preserving distributed deep learning, which can provide model training. Then, we describe the threat model and
data confidentiality, computation auditability, and incen- security requirements in Section 3. In Section 4, we present
tives for parties to participate in collaborative training. The our DeepChain, a framework for auditable and privacy-
system models of traditional distributed deep learning and preserving deep learning, and analyze security properties of
our DeepChain are given in Fig. 1. Specifically, DeepChain DeepChain in Section 5. We give implementation details of
can securely aggregate local intermediate gradients from un- DeepChain in Section 6, and conduct extensive experiments

to evaluate its performance. Finally, we conclude the paper 2.2 Deep learning and distributed deep learning
in Section 7. A typical deep learning model consists of three layers,
namely input layer, hidden layer and output layer. A deep
2 BACKGROUND
learning model can contain multiple hidden layers, where
Our work is closely related to Blockchain and deep learning the number of layers is called depth of the model. Each
training, and we give background knowledge in this section. hidden layer can have certain number of neurons, and
neurons at different layers can learn hierarchical features of
2.1 Blockchain technology
the input training data, which represent different levels of
Blockchain has arisen a surge of interests both in research abstraction. Each neuron has multiple inputs and a single
community and industry [30]. It becomes an emerging output. Generally, the output of neuron i at layer l − 1
technology as a decentralized, immutable, sharing and time- connects to the input of each neuron at layer l. For the
ordered ledger. Transactions are stored in blocks that contain connection between two neurons, there is a weight assigned
timestamps and references (i.e., the hash of previous block), to it. For example, wi,j is a weight assigned to the connection
which are maintained as a chain of blocks. In Bitcoin, between neuron i at layer l − 1 and neuron j at layer l.
transactions are created by pseudonymous participants and Each neuron i also has a bias bi . These weights and bias are
competitively collected to build a new block by an entity called model parameters, which need to be learned during the
called worker. The worker who generates a new and valid training.
block can gain some amount of rewards, hence the chain is Back-Propagation (BP) [40] is the most popular learning
continuously lengthened by workers. To achieve this, proof method for deep learning, which consists of feed forward
of work (PoW)-based consensus protocol and incentive step and back-propagation step. Specifically, in feed forward
mechanism are required. step, the outputs at each layer are calculated based on
There are a wide variety of consensus protocols, such as parameters at previous layer and current layer, respectively.
proof of stake (PoS)-based, byzantine fault tolerance (BFT)-
A key component in deep neural network training is
based and hybrid protocols. In general, when introducing a
called activation, which is the output of each neuron. Ac-
new consensus protocol for a Blockchain setting, one needs
tivation is used to learn non-linear features of inputs via
to consider six problems: (1) leader selection, i.e., how to select
function Act(·). To compute the output value of a neuron
a new block leader in each round, (2) network model, i.e.,
i at layer l, Act(·) takes all the n inputs of i from layer
the message communication mode, such as asynchronous,
l − 1 as the input. In addition, we assume that weight wj,i is
synchronous, and semi-synchronous, (3) system model, i.e.,
associated with the connection between neurons j at layer
permissioned or permissionless system model, explaining
l − 1 and neurons i at layer l, and bi is the bias of neuron
whether a party can join the system freely, (4) communication
i. Then, the value of neuron i at layer l can be obtained by
complexity, reflecting the communication cost to propagate
Acti (l) = Acti (Σnj=1 (wj,i ∗ Actj (l − 1)) + bi ).
a new block to all parties in the system in each round, (5)
The back-propagation step employs gradient descent
adversary assumption, defining the probability of tolerating
method, which gradually reduces the model error Etotal ,
fault parties in the system, and (6) consensus property, corre-
i.e., the gap between model output value Voutput and the
sponding to the Agreement-Validity-Termination properties
target value Vtarget . Assume that there are n output units at
defined in classic consensus protocols [31].
the output layer. Then, the gap can be calculated by Etotal =
The latest Algorand protocol [32], [33] is a hybrid con- 1 n 2
sensus protocol based on PoS and BFT. Different from PoW- 2 Σi=1 (Vtargeti −Voutputi ) . Once Etotal is available, weights
based consensus protocol, Algorand can guarantee consen-
wj,i can be updated through wj,i = wj,i − η ∗ ∂E ∂wj,i , where
total

sus finality with overwhelming probability in terms of con- η is the learning rate and ∂E ∂wj,i is the partial derivative of
total

sensus property. Here, consensus finality means that a valid Etotal with respect to wj,i . This is the main idea of gradient
block appended to the chain will never be removed in the descent method. The learning process repeats until the pre-
future, which is especially suitable for our problem. Without specified number of iterations to train is reached.
block data abandonment, we avoid spending excessive time When training a complex and multi-layer deep learn-
and computation power to retrain a huge model. Also, Al- ing model, the aforementioned training procedure requires
gorand protocol works in permissioned environment with high computational overhead. To alleviate this problem, dis-
the assumption of a synchronous network, which can be tributed deep learning training has been proposed recently,
adapted to our setting. and some research work [41], [42], [43], [44], [45] and sys-
Some latest Blockchain techniques, such as Ethereum tem implementations have been around, such as DistBelief
and Hyperledger, introduce smart contract that support- [46], Torch [47], DeepImage [48] and Purine [49]. Generally,
s Turing-complete programmability. Other researchers use there are two approaches for distributed training, namely,
these techniques to solve specific security issues in different model parallelism and data parallelism, where the former
application scenarios such as software-update management partitions a training model among multiple machines and
[34], cloud storage [35] and machine learning [36]. On the the latter splits up the whole training dataset.
other hand, a series of work on transaction privacy apply Our work focuses on the data parallelism approach, i.e.,
cryptographic tools in Blockchain, such as Zerocash [37], we have multiple machines and each machine maintains a
Zerocoin [38] and Hawk [39]. In general, consensus protocol copy of the training model while keeps a subset of the whole
and incentive mechanism in Blockchain are key ingredients dataset as model input. These machines share the same pa-
for us to solve our problems, i.e., absence of incentive rameters of the training model, by uploading/downloading
function and collaboration fairness guarantee. parameters to/from a centralized parameter server. Then,

machines upload their local training gradients, based on through transactions that will be recorded in DeepChain.
which the training model is updated by using SGD (Stochas- These transactions are auditable as well, and computation
tic Gradient Descent). They download updated parameters results are guaranteed to be correct only if at least 23 work-
from the parameter server and continue to train the local ers are honest. After parameters are updated, participants
model. This process repeats until machines obtain the final download and collaboratively decrypt the parameters by
trained model. providing their decryption shares and corresponding proofs
for correctness verification. Again, any third party can audit
whether the decryption shares are correct or not.
3 T HREATS AND SECURITY GOALS Security Goal 2: Fairness guarantee for partici-
In this section, we discuss threats to collaborative learning, pants. DeepChain provides fairness for participants through
and security goals that DeepChain can achieve to tackle timeout-checking and monetary penalty mechanism. Specif-
those threats. ically, for each function with smart contracts DeepChain
Threat 1: Disclosure of local data and model. Although defines a time point for it. At the time point after function
in distributed deep training each party only uploads her execution, results of the function are verified. If the verifi-
local gradients to the parameter server, adversaries still can cation failed, it means that (1) there exist participants not
infer through those gradients important information about being punctual by the time point, and (2) some participants
the party’s local data by initiating an inference attack or may incorrectly execute the function. For either of the two
membership attack [18]. On the other hand, based on the cases, DeepChain applies the monetary penalty mechanism,
gradients, adversaries may also launch parameter inferring revoking the pre-frozen deposit of dishonest participants
attack to obtain sensitive information of the model [19]. and re-allocating it to the honest participants. Therefore,
Security Goal: Confidentiality of local gradients. As- fairness can be achieved, because penalty will never be
sume that participants do not expose their own data and imposed on honest participants behaved punctually and
at least t participants are honest (i.e., no more than t correctly, and they will be compensated if there exist dis-
participants colluded to disclose parameters). Then each honest participants.
party’s local gradients cannot be exposed to anyone else,
unless at least t participants collude. In addition, if in any
circumstance participants do not disclose the downloaded 4 T HE D EEP C HAIN M ODEL
parameters from the collaborative model, then adversaries In this section, we present DeepChain, a secure and decen-
could not gain any information about the parameters. To tralized framework for privacy-preserving deep learning.
achieve this goal, in DeepChain each participant individu-
ally encrypts and then uploads gradients obtained from her
local model. All gradients are used to update parameters 4.1 System overview
of the collaborative model encrypted collaboratively by all Before introducing DeepChain, we give definitions of relat-
participants, who then obtain updated parameters via col- ed concepts and terms used in DeepChain.
laborative decryption in each iteration. Here, collaborative Party: In DeepChain, a party is the same entity as defined
decryption means that at least t participants provide their in traditional distributed deep learning model, who has
secret shares to decrypt a cipher. similar needs but unable to perform the whole training
Threat 2: Participants with inappropriate behaviors. task alone due to resource constraints such as insufficient
Consider a situation that participants may have malicious computational power or limited data.
behaviors during collaborative training. They may choose Trading: When a party gets her local gradients, she sends
their inputs at will and thus generate incorrect gradients, out the gradients by launching a transaction to a smart
aiming to mislead the collaborative training process. As a contract called trading contract to DeepChain. This process is
consequence, when updating parameters of collaborative called trading. Those contracts can be downloaded to process
model using the uploaded gradients, it is inevitable that by worker (an entity in DeepChain that will be defined
we will get erroneous results. On the other hand, in collab- shortly).
orative decryption phase dishonest participants may give Cooperative group: A cooperative group is a set of parties
a problematic decryption share and they may be selfish, who have the same deep learning model to train.
aborting local training process early to save their cost for Local model training: Each party trains her local model
training. In addition, dishonest participants may delay trad- independently, and at the end of a local iteration the party
ing or terminate a contract for her own benefit, which makes generates a transaction by attaching her local gradients to
the honest ones suffer losses. All these malicious behaviors the contract.
may fail the collaborative training task. Collaborative model training: Parties of a cooperative
Security Goal 1: Auditability of gradient collecting and group train a deep learning model collaboratively. Specif-
parameter update. In DeepChain, assume that majority of ically, after deciding a same deep learning model and pa-
the participants and at least 23 of the workers are honest rameter initialization, the model is trained in an iterative
in gradient collecting and parameter update, respectively. manner. In each iteration, all parties trade their gradients,
During gradient collecting, participants’ transactions con- and workers download and process the gradients. The pro-
tain encrypted gradients and correctness proofs, allowing cessed gradients are then sent out by workers to the smart
the third party to audit whether a participant gives a correct- contract called processing contract. These correctly processed
ly encrypted construction of gradients. For parameter up- gradients are used to update parameters of the collaborative
date, on the other hand, workers claim computation results model by the leader selected from the workers. Parties

download the updated parameters of the collaborative mod- asset statement, cooperative training and consensus proto-
el and update their local models accordingly. After that col.
parties begin next iteration of model training.
Worker: Similar to miners in BitCoin, workers are incented 4.2.1 DeepChain bootstrapping
to process transactions that contain training weights for DeepChain bootstrapping consists of two steps, i.e., Deep-
collaborative model update. Workers compete to work on Coin distribution and genesis block generation. Assume that
a block, and the first one finishes the job is a leader. The all parties and workers have registered (i.e., have a valid
leader will gain block rewards that can be consumed in the account) in DeepChain, where each one uses an address
future, for example, she may use rewards to pay for usage pk that corresponds to a DeepCoin unit for launching a
fee of trained models in DeepChain. transaction.
Iteration: Deep learning model training consists of multi- In the first step, DeepCoin distribution realizes Deep-
ple steps called iterations, where at the end of each iteration Coin allocation among parties and workers, and initially
all the weights of neurons of the model are updated once. each party or worker is allocated with the same amount
Round: In DeepChain, a round refers to the process of the of DeepCoins. Then in the second step, a genesis block is
creation of a new block. generated at round 0, which contains initial transactions
DeepCoin: DeepCoin, denoted as $Coin, is a kind of recording ownership statements for each DeepCoin.
asset on DeepChain. In particular, for each newly generated After the genesis block is created, a random seed seed0
block DeepChain will generate certain amount of $Coin as is also publicly known, which is randomly chosen by regis-
rewards. Participants in DeepChain consist of parties and tered users through a routine for distributed random num-
workers, where the former gain $Coin for their contribu- ber generation. When DeepChain keeps running, at round
tions to local model training, and the latter are rewarded i, seedi−1 is used for generating seedi . It is worth men-
with $Coin for helping parties update training models. tioning that these random seeds are crucial for DeepChain,
Meanwhile, a well-trained model will cost $Coin for those because they guarantee randomness when selecting a leader
who have no capability to train the model by themselves to create a new block at each round. The idea of introducing
and want to use the model. This setting is reasonable random seeds is motivated by Algorand’s cryptographic
because recent work on model-based pricing for machine sortition [32], [33], and details will be given in Section 4.2.5.
learning has found applications in some scenarios [50], [36].
We define a validity value for $Coin, which essentially is 4.2.2 Incentive mechanism
the time interval of a round. Validity value is related to An incentive can act as a driving force for participants to
consensus mechanism in DeepChain, and we will discuss actively and honestly take part in a collaborative training
it in detail in Section 4.2.5. task, and the goal of incentive mechanism is to produce and
DeepChain combines together Blockchain techniques distribute value, so that a participant gets rewards or penal-
and cryptographic primitives to achieve secure, distributed, ties based on her contribution. The introduction of incentive
and privacy-preserving deep learning. Suppose there are mechanism is crucial for collaborative deep learning, due
N parties Pj , j ∈ {1, ..., N }, and they agree on some to the following reasons. First, for those parties who want
pre-defined information such as a concrete collaborative a deep learning model but have insufficient data to train
model and initial parameters of the collaborative model. The the model on their own, incentive can motivate them to
information is attached to a transaction T x0co signed by all join the collaborative training with their local data. Second,
parties. Assume the address corresponding to transaction with reward and penalty, incentive mechanism ensures that
T x0co is pkit0 , where it0 is the initial iteration. At the end of (1) parties are honest in local model training and gradient
iteration i, the updated model in T xico is attached to a new trading, and (2) workers are honest in processing parties’
address pkiti . All addresses are known to the parties. transactions.
Intermediate gradients from party Pj are enveloped in For ease of understanding the incentive mechanism,
transaction T xiPj , and all those transactions are collected by we give an example consisting of two parties. These two
a trading contract at round i. Note that intermediate gradi- parties contribute their data to collaborative training via
ents are local weights CPj (4Wi,j ), where C is a cipher used launching transactions. Suppose the data possessed by the
by party Pj to encrypt the weights. When all transactions two parties is not equal in quantity. Each party can launch
{T xiPj } at round i have been collected, trading contract transactions and pay transaction fee based on the amount
uploads them to DeepChain. After that, workers download of data she owned. Generally, the large amount of data a
those transactions {T xiPj } to process via processing contrac- party has, the less fee she will pay. The two parties agree
t. Specifically, workers update the weights by computing on the total amount of fees for collaboratively training the
C(Wi+1 ) = N1 · C(Wi ) · N model. The worker who successfully creates a new block
Q
j=1 CPj (4Wi,j ), where C(Wi ) is
the weight at round i in T xico , and C(Wi+1 ) is the updated when processing transactions can be the leader and earn
weights that will be attached to T xi+1 the rewards. Note that transaction issuing and processing
co for updating the local
models in next round i + 1. are verifiable, meaning that if some party poses an invalid
transaction, the party would be punished. On the other
hand, if a leader incorrectly processes a transaction, she
4.2 Components of DeepChain will be punished accordingly. When collaborative training
DeepChain consists of five building blocks that collectively finished, parties themselves can benefit from the trained
achieve distributed and privacy-preserving deep learning, model that can bring revenue for them through charged
namely, DeepChain bootstrapping, incentive mechanism, services to those users who want to use the trained model.

1545-5971 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TDSC.2019.2952332, IEEE
Transactions on Dependable and Secure Computing
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 6
V V
To give a formal description of the incentive mechanism, reward, i.e., (πP = 0) (πW = 0). Here, means ’and’ and
W
we first introduce two properties, i.e., compatibility and means ’or’. So we have
liveness of the incentive mechanism for participants. Then, Payoff=
we further explain that parties and workers have incentive V
to behave honestly. Assume that we guarantee data privacy M ax(πP )V+ M ax(πW ) If M ax(ωP )W M ax(ωW )
and security of the consensus protocol (explained in Section (πP = 0) (πW = 0) If (ωP = 0) (ωW = 0)
4.2.5). We use vc and vi to denote the value of the trained
collaboratively model and the trained individual model i, Next, we explain the assumption that participant’s con-
respectively, and we assume that vc is greater than vi . tribution originates from her correct behaviors with a high
probability. We show that each party or worker is value-
driven to behave correctly in each round so that she could
obtain the highest payoff [51]. If the probability that a
party’s behavior is correct is P rc (P ), then the corresponding
value is V alue(P rc (P )). Clearly, if the party’s behavior is
correct with probability P rc (P )=1, then she will obtain the
highest value, i.e., V alue(1). Similarly, a worker can get
value V alue(P rc (W )) if she behaves correctly with prob-
ability P rc (W ). Assume that a method verifies a party’s
malicious behavior to be correct with probability P rv (P ),
then the probability that a dishonest party is caught is
P rvc (P ) = P rv (P )∗(1−P rc (P )). Once the dishonest party
is caught, she is punished by forfeiting her deposit and the
Fig. 2. The incentive mechanism of DeepChain, where ωP and ωW loss is denoted as fP .
represent the contribution of a party and a worker for maintaining vc , Thus, the final value according to the party’s correct
respectively, and πP and πW represent their payoffs, respectively. behavior can be computed as

First, we say the incentive mechanism exhibits compati- V alue(P rc (P )) =πP ∗ (1 − P rvc (P )) − fP ∗ P rvc (P )
bility if each participant can obtain the best result according − ωP ∗ P rc (P )
to their contributions. Meanwhile, it has liveness only if
each party is willing to update her local training model where P rvc (P ) = P rv (P ) ∗ (1 − P rc (P )). The above value
with value vi by continuously launching transactions and reaches maximum only when the party behaves honestly,
each worker also has incentive to update the parameters i.e., P rc (P ) = 1. Therefore, V alue(1) = πP − ωP (1) holds.
of the collaborative training model with value vc . Below we This indicates the importance of the incentive mechanism.
describe the importance of these two properties with respect Specifically, the values of P rv (P ), πP , and fP can be deter-
to participant’s true contribution and the corresponding mined through the following theorems.
payoff. Let ωP and ωW be the contributions of a party and a
Theorem 1. If fP /πP > (1 − P rvc (P ))/P rvc (P ), where
worker to the final trained model, respectively, and πP and
P rvc (P ) = P rv (P ) ∗ (1 − θ), then a party is honest at
πW be their corresponding payoffs, respectively. At first, we least with probability θ.
assume that participant’s contribution originates from her
correct behaviors with a high probability, and later we will
Proof. We need to prove that for any P rc 0 (P ) < θ,
explain that this assumption is reasonable.
V alue(P rc 0 (P )) is smaller than V alue(θ). Without the
Liveness: both the party and the worker have the same loss of generality, we prove that for any P rc 0 (P ) < θ,
common interest to obtain a trained collaborative model. we have V alue(P rc 0 (P )) < 0. In other words, we have
Because if a party costs vi during the whole training process, V alue(P rc 0 (P )) = πP ∗ (1 − P rvc 0 (P )) − fP ∗ P rvc 0 (P ) −
then she would gain vc in the end, which is attractive for her ωP (P rc 0 (P )) < 0. When we set fP /πP > 1/P rvc 0 (P ) − 1,
because vc is greater than vi . On the other hand, a worker then we have πP ∗(1−P rvc 0 (P ))−fP ∗P rvc 0 (P ) < 0. Thus,
will process transactions for collaboratively constructing the V alue(P rc 0 (P )) < 0 holds.
training model in order to earn rewards with probability,
For a worker, analysis of the incentive mechanism is
with which she could pay for the deep learning services
similar to the above analysis for a party, expect that the
in DeepChain. Note that the probability a worker obtains
worker’s payoff is obtained with probability. We denote
reward depends on the quantity of rewards she has already
this probability by P rleader , then we could determine the
earned. The larger the quantity, the higher probability she
relationship between the four values P rleader , P rv (W ), πW ,
can get reward. As a result, both the party and the worker
and fW by the following theorem, so as to encourage a
are incented to build the collaborative training model.
worker to be honest.
Compatibility: the more a party contributes ωP , the
more she will gain πP . This holds for a worker too. During Theorem 2. If fW /πW ∗P rleader > (1−P rvc (W ))/P rvc (W ),
the collaborative training process, both party and worker are where P rvc (W ) = P rv (W ) ∗ (1 − ), then a worker will
incentivized to do theirVbest to contribute to building a train- be honest at least with probability .
ing model M ax(ωP ) M ax(ωW ), where the maximum
total payoff is M ax(πP ) + M ax(πW W ). If any participant did Proof. The proof is similar to the proof of Theorem 1, so
not perform well, i.e., (ωP = 0) (ωW = 0), then there is no we omit it.

TABLE 1 TABLE 2
Summary of notations Example of Threshold Configuration for #adversaries

Notation Description Number of parties Maximum number Threshold

psu
pkP a pseudo-generated public key of party P of adversaries
skP a secret key of the party P n=4 1 t ∈ {2, 3, 4}
q a randomly selected big prime n=5 1 t ∈ {2, 3, 4, 5}
G1 cyclic multiplicative cyclic groups of prime order q n=6 1 t ∈ {2, 3, 4, 5, 6}
g a generator of group G1 n=7 2 t ∈ {3, 4, 5, 6, 7}
Zq∗ {1,2, ..., q-1} n=8 2 t ∈ {3, 4, 5, 6, 7, 8}
H1 a collision-resistant hash function mapping
any string into an element in Zq∗
H2 a collision-resistant hash function mapping
any string into an element in G1
4.2.4 Collaborative training
C() a cipher generated by Paillier.Encrypt algorithm Based on stated assets, parties who have similar deep
Enc() the encryption by individual parties
learning task can constitute a collaborative group, and the
modelco collaborative deep learning model
(collaborative model for short) to train collaborative training process consists of the following four
ΣP K correctness proof for a ciphertex that steps.
is indeed encrypted by a party’s public key Collaborative group establishment. According to simi-
ΣCD correctness proof for a decryption share
lar ”Keywords”, parties can establish a collaborative group.
It is worth noting that parties may get more detailed in-
formation about ”Keywords” through off-line interactions
4.2.3 Asset statement and this is not the focus of our paper. Before forming a
For ease of presentation, we list related cryptographic no- collaborative group, parties can audit cooperator’s asset to
tations used in this section in Table 1. A party needs to ensure authenticity of the asset ownership. The auditing
state her asset, which enables her to find cooperators and process can be done by using the method in [52], and we
accomplish her deep learning task. Asset statement does omit the details for brevity.
not reveal the content of asset, since it is simply some Suppose there are N (N > 3) parties P1 , P2 , ..., PN
description of the asset, e.g., what kind of deep learning that constitute a group with pseudonymity, i.e., pseudo
psu psu psu
tasks the asset can be used for. Specifically, party P states public keys pkP1 , pkP2 , ..., pkPN and their corresponding
an asset by sending an asset transaction, which will be secret keys skP1 , skP2 , ..., skPN are privately kept, respec-
introduced later. tively. Since different party launches transactions using her
psu
We recall the formation of a transaction. Note that a own pseudo public key pkPi , transactions signed by the
transaction is launched by a pseudo public key address corresponding secret key skPi can be verified to ensure that
pkPpsu generated by P according to her wish in the following those transactions are from the same cooperative party Pi .
form. Collaborative information commitment. After the col-
laborative group is formed, parties agree on the information
pkPpsu ∈ {g1skP , g2skP , ..., gnskP } for securely training a deep learning model. In this step, we
assume that a trusted component (e.g., a trusted hardware
Here, n is an integer. P selects a secret key skP ∈ Zq∗ and like Intel SGX [53]) only takes part in the setup phase in
generates n public keys giskP ∈ G1 , i ∈ [1, n]. q and g are Threshold Paillier algorithm [54], and it is not involved in
pre-specified parameters, and gi equals to g ri , where ri is any other process. If there does not exist such a trusted
a random element in Zq∗ . Suppose that party P1 sends a
psu component, we can accomplish the setup phase by using
transaction with her address pkP1 to state her asset data P1
a distributed method such as the one in [55]. Parties agree
as follows
on the following information.
(1) Number of cooperative parties, N .
n
T ranP1 = pkPpsu
1
→ pkdata P1 = g H1 (data P1 )
,
(2) Index of the current round, r.
σj = (H2 (j) · g H1 (dataj P1 ) )H1 (data
P1
P1 )
, (3) Parameters of Threshold Paillier algorithm.
o We have the following equation
”Keywords” .
P Kmodel = (nmodel , gmodel , θ = as, V = (v, {vi }i∈[1,...,N ] ))
In this transaction, the first part in the braces consists of
pkdata P1 and σj P1 , which is the statement proof that party where modulus nmodel is the product of two selected safe
P1 indeed possesses asset H1 (data P1 ) without leaking the primes, and gmodel ∈ Zn∗2 , a, s, θ, v, vi ∈ Zn∗model . And
model
content of data P1 . In particular, σj P1 contains l compo- SKmodel = s is randomly divided into N parts, where
nents, where data P1 is divided into l blocks represented s = f (s1 + ... + sN ) and f is a function of secret sharing
by dataj P1 , j ∈ [1, l]. The second part ”Keywords” is the protocol (i.e., Shamir’s secret sharing protocol [56]). Each
description of the asset data P1 . In our implementation, party owns a proportion of secure key si as well as v and
”Keywords” is in JSON form that includes four fields, i.e., {vi }, i ∈ [1, ..., N ] are public verification information, where
data size, data format, data topic and data description. With vi corresponds to si . A threshold t ∈ { N3 + 1, ..., N } is
this transaction T ranP1 , P1 can fulfill her asset statement. set as such that at least t parties together can decrypt a
We assume that the first stated asset is authentic, which is cipher. Specifically, we give the threshold configuration with
reasonable in Blockchain. respect to the number of adversaries in Table 2.

Note that training gradients to be encrypted are vec- First of all, parties iteratively trade their gradients
1 l
tors with multiple elements, i.e., 4Wi,j = (wi,j , ..., wi,j ) through Trading Contracts that are executed by a manager
where the length of 4Wi,j is l, i is the index of current selected from cooperative parties. The trading gradients
training iteration, and j ∈ {1, ..., N }. Due to the problem are honestly encrypted by each trader and meanwhile the
of cipher expansion, we encrypt a vector into one cipher correct proofs of encryption are attached that indicate two
instead of multiple ciphers with respect to multiple ele- security requirements, i.e., confidentiality and auditability.
1 l
ments. Suppose that each value wi,j , ..., wi,j is no larger Herein, we say gradient transactions are generated. In terms
than integer d, d > 0. We choose a l-length super increas- of confidentiality, if a trader does not disclose her gradients,
ing sequence α ~ = (αP 1 = 1, ..., αl ) that simultaneously then no one can gain information about the gradients. In
i−1
meets conditions (1)
Pl l=1 αl · N · d < αi , i = 2, ..., l, addition, traders (at most t parties) need to cooperatively
and (2) i=1 αi · N · d < nmodel . We then compute
decrypt the updated parameters. Similar to [39], we assume
1 l α1 αl that the manager does not disclose what she knows. In
(gmodel , ..., gmodel ) = (gmodel , ..., gmodel ).
(4) A collaborative model modelco to be trained. terms of auditability, there exist proofs of correct encryption
For a collaborative model modelco , parties agree on which can be auditable. When cooperatively decrypting,
the training neural network, the training algorithms, and each trader presents her own decryption proof. Those proofs
configurations of the network such as number of network are generated non-interactively and publicly auditable by
layers, number of neurons per layer, size of mini-batch any party on DeepChain.
and number of iterations. Beside those information, they Through timeout-checking and monetary penalty mech-
also reach a consensus on initial weights W0 of modelco . anism, behaviors of the traders and the manager are forced
Note that weights Wi would be updated to Wi+1 after to be authentic and fair. Even if the manager colludes with
the i-th iteration of training. They protect W0 by applying traders, the outcome of Trading Contract cannot be modified
W0 [39]. In addition to Trading Contract, Processing Contract is re-
Paillier.Encrypt algorithm, i.e., C(W0 ) = gmodel ·(k0 )nmodel ,
∗
where k0 is randomly selected from Znmodel . Note that we sponsible for parameter updating. Workers process transac-
W0
compute gmodel with the help of the chosen super increasing tions by adding up gradients, and send computation results
W0 1 l α ·w1 +...+α ·wl to Processing Contract. Processing Contract verifies correct
sequence, i.e., gmodel = gmodel
0 0
, so that we generate
computation results and updates model parameters for the
a cipher for weight vector W0 = (w01 , ..., w0l ).
group. Note that once smart contract is defined, it can be
(5) A commitment on SKmodel = s, with respect to automatically executed in response to some trigger event. In
P Kmodel . this setting, ’computation results sent to Processing Contract’
Commitment commitSKmodel is obtained by combining
parties’ commitments on their secret shares si . Recall that r is the trigger event, and Processing Contract has a pre-defined
is the index number of the current round. We have function to verify those computation results by the rule of
majority voting. These two contracts are iteratively invoked,
commitSKmodel = (Enc(s1 ||r||Sign(s1 ||r)), so as to accomplish the whole training process. Details of the
..., Enc(sN ||r||Sign(sN ||r))) two steps are given below.
Gradient collecting via Trading Contract. As shown in
here, || denotes concatenation. Algorithm 1, Trading Contract invokes six functions, i.e., line
(6) The initial weights W0,j of local model of party j . 1, 4, 7, 10, 13 and 16 of Algorithm 1, for training modelco .
Each party provides her local model’s initial weights that At the end of each of the functions, we declare a time
are encrypted by Paillier.Encrypt algorithm, i.e., C(W0,j ) = point Tti to check time-out events, and these six time points
W0,j
gmodel · (kj )nmodel , where kj ∈ Zn∗model , j ∈ {1, ..., N }. satisfy Tti < Tti+1 , i = 1, 2, ..., 5. We set up the time points
(7) An amount of deposits d($Coin). according to Greenwich Mean Time. The time interval be-
Each cooperative party is required to commit some tween Tt1 and Tt6 can be determined according to the time
amount of deposits for secure computation. During col- interval between two consecutive training iterations, e.g.,
laborative training, if a party misbehaves on purpose, her for iteration i and i + 1, we have |Tt6 − Tt1 | ≤ |Ti+1 − Ti |.
deposit d($Coin) would be forfeited and compensated for By the end of a time point Tti , function checkT imeout
other honest parties. Otherwise, those deposits would be checks whether the parties finish the events or not by Tti .
refunded after the training process finished. If some party is caught, the monetary penalty mechanism
All the above collaborative information are recorded will be performed to forfeit deposit of the party, and the
in a transaction T ranco that is uploaded to DeepChain. failed step is re-executed. During collaborative training, the
Specifically, T ranco is in the following form and is attached six time points are updated accordingly with iterations, e.g.,
psu
to a commonly coordinated address pkco .
Tt01 = Tt1 + |Ti+1 − Ti |.
Algorithm 1 works as follows. As shown in line 1, at
n
psu
T ranco = pkco → N, r, P Kmodel , d, α
~ , modelco ,
o the i-th iteration each party Pj , j ∈ {1, ..., N } sends a
commitSKmodel , C(W0,j ), d($Coin) . gradient transaction T raniPj to receiveGradientT X(). A
publicly auditable proof P roofP K i,j is also attached to the
In addition, two roles called trader and manager are defined transaction to guarantee encryption correctness. We have
for parties in a collaborative group, which will be explained
shortly. Next we introduce how collaborative training is T raniPj = {pkPpsu
j
psu
: (C(4Wi,j ), P roofP K i,j ) → pkco }
securely accomplished through the remaining two steps, P roofP K i,j = f sprove1 (ΣP K ; C(4Wi,j ); 4Wi,j , kj ; pkPpsu
j
)
namely, Gradient collecting via Trading Contract and Parameter
updating via Processing Contract. Then in line 4, function verif yGradientT X() veri-

Algorithm 1: Trading(T raniP1 ,...,T raniPN ) Algorithm 2: Processing()

1 receiveGradientTX() 1 updateTX()
2 checkTimeout(Tt1 ) 2 checkTimeout(Tt7 )
0
3 updateTime() // Tt01 = Tt1 + |Ti+1 − Ti | 3 updateTime() // Tt7 = Tt7 + Tr
4 verifyGradientTX() 4 verifyTX()
5 checkTimeout(Tt2 ) 5 checkTimeout(Tt8 )
0
6 updateTime() 0
// Tt2 = Tt2 + |Ti+1 − Ti | 6 updateTime() // Tt8 = Tt8 + Tr
7
psu
uploadGradientTX()#attaching to the address pkco 7 appendTX()
8 checkTimeout(Tt3 ) 8 checkTimeout(Tt9 )
0
9 updateTime() 0
// Tt3 = Tt3 + |Ti+1 − Ti | 9 updateTime() // Tt9 = Tt9 + Tr
psu
10 downloadUpdatedParam()#from the address pkco
11 checkTimeout(Tt4 )
0
12 updateTime() // Tt4 = Tt4 + |Ti+1 − Ti | gradients C(4Wi,j ), j ∈ {1, ..., N }, have been uploaded,
13 decryptUpdatedParam() then workers competitively execute update operations by
14 checkTimeout(Tt5 ) 1
15 updateTime() 0
// Tt5 = Tt5 + |Ti+1 − Ti | C(Wi ) = C(Wi−1 ) · ·(C(− 4 Wi,1 ) · C(− 4 Wi,2 )·
N
16 return() ... · C(− 4 Wi,N )).
17 checkTimeout(Tt6 )
18 updateTime() 0
// Tt6 = Tt6 + |Ti+1 − Ti | Once update operation finished, workers then send the up-
dated results through transactions to function updateT X()
in Processing Contract, as shown in line 1.
At the meantime, a leader is randomly chosen from the
fies correctness of the encrypted gradients via function workers by using the consensus protocol of DeepChain (we
f sver1 (ΣP K ; C(4Wi,j ); P roofP K i,j ; pkPpsu
j
). Specifically, it will discuss it in Section 4.2.5). Note that at this moment
verifies whether C(4Wi,j ) is indeed the encryption of we defer the reward to the leader until her computational
4Wi,j with random number kj . Here, pkPpsu j
can be re- work is verified by using function verif yT X() as shown
garded as the identity information attached to the proof, in line 4, that employs majority voting policy. In other
avoiding replay attack by a malicious party. In line 7, function words, the leader’s computational result C(Wi ) will be
uploadGradientT X() uploads the transactions that have compared against those of the other workers, and her result
been verified successfully. When model parameter update is admitted only if the majority of the workers produce
finished, downloadU pdatedP aram() retrieves the latest pa- the same result. Otherwise, the leader would be punished
rameters, as can be seen in line 10. Recall that Processing Con- according the monetary penalty mechanism and she gains
PN
tract computes gradients j=1 4Wi,j for model modelco . no reward. In such case, we repeat the procedure to chosen
Suppose that the latest iteration is i, the cipher of the lat- a new leader from the remaining workers. The more often a
est parameters is C(Wi ) and we denote it as Ci for brevity. worker is punished, the lower probability she will be chosen
Then decryptU pdatedP aram() collects parties’ decryption as a leader. Once we get a legitimate leader, her block with
shares on Ci for collaborative decryption, which generates correctly updated result is appended to DeepChain through
Ci,j , j ∈ 1, ..., N . Meanwhile, the corresponding proofs for appendT X(), as shown in line 7.
correct shares P roofCDi,j are also provided, as follows.

2∆s
Ci,j = Ci j
P roofCDi,j = f sprove2 (ΣCD ; (Ci , Ci,j , v, vj ); ∆sj ; pkPpsu
j
)

The proof P roofCDi,j is used to verify validity of the de-

2
cryption shares, i.e., ∆sj = logCi4 (Ci,j ) = logv (vj ), through
psu
function f sver2 (ΣCD ; (Ci , Ci,j , v, vj ); P roofCDi,j ; pkPj ).
If majority of the parties are honest, then Ci can be correctly
recovered to plaintext by
Fig. 3. Configuration of time points in Processing Contract. From top
2µ to bottom: (1) the timeline of collaborative training, (2) the timeline of
((Πj∈H Ci,j j − 1)/nmodel )(4∆2 θ)−1 mod nmodel trading (in Trading Contract), (3) the timeline of block creation. Here, the
vertical orange bar refers to the interval between verified gradient trans-
action being uploaded and updated parameters to be downloaded. The
where µj is the Lagrange interpolation coefficient with vertical green bar refers to the interval between worker’s transactions
respect to Pj , and the plaintext is pushed to parties by being sent and final updated result being uploaded.
function return() in line 16.
• Parameter updating via Processing Contract. Algo- In Processing Contract, time points Tt7 , Tt8 and Tt9 will
0 0
rithm 2 summarizes the process of Processing Contract, which be updated to Tt7 = Tt7 + Tr , Tt8 = Tt8 + Tr and
0
contains three functions, as shown in line 1, 4, and 7. Sup- Tt9 = Tt9 + Tr , respectively, where Tr is the time need-
pose that at the i-th iteration of collaborative training, local ed to create a new block between consecutive rounds in

DeepChain. Figure 3 gives an example of time point con- set of the remaining honest parties, where |H 0 | = h0 . In
psu
figuration scheme to illustrate relationship of time points line 2, the contract verifies the ciphertext for all pkPj ∈
of the trading and processing contracts. Suppose that at H 0 , and records the correct parties {1, ..., N } \ C0 , where C0
the i-th iteration, the time points are set as such that refers to corrupted parties in this step. In line 3, the contract
psu
Tt1 < Tt2 < Tt3 ≤ Tt7 < Tt8 < Tt9 ≤ Tt4 < Tt5 < Tt6 . sends return messages to pkPj for j ∈ {1, ..., N } \ C0 . In
At the meantime, the relationship between the three time line 4, we wait for a return message from S . If the returned
intervals is Tr ≤ |Tt6 − Tt1 | ≤ |Ti+1 − Ti |. message is continue, then the contract outputs normally to
psu
all pkPj (j ∈ {1, ..., N }), by sending payback message to
∗ 00 00
Algorithm 3: FGradientCollecting S and extrapay to pkPpsu in H , where H = H 0 \ C0 and
j
00 00
1 Receive (input, |H | = h ; otherwise, the contract sends penalty to pkPpsu j
,
sid, Tt , pkPpsu
j
, C(4W), P roofP Kj , d($Coin)) from j ∈ {1, ..., N }.
pkPpsu
j∈{1,...,N }
. Assert time Tt < Tt1 . Receive (input, Similarly, fairness is also achieved in Collaborative decryp-
sid, Tt , pkPpsu
j∈C
, C(4W), P roofP Kj , H 0 , tion (Algorithm 4), since (1) a party who gives a correct
0
h × d($Coin)) from S . Assert time Tt < Tt1 . decryption share no later than the pre-defined time point
2 Compute f sver1 (C(4W), P roofP Kj ) for receives no penalty, and (2) If an adversary successfully
pkPpsu
j∈{1,...,N }
, and record {1, ..., N } \ C0 . decrypts the cipher but a legitimate party fails to do so,
3
psu
Send(return, d($Coin)) to pkPj∈{1,...,N }\C0 after Tt1 . then the party should be compensated for.
4 If S returns (continue, H 00 ), then send (output, Y es or 4.2.5 Consensus protocol
N o) to pkPpsuj∈{1,...,N }
, and send (payback, Consensus protocol is essential in DeepChain, since it en-
(h − h00 )d($Coin)) to S , and send (extrapay, d($Coin)) ables all participants to make a consensus upon some event
psu
to pkPj∈H 00 , else if S returns (abort), send (penalty, in a decentralized environment. In this section, we introduce
d($Coin)) to pkPpsu j∈{1,...,N }
. blockwise-BA protocol of DeepChain, based on the work
of Algorand [32], [33]. The blockwise-BA protocol includes
three main steps — (1) A leader who creates a new block
∗ is randomly selected by using cryptographic sortition, (2) A
Algorithm 4: FCollaborativeDecryption
committee, consisting of participants whose transactions are
1 Receive (input, included in the new block, verifies and agrees on the new
sid, Tt , pkPpsu
j
, C, Cj , P roofCDj , d($Coin)) from block by executing a Byzantine agreement protocol [31], and
pkPpsu
j∈{1,...,N }
. Assert time Tt < Tt5 . Receive (input, (3) Each verifier in the committee tells neighbors the new
sid, Tt , pkPpsu
j
∈ C, C, Cj , P roofCDj , H 0 , block by using a gossip protocol [59], [60], so that the new
0
h ∗ d($Coin)) from S . Assert time Tt < Tt5 . block is known to all participants in DeepChain.
2
psu
Compute f sver2 (C, Cj , P roofCDj ) for pkPj∈{1,...,N } Our consensus protocol possesses three properties, i.e.,
and record {1, ..., N } \ C0 . safety, correctness, and liveness. In particular, safety means
3
psu
Send(return, d($Coin)) to pkPj∈{1,...,N }\C0 after Tt5 ; that all honest parties agree on a same transaction his-
tory in DeepChain, whereas correctness requires that any
4 If S returns (continue, H 00 ), then send (output, Y es or
transaction agreed by a honest party comes from a honest
N o) to pkPpsuj∈{1,...,N }
, and send (payback,
party. Liveness says that parties and workers are willing
(h − h00 )d($Coin)) to S , and send (extrapay, d($Coin)) to continuously perform activities in DeepChain, hence
psu
to pkPj∈H 00 , else if S returns (abort), send (penalty, keeping DeepChain alive. Based on these three properties,
d($Coin)) to pkPpsu j∈{1,...,N }
. we assume that message transmission is synchronous and
there are no more than 13 malicious parties. In this setting,
In addition to the above configuration scheme for time all parties agree on a chain with the largest amount of assets.
points, we employ secure monetary penalty mechanism to We give details of the three steps of our consensus protocol
guarantee fairness in gradient collecting and collaborative below. Suppose block blocki is created at round ri .
Leader selection. At round ri , a leader leaderi is ran-
decryption. Specifically, enlightened by the penalty mecha- domly chosen from workers who collect transactions and
nism proposed by Bentov et al [57] and Kumaresan et. al [58], put them into block blocki . To choose a leader, we invoke
we design our secure monetary penalty mechanism based the sortition function of Algorand [32], which includes two
on Trading Contract, presented in Algorithm 3 and 4. functions leader selection and leader verification, as follows.
In particular, in Gradient collecting (Algorithm 3) fairness Sortition(sk, seedi , τ = 1, role = worker,
is guaranteed due to (1) honest collaborative parties must
w, wtotal ) → hhash, π, ji
launch gradient transactions to be correctly verified before
V erif ySort(pk, hash, π, seedi , τ, role = worker,
the pre-specified time point, and (2) dishonest parties who
launch incorrect transactions or delayed transactions will w, wtotal ) → j
be penalized, and the honest ones will be compensated for. Here, sk and pk are owned by worker, and seedi is
In line 1, Trading Contract waits to receive a input message a random seed selected based on seedi−1 , i.e., seedi =
psu
from pkPj for all j = 1, ..., N before time Tt1 . By defining H(seedi−1 ||ri ), where H is an hash function. τ = 1 mean-
C ⊆ {1, ..., N } as adversarial parties S in the input step, s that only one leader is selected from workers role =
the contract also waits an input message from S . Here, sid worker. w represents the amount of $Coins that the par-
is session identifier, d($Coin) is deposit, and H 0 means the ticipant possesses. Parameter wtotal is the total amount of

$Coins in DeepChain. It is worth mentioning that w is C ← S(1λ , wb ) :

1
crucial, because it is used to control the probability that A(C, 1λ , F t (·)) = b ≤ negl(1λ ) + 2
worker can gain reward according to the amount of rewards
she has already earned (see Section 4.2.2). which indicates that the probability for adversaries to dis-
Our definition of w is different from that of Alogrand, tinguish w0 or w1 is negligible, in system security parameter
in that in our Leader Selection w only contains $Coins that λ. Here, F t (·) means that A has at most t corrupted parties
have available validity value, while those without validity and A learns their information including public parameters,
value are not considered. In this way, we can eliminate secret shares of the corrupted parties, public verification
the phenomenon of wealth accumulation, in which a rich keys, all decryption shares and validity of those shares. In
participant may become richer because she has a high- addition, t-robust means that a Threshold Paillier cipher-
er probability than her peers to be chosen as the leader. text can be correctly decrypted, even in the case that A
Through the two functions, we can randomly select a leader can have up to t corrupted parties. Semantic security is a
and all participants can also verify whether the selected general security proof methodology to measure the security
leader leaderi is legitimate. of an encryption algorithm and in our context it measures
Committee agreement. After leader verification, the se- confidentiality of the encrypted information by using the
lected block blocki is sent to the committee. Each participant Threshold Paillier algorithm.
in the committee verifies the transactions processed by
leaderi , i.e., to verify whether weight update operations 5.2 Auditability of Gradient Collecting and Parameter
are correct or not. If the committee admits that blocki is Update.
right based on a majority voting policy, then participants
sign blocki on behalf of the committee; otherwise, blocki is Auditability ensures that any third party can audit cor-
rejected. Note that blocki is valid only if more than 23 of rectness of encrypted gradients and decryption shares in
the committee members signed and agreed on it. If blocki gradient collecting stage and parameter updating stage,
is valid, then leaderi gains $Coins from block reward and receptively. We achieve auditability by following univer-
transaction coins of blocki ; otherwise, blocki is discarded sally verifiable CDN (UVCDN) protocol [62]. Specifically,
and a new empty block is created to replace blocki in correctness
P proof provided in UVCDN protocol is based
DeepChain. This process repeats until the committee agrees on -protocols, where a possible malicious prover proves
on blocki . correctness to a honest verifier that he indeed knows a
Neighbor gossip. Suppose blocki has been agreed on witness w for a certain statement v leading to (v; w) ∈ R
by the committee, then participants in the committee are , here R is a binary relation. Generally, a correctness proof
responsible for telling their neighbors blocki , by using the consists
P of three protocols, namely, P announcement (denoted
popular gossip protocol [59], [60]. Therefore, after this step as .ann), response
P (denoted as .res) and verification
all participants arrive at a consensus in DeepChain. (denoted as .ver), which are defined according to the
purpose of correctness proof (i.e., R). We will give the
correctnessPproof P by using twoP procedures f sprove and
5 S ECURITY A NALYSIS f sver for .ann, .res and .ver.
In this section, we revisit our security goals of DeepChain
presented in section 3 and give security analysis for them. Algorithm 5: f sprove1 (ΣP K ; C(4Wi,j ); 4Wi,j , kj ;
pkPpsu
j
)
5.1 Confidentiality Guarantee for Gradients #announcement
1 ΣP K .ann(C(4Wi,j ); 4Wi,j , kj ) :=
To achieve this goal, DeepChain employs Threshold Paillier a1
a1 ∈R Znmodel , b1 ∈R Zn∗model , a = gmodel bn1 model ;
algorithm that provide additive homomorphic property. (a; s) = (a; a1 , b1 )
We assume there exists a trusted setup (refer to Section #challenge
4.2.4) and the secret key cannot leak without collaboration 2 c = H(C(4Wi,j )||a||pkPj )
psu
of at least t participants. We also assume that at least t #response
participants are honest. Without loss of generality, both local 3 ΣP K .res(C(4Wi,j ); 4Wi,j , kj ; a; s; c) :=
gradients and model parameters W are encrypted with (a +c4W )
W t = 1 nmodel i,j , d = a1 + c 4 Wi,j ,
the Threshold Paillier algorithm, C(W) = gmodel (k)nmodel .
Based on the following lemma that is derived from the work
t
e = b1 kjc gmodel ; r = (d, e)
[54]’s Theorem 1, we can guarantee confidentiality of local 4 return P roofP Ki,j := (a; c; r)
gradients and model parameters.
Lemma 1. With the Decisional Composite Residuosity In Section 4.2.4, to prove the correctness of en-
Assumption (DCRA) [61] and the random oracle model crypted gradients R = {(C(4Wi,j ); 4Wi,j , kj )} where
4W
S , Threshold Paillier algorithm is t-robust semantically C(4Wi,j ) = gmodeli,j (kj )nmodel , a publicly auditable proof
secure against active non-adaptive adversaries A with P roofP Ki,j is generated by procedure f sprove1 . Then, any
polynomial time power to attack, if the following are party can execute procedure f sver1 , by taking P roofP Ki,j
satisfied as input, to verify whether C(4Wi,j ) is indeed the en-
cryption of 4Wi,j with random number kj under public
Pr (w0 , w1 ) ← A(1λ , F t (·));

key P Kj (i ∈ {1, ..., #iteration}, j ∈ {1, ..., NP
}). The con-
b ← {0, 1}; crete procedures of f sprove1 and f sver1 for -protocols

x
Algorithm 6: f sver1 (ΣP K ; C(4Wi,j ); P roofP K i,j ; pkPj )
psu Lemma 3. Let X = C(x) = gmodel rnmodel , where x = 4W
and r = k . Given (a; s) that is generated by the
#verification announcement ΣP K .ann and two different challenges
#P roofP Ki,j := (a; c; r), r = (d, e) c, c0 with respect to the announcement, there exists an
1 ΣP K .ver(C(4Wi,j ); a; c; r) := V extractor E that can extract the witness of an adversary
(c == H(C(4Wi,j )||a||pkPpsu j
d
)) (gmodel enmodel == A, if A can present two conversations (d, e) and (d0 , e0 )
c
a(C(4Wi,j )) ) for (a; s), that is,
2 return Yes or No
|1 − Pr[A(X; x, r; a; s; c; c0 ) →
(d, e; d0 , e0 ); E(X; a; c; c0 d, e; d0 , e0 ) → (x0 , r0 ) =
are given in PAlgorithm 5 and Algorithm
P 6, respectively. (x, r)]| ≤ negl(1λ )
In addition, PK refers to the -protocols achieved Pby
P
f sprove1 and f sver1 . Correspondingly, CD is for the - Lemma 2 and Lemma 3 refer to the property of zero-
protocols realized by f sprove2 (Algorithm 7) and f sver2 knowledgeness and soundness in UVCDN protocol’s Defi-
(Algorithm 8) for proving decryption correctness. Note that nition 1, respectively. The concrete proofs can be found in
in Algorithm 6 and Algorithm 8, the sign ‘==’ is used to Section 3.2 of [63]. As described in Lemma 2, a simula-
judge whether two values between ‘==’ are equal. tor without any knowledge of witness of an honest party
can provide a proof of encryption correctness, which has
psu statistically indistinguishable distribution compared with
Algorithm 7: f sprove2 (ΣCD ; Ci , Ci,j , v, vj ; 4sj ; pkPj ) a real one. Lemma 3 means that the probability that the
#announcement extractor E fails to extract the witness (x, r) of an adversary
1 ΣCD .ann(Ci , Ci,j , v, vj ; 4sj ) := P respect to system security parameter λ.
is negligible, with
u ∈R [0, 22k+2k2 ], a = Ci4u , b = v u In terms of CD , the corresponding properties of zero-
#k = log2 nmodel , k2 is the security param. knowledge and soundness for public auditability of correct-
#challenge ness decryption are described in Lemma 4 and Lemma 5,
psu
2 c = H(Ci ||Ci,j ||v||vj ||a||b||pkPj ) respectively. Also, the concrete proofs can refer to Section 4
#response of reference [64].
3 ΣP K .res(Ci , Ci,j , v, vj ; 4sj ; a, b; u, c) := Lemma 4. Given (Ci , Ci,j , v, vj ), and c ∈ C where C is a
r = u + c 4 sj finite set called the challenge space, then we have
4 return P roofCDi,j := (a, b; c; r) {r ∈R [0, 22k+2k2 ]; a = Ci4r (Ci,j )−2c , b = v r (vj )−c :
(a, b; c; r)}
≈
{u ∈R [0, 22k+2k2 ], a = Ci4u , b = v u ; r = u + c 4 sj :
Algorithm 8: f sver2 (ΣCD ; (Ci , Ci,j , v, vj ); P roofCDi,j ; (a, b; c; r)}
pkPpsu
j
)
#verification where symbol ≈ means that the two distributions
#P roofCDi,j := (a, b; c; r) are statistically indistinguishable.
1 ΣCD .ver(Ci , Ci,j , v, vVj ; a, b; c; r) :=
The above formula means that there exists a simulator
(Ci4r == a(Ci,j )2c ) (v r == b(vj )c )
without knowledge of 4sj can provide a proof that has a
2 return Yes or No
statistically indistinguishable distribution compared with a
P real one.
Under the framework of UVCDN protocol, P K guar- Lemma 5. Given (Ci , Ci,j , v, vj ) and (a, b; u) is generated
antees public auditability if there exist a simulator that can by the announcement ΣCD .ann. Malicious prover
simulate correctness proofs of honest parties, and an extrac- provides two different challenges c, c0 with respect to
tor that can extract witnesses of corrupted parties which the announcement. There exists an extractor E that can
are illustrated
P by Lemma 2 and Lemma 3, respectively. extract the witness of an adversary A, if A can present
Similarly, CD also guarantees public auditability shown two conversations r and r0 for (a, b; u), that is,
by Lemma 4 and Lemma 5.
x
Lemma 2. Given X = C(x) = gmodel rnmodel , x = 4W, |1 − Pr[A(Ci , Ci,j , v, vj ; 4sj ; a, b; u; c; c0 ) →
r = kj , and c ∈ C where C is a finite set called the (r; r0 ); E(Ci , Ci,j , v, vj ; a, b; c; c0 ; r; r0 ) → 4sj ]| ≤
challenge space, then we have negl(1λ )
{d ∈R Znmodel ; e ∈R Zn∗model ; a := gmodel
d
enmodel X −c :
(a; c; d, e)} Lemma 5 shows that extractor E can extract the witness
≈ 4sj of A with overwhelming probability, with respect to
a1
{a1 ∈R Znmodel ; b1 ∈R Zn∗model ; a := gmodel bn1 model ; t := system security parameter λ.
(a1 + cx)/nmodel; d := a1 + cx; e := b1 kjc gmodel t
:
(a; c; d, e)} 5.3 Fairness Guarantee for Collaborative Training.
Recall that we employ two security mechanisms in
where symbol ≈ means that the two distributions
Blockchain, namely, the trusted time clock mechanism and
are statistically indistinguishable.
secure monetary penalty mechanism, to enhance fairness

during collaborative training, by following the work [57]. collaborative decryption, as shown in Algorithm 3 and 4,
With the trusted time clock mechanism, operations in a respectively.
contract are forced to finish before the respective time point,
as shown in function checkTimeout() in Algorithm 1 and 6 I MPLEMENTATION AND E VALUATION
2. On the other hand, we also define two secure monetary In this section, we implement our DeepChain proto-
penalty functions for gradient collecting and collaborative type. First, we build a Blockchain to simulate DeepChain.
decryption, respectively. Blockchain nodes are regarded as parties and workers, and
In order to prove the property of fairness, Bentov et. al they participate in trading and interact with two pre-defined
[57] introduced the definition of secure computation with smart contracts, i.e., Trading Contract and Processing Contract.
coins (SCC security) in the multi-party setting in a hybrid Generated transactions are serialized in the Blockchain.
model that not only involves standard secure computation We use Corda V3.0 [67] to simulate DeepChain for its
[65], but also special secure computation dealing with coins. adaptability and simplification. Specifically, Corda project is
Here, the goal of security refers to fairness presented in their created by R3CEV and has been widely used in banks and
paper. Also, they considered universally composable (UC) financial institutes. It is a decentralized ledger that has some
security proof for SCC security. In particular, compared to features of Bitcoin and Ethereum [68], such as data sharing
the initial definition of UC security, the view of environment based on need-to-know basis and deconflicting transactions
in SCC security additionally indicates the distribution of with pluggable notaries. A Corda network contains mul-
coins because of the added functionality of monetary penal- tiple notaries, and our consensus protocol introduced in
ty. section 4.2.5 can be executed on them. We build nodes and
In DeepChain setting, based on the tutorial in Bentov et. divide them into parties and workers. Specifically, we set
al’s work, the property of fairness for gradient collecting up two CorDapps which agree on Blockchain. The nodes
and collaborative decryption is claimed in Section 4.2.4. of one CorDapp serve as parties, and the nodes of the
Our work only replaces the general computation with the other CorDapp play the role of workers. According to the
special computation to realize functionalities of gradient application program interface (API) of Corda, we implement
collecting and collaborative decryption. Other components our business logic by integrating three main components,
based on Blockchain, including trusted time clock and mon- namely, State, Contract, and Flow. In particular, an instance
etary penalty exchange, remain unchanged. Thus, the UC- of State is used to represent a fact of a kind of data, and
style SCC security defined in Bentov et. al’s work can be it is immutable once an instance of State is known by all
guaranteed for the specialized functionalities in DeepChain nodes at a specific time point. Contract is used to instantiate
setting, only if SCC security has been proved according to some rules on transactions. A transaction is considered to
UC composition theorem (refer it to Section 5 of reference be contractually valid if it follows every rule of the contract.
[66]). This is demonstrated by Lemma 6, where the environ- An instance of Flow defines a sequence of steps for ledger
ment Z becomes a distinguisher, by following the UC-style updates, e.g., how to launch a transaction from a node to
proof. If Z with non-uniform probabilistic polynomial-time another node.
computation could not distinguish the distribution in the We build the deep learning environment with Python
ideal model from that of the hybrid model, then a protocol (version 3.6.4), Numpy (version 1.14.0), and Tensorflow
π SCC realizes a functionality f . (version 1.7.0). We select the popular MNIST dataset [69]
Lemma 6. Given an input z , security parameter λ, a which contains 55, 000 training samples, 5, 000 verification
distinguisher Z , an ideal process IDEAL, an ideal samples and 10, 000 test samples. Then, we split randomly
adversary S in IDEAL, an ideal function f , and a this dataset into 10 equi-sized subsets, i.e., each contains
protocol π that interacts with ideal function g in a model 55, 000/10 = 5, 500 samples. Then, we conduct multiple
with adversary A, then we have training experiments with 4, 5, 6, 7, 8, 9, and 10 parties,
{IDEALf,S,Z (λ, z)}λ ∈ N, z ∈ 0, 1∗ denoted as E-4, E-5, E-6, E-7, E-8, E-9, and E-10, respectively.
≡c In each experiment, each party possesses one subset of
{HYBRIDg,π,A,Z (λ, z)}λ ∈ N, z ∈ 0, 1∗ the dataset. Our training model derives from Convolution
Neural Network (CNN) with structure: Input → Conv →
where ≡c means that the distributions are computation- Maxpool → Fully Connected → Output. The weights
ally indistinguishable. and bias parameters in Conv layer, Fully Connected layer
and Output layer are w1 = (10, 1, 3, 3) and b1 = (10, 1),
Lemma 7. Let π be a protocol and f a multiparty function.
w2 = (1960, 128) and b2 = (1, 128), w3 = (128, 10) and
We say that π securely computes f with penalties if π
b3 = (1, 10), respectively. We summarize other training
SCC-realizes the functionality f ∗ .
parameters in Table 3.
Furthermore, based on Lemma 7 where f is a multiparty Threshold Paillier algorithm is implemented in JAVA. We
function, the security defined for fairness is extended to the set the number of bits of modulus nmodel to 1024 bits, which
multi-party setting as shown in Lemma 6 (as shown by corresponds to security level of 80 bits. It is worth noting
Definition 2 of the work [57]). With protocol π , F is SSC- that before executing the encryption algorithm, the weight
∗ ∗
realized as FGradientCollecting and FCollaborativeDecryption , matrices are assembled into a vector, so that only one cipher
meaning that they achieve secure gradient collecting and is generated for a party.
collaborative decryption with penalties, respectively. With We implement the above building blocks to form three
these two functionalities and the trusted time clock mecha- modules, i.e., CordaDeepChain, TrainAlgorithm, and Cryp-
nism, we can guarantee fairness in gradient collecting and toSystem. We evaluate the feasibility of model training on

TABLE 3
Training configuration

Parameter Value
No. of iterations 1500
No. of epochs 1
Learning rate 0.5
Minimal batch size 64

Fig. 6. Training accuracy for the case of four parties.

Fig. 4. Impact of No. of gradients on cipher size.

DeepChain in a multi-party setting by using 4 metrics, that

is cipher size, throughput, training accuracy and total cost
of time. In particular, we evaluate DeepChain on a desktop
computer with 3.3GHz Intel(R) Xeon(R) CPU and 16GB
memory. Then, for each metric we average the final results
over 10 trails. As can be seen in Figure 4, the size of cipher Fig. 7. Training accuracy for the case of ten parties.
remains constant when we encrypt different amounts of
gradients. On the other hand, as the number of gradients TABLE 4
Interpretations of time variants
increases, the throughput decrease steadily, as shown in
Figure 5. Variant Interpretation
In terms of training accuracy, we show that the more t15 iteration time of 15 iterations
parties participate in collaborative training, the higher the tencrypt time to encrypt gradients
training accuracy. For this purpose, we create 7 experimental tuploadByP arty time to upload gradients
tdownloadByW orker time to download gradients from
scenarios with 4, 5, 6, 7, 8, 9, and 10 parties, respectively. all parties
Each party trains the local model with her training dataset taverage time to average all gradients
that contains 5,500 samples. Obviously, the more parties in a tuploadByW orker time to upload updated parameters
scenario, the larger the size of the total dataset, for example, tdownloadByP arty time to download parameters
tdecrypt time to decrypt parameters
the size of the total dataset is 5,500*4 for E-4, and 5,500*10
for E-10.
By sharing gradients on DeepChain, each individual par-
calculated by taking the average of the gradients of all
ty obtains updated parameters contributed by the gradients
parties. We also create an external party, denoted as baseline
from other parties. Specifically, the updated parameters are
party, who only trains the local model on her dataset with
5,500 samples, without taking into account the gradients
from other parties. For space limitation, we only give the
results for E-4 and E-10.
As shown in Fig. 6 and 7, we can see that for both
cases collaborative parties achieve higher training accuracy
than the baseline party. Specifically, in Fig. 6 the baseline
party has an accuracy of 96.75%, whereas Party 1, Party
2, Party 3, and Party 4 achieve 97.08%, 96.96%, 97.20%
and 97.32% accuracy, respectively. Similarly, in Fig. 7, the
accuracy of the baseline party is 96.51%, while the accuracy
numbers of Party 1 to Party 10 are 96.72%, 97.02%, 96.63%,
97.12%, 97.14%, 96.62%, 96.75%, 97.14%, 96.90%, 97.02%,
respectively.
Next, we investigate the time costs of our framework.
Fig. 5. Impact of No. of gradients on throughput. Note that we use the same training model on MNIST dataset

slightly with N . For tuploadByW orker and tdownloadByP arty ,

no matter how large N is, all parties’ ciphers are aggregated
into one cipher that is uploaded/downloaded by work-
ers/parties by using relatively constant time. It is worth
noting that the time cost is also determined by the size of
the training model. Hence, both the number of iterations
and the time for cryptographic operations increase with the
size of the training model.
From Fig. 9 we can see that the total time cost grows
steadily with the number of parties. Specifically, the to-
tal time costs for E-4, E-5, E-6, E-7, E-8, E-9, and E-10
are 391, 861 s, 392, 359 s, 394, 533 s, 394, 287 s, 395, 938 s,
397, 252 s and 398, 079 s, respectively. The reasons for this
trend are that (1) when the number of parties increases,
Fig. 8. Interaction in the entire collaborative training process. tuploadByP arty , tdownloadByW orker , taverage , tuploadByW orker
and tdownloadByP arty also increase, but the increments are
not obvious in the total time, (2) more parties need longer
waiting time to synchronize when sharing gradients, and
(3) tencrypt and tdecrypt dominate the large proportion
of the total time cost that depends on the size of the
training model instead of on the number of parties. In
addition, based on our experiments it is expected that
when the number of parties is more than 1306, the total
time would increase significantly with N , because the time
N × (tdownloadByW orker + taverage ) is greater than time
(tencrypt + tdecrypt ).

7 C ONCLUSION AND F UTURE W ORK

In this paper we present DeepChain, a robust and fair
Fig. 9. Total time cost with increasing number of parties. decentralized platform based on Blockchain for secure col-
laborative deep training. We introduce an incentive mech-
anism and achieve three security goals, namely confiden-
for each experiment E-4, E-5, E-6, E-7, E-8, E-9 and E-10. The tiality, auditability, and fairness. Specifically, we formalize
total execution time is the time spent by a party in the entire the incentive mechanism based on Blockchain, which pos-
collaborative training process on DeepChain. We depict the sesses compatibility and liveness properties. Through our
training process in Fig. 8 by using an interaction diagram. incentive mechanism we demonstrate that participants are
It is worth noting that for efficiency, parties only share incentive to behave correctly with high probability.
gradients with 100 times, i.e., every 15 iterations, instead For confidentiality of local gradients, we employ Thresh-
of No. of iterations in each experiment. This assumption old Paillier algorithm to protect the data. In particular,
follows some researcher’s suggestion that training accuracy by combining a carefully designed component into the
is still acceptable when averaging gradients every 10 to 20 encryption algorithm, we achieve the goal that only one
iterations [70]. cipher is generated for a party. In addition to confidential-
Let the frequency of sharing gradient be #share, the ity, our DeepChain provides auditability and fairness. We
number of iteration be #iteration, the period of shar- use non-interactive zero-knowledge to prove auditability of
ing be #period ∈ [10, 20] and the number of parties the collaborative training process. We also design timeout-
be N . Then, the total time cost can be computed as checking and monetary penalty mechanisms to guarantee
T ime ≈ #iteration
#period × (t15 iteration ) + #share × (tencrypt + fairness among the participants. We implement a prototype
tuploadByP arty + N × tdownloadByW orker + N × taverage + of DeepChain and evaluate it on real dataset, in terms of
tuploadByW orker + ×tdownloadByP arty + tdecrypt ). For ease of cipher size, throughput, training accuracy and training time.
explanation, we summarize all the time-related variables in Our DeepChain may also have some impact on model-
Table 4. based pricing market. Since DeepChain stores not only the
We assume that in the above formula all the time- training parameters, but also the trained models, which
related variables achieve their corresponding time costs means that the trained models can be used for paid deep
when N = 1. When number of parties N increases, the total learning services when the model-based pricing market is
time cost also increases slightly, since tdownloadByW orker and mature. Participants who possess the trained models may
taverage become greater after multiplication. On the other have long-term financial benefits, since they can provide
hand, tuploadByP arty , tuploadByW orker and tdownloadByP arty deep learning services to those who are unable to build the
grow a little bit with N , but their growth rates are negligible. model by themselves but willing to pay for the services.
The reason is that the time used to synchronize State in Cor- On the other hand, all training processes and the model
da platform is very short, so that tuploadByP arty increases parameters are recorded, which could be used in transfer

learning, for example, some information of the trained mod- [14] K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan,
els could be re-used to train a new similar model. Of course, S. Patel, D. Ramage, A. Segal, and K. Seth, “Practical secure aggre-
gation for privacy-preserving machine learning,” in Proceedings of
for transfer learning case, the security problems should be the 2017 ACM SIGSAC Conference on Computer and Communications
re-defined and analyzed, which are left in our future work. Security. ACM, 2017, pp. 1175–1191.
[15] P. Mohassel and Y. Zhang, “Secureml: A system for scalable
privacy-preserving machine learning,” in Security and Privacy (SP),
2017 IEEE Symposium on. IEEE, 2017, pp. 19–38.
ACKNOWLEDGEMENT [16] Y. Aono, T. Hayashi, L. Wang, S. Moriai et al., “Privacy-preserving
deep learning via additively homomorphic encryption,” IEEE
This work was supported by National Key R&D Plan of Transactions on Information Forensics and Security, vol. 13, no. 5, pp.
China (Grant No. 2017YFB0802203 and 2018YFB1003701), 1333–1345, 2018.
National Natural Science Foundation of China (Grant [17] C. Song, T. Ristenpart, and V. Shmatikov, “Machine learning
models that remember too much,” in Proceedings of the 2017
Nos. U1736203, 61732021, 61472165 and 61373158), Guang- ACM SIGSAC Conference on Computer and Communications Security.
dong Provincial Engineering Technology Research Center ACM, 2017, pp. 587–601.
on Network Security Detection and Defence (Grant No. [18] L. Melis, C. Song, E. De Cristofaro, and V. Shmatikov, “Infer-
ence attacks against collaborative learning,” arXiv preprint arX-
2014B090904067), Guangdong Provincial Special Funds for
iv:1805.04049, 2018.
Applied Technology Research and Development and Trans- [19] B. Hitaj, G. Ateniese, and F. Pérez-Cruz, “Deep models under the
formation of Important Scientific and Technological Achieve gan: information leakage from collaborative deep learning,” in
(Grant No. 2016B010124009), the Zhuhai Top Discipline– Proceedings of the 2017 ACM SIGSAC Conference on Computer and
Communications Security. ACM, 2017, pp. 603–618.
Information Security, Guangzhou Key Laboratory of Data
[20] T. Orekondy, S. J. Oh, B. Schiele, and M. Fritz, “Understanding
Security and Privacy Preserving, Guangdong Key Laborato- and controlling user linkability in decentralized learning,” arXiv
ry of Data Security and Privacy Preserving, National Joint preprint arXiv:1805.05838, 2018.
Engineering Research Center of Network Security Detection [21] A. Pyrgelis, C. Troncoso, and E. De Cristofaro, “Knock knock,
and Protection Technology. who’s there? membership inference on aggregate location data,”
arXiv preprint arXiv:1708.06145, 2017.
[22] E. Bagdasaryan, A. Veit, Y. Hua, D. Estrin, and V. Shmatikov, “How
to backdoor federated learning,” arXiv preprint arXiv:1807.00459,
2018.
R EFERENCES [23] “Health insurance portability and accountability act,” http-
s://www.hhs.gov/hipaa/index.html.
[1] G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly,
[24] J. Vaidya, B. Shafiq, X. Jiang, and L. Ohno-Machado, “Identify-
A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath et al., “Deep
ing inference attacks against healthcare data repositories,” AMIA
neural networks for acoustic modeling in speech recognition: The
Summits on Translational Science Proceedings, vol. 2013, p. 262, 2013.
shared views of four research groups,” IEEE Signal processing
magazine, vol. 29, no. 6, pp. 82–97, 2012. [25] G. Heigold, V. Vanhoucke, A. Senior, P. Nguyen, M. Ranzato,
[2] T.-H. Chan, K. Jia, S. Gao, J. Lu, Z. Zeng, and Y. Ma, “Pcanet: M. Devin, and J. Dean, “Multilingual acoustic models using dis-
A simple deep learning baseline for image classification?” IEEE tributed deep neural networks,” in Acoustics, Speech and Signal
Transactions on Image Processing, vol. 24, no. 12, pp. 5017–5032, 2015. Processing (ICASSP), 2013 IEEE International Conference on. IEEE,
2013, pp. 8619–8623.
[3] E. Gawehn, J. A. Hiss, and G. Schneider, “Deep learning in drug
discovery,” Molecular informatics, vol. 35, no. 1, pp. 3–14, 2016. [26] R. Jurca and B. Faltings, “An incentive compatible reputation
[4] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature, vol. mechanism,” in EEE International Conference on E-Commerce, 2003.
521, no. 7553, p. 436, 2015. CEC 2003. IEEE, 2003, pp. 285–292.
[5] P. Danaee, R. Ghaeini, and D. A. Hendrix, “A deep learning [27] U. Shevade, H. H. Song, L. Qiu, and Y. Zhang, “Incentive-aware
approach for cancer detection and relevant gene identification,” routing in dtns,” in 2008 IEEE International Conference on Network
in PACIFIC SYMPOSIUM ON BIOCOMPUTING 2017. World Protocols. IEEE, 2008, pp. 238–247.
Scientific, 2017, pp. 219–229. [28] S. Zhong, J. Chen, and Y. R. Yang, “Sprite: A simple, cheat-
[6] S. Gupta, W. Zhang, and F. Wang, “Model accuracy and runtime proof, credit-based system for mobile ad-hoc networks,” in IEEE
tradeoff in distributed deep learning: A systematic study,” in Data INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE
Mining (ICDM), 2016 IEEE 16th International Conference on. IEEE, Computer and Communications Societies (IEEE Cat. No. 03CH37428),
2016, pp. 171–180. vol. 3. IEEE, 2003, pp. 1987–1997.
[7] T. Chilimbi, Y. Suzue, J. Apacible, and K. Kalyanaraman, “Project [29] B. B. Chen and M. C. Chan, “Mobicent: a credit-based incentive
adam: building an efficient and scalable deep learning training system for disruption tolerant network,” in 2010 Proceedings IEEE
system,” in Usenix Conference on Operating Systems Design and INFOCOM. IEEE, 2010, pp. 1–9.
Implementation, 2016, pp. 571–582. [30] S. Nakamoto, “Bitcoin: A peer-to-peer electronic cash system,”
[8] T. Chen and S. Zhong, “Privacy-preserving backpropagation 2008.
neural network learning,” IEEE Transactions on Neural Networks, [31] M. Ben-Or and A. Hassidim, “Fast quantum byzantine agreemen-
vol. 20, no. 10, p. 1554, 2009. t,” in Proceedings of the thirty-seventh annual ACM symposium on
[9] A. Bansal, T. Chen, and S. Zhong, “Privacy preserving back- Theory of computing. ACM, 2005, pp. 481–485.
propagation neural network learning over arbitrarily partitioned [32] S. Micali, “Algorand: The efficient and democratic ledger,” arXiv
data,” Neural Computing Applications, vol. 20, no. 1, pp. 143–150, preprint arXiv:1607.01341, 2016.
2011. [33] Y. Gilad, R. Hemo, S. Micali, G. Vlachos, and N. Zeldovich,
[10] J. Yuan and S. Yu, “Privacy preserving back-propagation learning “Algorand: Scaling byzantine agreements for cryptocurrencies,”
made practical with cloud computing,” IEEE Transactions on Paral- in Proceedings of the 26th Symposium on Operating Systems Principles.
lel Distributed Systems, vol. 25, no. 1, pp. 212–221, 2014. ACM, 2017, pp. 51–68.
[11] R. Shokri and V. Shmatikov, “Privacy-preserving deep learning,” [34] K. Nikitin, E. Kokoris-Kogias, P. Jovanovic, N. Gailly, L. Gasser,
in Allerton Conference on Communication, Control, and Computing, I. Khoffi, J. Cappos, and B. Ford, “{CHAINIAC}: Proactive
2015, pp. 909–910. software-update transparency via collectively signed skipchain-
[12] P. Li, J. Li, Z. Huang, C. Z. Gao, W. B. Chen, and K. Chen, “Privacy- s and verified builds,” in 26th {USENIX} Security Symposium
preserving outsourced classification in cloud computing,” Cluster ({USENIX} Security 17), 2017, pp. 1271–1287.
Computing, no. 1, pp. 1–10, 2017. [35] S. Hu, C. Cai, Q. Wang, C. Wang, X. Luo, and K. Ren, “Searching
[13] Q. Zhang, L. Yang, and Z. Chen, “Privacy preserving deep com- an encrypted cloud meets blockchain: A decentralized, reliable
putation model on cloud for big data feature learning,” IEEE and fair realization,” in IEEE INFOCOM 2018-IEEE Conference on
Transactions on Computers, vol. 65, no. 5, pp. 1351–1362, 2016. Computer Communications. IEEE, 2018, pp. 792–800.

[36] A. B. Kurtulmus and K. Daniel, “Trustless machine learning con- [61] P. Paillier, “Public-key cryptosystems based on composite degree
tracts; evaluating and exchanging machine learning models on the residuosity classes,” in International Conference on the Theory and
ethereum blockchain,” arXiv preprint arXiv:1802.10185, 2018. Applications of Cryptographic Techniques. Springer, 1999, pp. 223–
[37] E. B. Sasson, A. Chiesa, C. Garman, M. Green, I. Miers, E. Tromer, 238.
and M. Virza, “Zerocash: Decentralized anonymous payments [62] B. Schoenmakers and M. Veeningen, “Universally verifiable multi-
from bitcoin,” in Security and Privacy (SP), 2014 IEEE Symposium party computation from threshold homomorphic cryptosystems,”
on. IEEE, 2014, pp. 459–474. in International Conference on Applied Cryptography and Network
[38] I. Miers, C. Garman, M. Green, and A. D. Rubin, “Zerocoin: Security. Springer, 2015, pp. 3–22.
Anonymous distributed e-cash from bitcoin,” in Security and Pri- [63] I. B. Damgård and M. J. Jurik, “Efficient protocols based on
vacy (SP), 2013 IEEE Symposium on. IEEE, 2013, pp. 397–411. probabilistic encryption using composite degree residue classes,”
[39] A. Kosba, A. Miller, E. Shi, Z. Wen, and C. Papamanthou, “Hawk: BRICS Report Series, vol. 7, no. 5, 2000.
The blockchain model of cryptography and privacy-preserving [64] V. Shoup, “Practical threshold signatures,” 1999.
smart contracts,” in Security and Privacy (SP), 2016 IEEE Symposium [65] O. Goldreich, Foundations of cryptography: volume 2, basic applica-
on. IEEE, 2016, pp. 839–858. tions. Cambridge university press, 2009.
[40] S. Haykin, Neural networks: a comprehensive foundation. Prentice [66] R. Canetti, “Universally composable security: A new paradigm
Hall PTR, 1994. for cryptographic protocols,” in Proceedings 2001 IEEE International
[41] H. Cui, G. R. Ganger, and P. B. Gibbons, “Scalable deep learning Conference on Cluster Computing. IEEE, 2001, pp. 136–145.
on distributed gpus with a gpu-specialized parameter server,” pp. [67] “Corda: an open source distributed ledger platform,” http-
1–16, 2016. s://docs.corda.net/.
[42] H. Ma, F. Mao, and G. W. Taylor, “Theano-mpi: A theano-based [68] W. Gavin, “Ethereum: A secure decentralised generalised transac-
distributed training framework,” CoRR, pp. 800–813, 2016. tion ledger,” Ethereum Project Yellow Paper, vol. 151, 2014.
[43] Poseidon: An Efficient Communication Architecture for Distributed [69] C. J. B. Yann LeCun, Corinna Cortes, “The mnist database of
Deep Learning on GPU Clusters. handwritten digits,” http://yann.lecun.com/exdb/mnist/.
[44] S. Rajendran, W. Meert, D. Giustiniano, V. Lenders, and S. Pollin, [70] H. Su and H. Chen, “Experiments on parallel training of deep
“Distributed deep learning models for wireless signal classifica- neural network using model averaging,” arXiv preprint arX-
tion with low-cost spectrum sensors,” CoRR, vol. abs/1707.08908, iv:1507.01239, 2015.
2017.
[45] Distributed deep learning on edge-devices: Feasibility via adaptive com-
pression, 2017.
[46] J. Dean, G. Corrado, Monga et al., “Large scale distributed deep
networks,” in Advances in neural information processing systems,
2012, pp. 1223–1231.
[47] N. Vasilache, J. Johnson, M. Mathieu, S. Chintala, S. Piantino, and
Y. LeCun, “Fast convolutional nets with fbfft: A gpu performance
evaluation,” arXiv preprint arXiv:1412.7580, 2014. Jiasi Weng received the B.S. degree in soft-
[48] R. Wu, S. Yan, Y. Shan, Q. Dang, and G. Sun, “Deep image: Scaling ware engineering from South China Agriculture
up image recognition,” arXiv preprint arXiv:1501.02876, vol. 7, University in June 2016. Currently, she is a
no. 8, 2015. Ph.D. student with School of Information Sci-
[49] M. Lin, S. Li, X. Luo, and S. Yan, “Purine: A bi-graph based deep ence and Technology in Jinan University. Her
learning framework,” arXiv preprint arXiv:1412.6249, 2014. research interests include applied cryptography,
[50] L. Chen, P. Koutris, and A. Kumar, “Model-based pricing for Blockchain, network security, etc.
machine learning in a data marketplace,” arXiv preprint arX-
iv:1805.11450, 2018.
[51] M. Belenkiy, M. Chase, C. C. Erway, J. Jannotti, A. Küpçü, and
A. Lysyanskaya, “Incentivizing outsourced computation,” in Pro-
ceedings of the 3rd international workshop on Economics of networked
systems. ACM, 2008, pp. 85–90.
[52] J.-S. Weng, J. Weng, M. Li, Y. Zhang, and W. Luo, “Deepchain:
Auditable and privacy-preserving deep learning with blockchain-
based incentive,” Cryptology ePrint Archive, Report 2018/679,
2018, https://eprint.iacr.org/2018/679.
[53] F. McKeen, I. Alexandrovich, A. Berenzon, C. V. Rozas, H. Shafi,
V. Shanbhogue, and U. R. Savagaonkar, “Innovative instructions Jian Weng is a professor and the Executive
and software model for isolated execution.” HASP@ ISCA, vol. 10, Dean with College of Information Science and
2013. Technology in Jinan University. He received B.S.
[54] P.-A. Fouque, G. Poupard, and J. Stern, “Sharing decryption in degree and M.S. degree from South China U-
the context of voting or lotteries,” in International Conference on niversity of Technology in 2001 and 2004 re-
Financial Cryptography. Springer, 2000, pp. 90–104. spectively, and Ph.D. degree at Shanghai Jiao
[55] T. Nishide and K. Sakurai, “Distributed paillier cryptosystem Tong University in 2008. His research areas in-
without trusted dealer,” in International Workshop on Information clude public key cryptography, cloud security,
Security Applications. Springer, 2010, pp. 44–60. blockchain, etc. He has published 80 papers
[56] A. Shamir, “How to share a secret,” Communications of the ACM, in international conferences and journals such
vol. 22, no. 11, pp. 612–613, 1979. as CRYPTO, EUROCRYPT, ASIACRYPT, TC-
[57] I. Bentov and R. Kumaresan, “How to use bitcoin to design fair C, PKC, CT-RSA, IEEE TPAMI, IEEE TDSC, etc. He also serves as
protocols,” in International Cryptology Conference. Springer, 2014, associate editor of IEEE Transactions on Vehicular Technology. He
pp. 421–439. received the Young Scientists Fund of the National Natural Science
[58] R. Kumaresan and I. Bentov, “How to use bitcoin to incentivize Foundation of China in 2018, and the Cryptography Innovation Award
correct computations,” in Proceedings of the 2014 ACM SIGSAC from Chinese Association for Cryptologic Research (CACR) in 2015. He
Conference on Computer and Communications Security. ACM, 2014, served as General Co-Chair for SecureComm 2016, TPC Co-Chairs for
pp. 30–41. RFIDsec’13 Asia and ISPEC 2011, and program committee members
[59] A. Demers, D. Greene, C. Hauser, W. Irish, J. Larson, S. Shenker, for more than 40 international cryptography and information security
H. Sturgis, D. Swinehart, and D. Terry, “Epidemic algorithms for conferences. He also serves as associate editor of IEEE Transactions
replicated database maintenance,” in Proceedings of the sixth annual on Vehicular Technology.
ACM Symposium on Principles of distributed computing. ACM, 1987,
pp. 1–12.
[60] E. Buchman, “Tendermint: Byzantine fault tolerance in the age of
blockchains,” Ph.D. dissertation, 2016.

Jilian Zhang received his M.S. degree from

Guangxi Normal University, China in 2006, and
Ph.D degree from Singapore Management U-
niversity in 2014. He is currently an associate
professor with College of Cyber Security, Ji-
nan University, Guangzhou China. Jilian Zhang’s
research interests include data management,
query processing, and database security. His
work has been published on international jour-
nals and conferences, including IEEE TKDE,
ACM SIGMOD, VLDB, and IJCAI.

Ming Li received his B.S. in electronic informa-

tion engineering from University of South China
in 2009, and M.S. in information processing from
Northwestern Polytechnical University in 2012.
From 2016, he started his Ph. D. at Jinan Univer-
sity. His research interests include crowdsourc-
ing, blockchain and its privacy and security.

Yue Zhang received his M.S. degree and B.S.

degree from Xi’an University of Posts & Telecom-
munications in 2014 and 2016, respectively. S-
ince September 2016, he is a Ph.D. student with
School of Information Science and Technology in
Jinan University. His research interests include
Blockchain, system security, android security,
etc.

Weiqi Luo received his B.S. degree and M.S.

degree from Jinan University in 1982 and 1985
respectively, and Ph.D. degree from South China
University of Technology in 1999. Currently, he is
a professor with School of Information Science
and Technology in Jinan University. His research
interests include network security, big data, artifi-
cial intelligence, etc. He has published more than
100 high-quality papers in international journals
and conferences.

Customer Service Case Studies
88% (8)
Customer Service Case Studies
4 pages
Kubota ZD1011 - ZD1021 Zero Turn Mower Operators Manual
No ratings yet
Kubota ZD1011 - ZD1021 Zero Turn Mower Operators Manual
100 pages
3006-0510-2019-07-23 - Method Statements PDF
No ratings yet
3006-0510-2019-07-23 - Method Statements PDF
1 page
Developing DApps On EOS MuditMarda
No ratings yet
Developing DApps On EOS MuditMarda
18 pages
Differential Privacy in Deep Learning: An Overview
No ratings yet
Differential Privacy in Deep Learning: An Overview
6 pages
Decentralised Deep
No ratings yet
Decentralised Deep
14 pages
SecureBoost A Lossless Federated Learning Framework
No ratings yet
SecureBoost A Lossless Federated Learning Framework
9 pages
Privacy-Preserving Deep Learning
No ratings yet
Privacy-Preserving Deep Learning
12 pages
1912.05897
No ratings yet
1912.05897
11 pages
COM3030 Week 10 Slides (2)
No ratings yet
COM3030 Week 10 Slides (2)
63 pages
Privacy-Preserving Federated Learning Based on Differential Privacy and Momentum
No ratings yet
Privacy-Preserving Federated Learning Based on Differential Privacy and Momentum
6 pages
Federated Learning a Survery
No ratings yet
Federated Learning a Survery
31 pages
NIT Research Proposal1
No ratings yet
NIT Research Proposal1
2 pages
22-tbd
No ratings yet
22-tbd
14 pages
A Generic Framework For Privacy Preserving Deep Learning: Member of The Openmined Community
No ratings yet
A Generic Framework For Privacy Preserving Deep Learning: Member of The Openmined Community
5 pages
BIT-FL_Blockchain-Enabled_Incentivized_and_Secure_Federated_Learning_Framework.pdf
No ratings yet
BIT-FL_Blockchain-Enabled_Incentivized_and_Secure_Federated_Learning_Framework.pdf
18 pages
A Federated Learning Framework Based on CSP Homomorphic Encryption (1)
No ratings yet
A Federated Learning Framework Based on CSP Homomorphic Encryption (1)
6 pages
Blockchain For Federated Learning
No ratings yet
Blockchain For Federated Learning
18 pages
1 s2.0 S0925231223010202 Main
No ratings yet
1 s2.0 S0925231223010202 Main
18 pages
1-s2.0-S1568494624011797-main
No ratings yet
1-s2.0-S1568494624011797-main
13 pages
A Review of Machine Learning and Cryptography Applications
No ratings yet
A Review of Machine Learning and Cryptography Applications
7 pages
2004.12088v5
No ratings yet
2004.12088v5
14 pages
Robust and Communication-Efficient Federated Learning From Non-i.i.d. Data
No ratings yet
Robust and Communication-Efficient Federated Learning From Non-i.i.d. Data
14 pages
Research Article - A Privacy-Preserving Data Stream Mining Technique Based On Cumulative and Independent Additive Noise For Improving Random Projection
No ratings yet
Research Article - A Privacy-Preserving Data Stream Mining Technique Based On Cumulative and Independent Additive Noise For Improving Random Projection
26 pages
Sec22 Stevens
No ratings yet
Sec22 Stevens
18 pages
Federated Learning
100% (1)
Federated Learning
22 pages
105 Development of Javascript Base
No ratings yet
105 Development of Javascript Base
10 pages
Federated Learning With Differential Privacy Algorithms and Performance Analysis
No ratings yet
Federated Learning With Differential Privacy Algorithms and Performance Analysis
16 pages
Deepsecure: Scalable Provably-Secure Deep Learning
No ratings yet
Deepsecure: Scalable Provably-Secure Deep Learning
6 pages
Recent Advances
No ratings yet
Recent Advances
18 pages
Practical Secure Aggregation For Federated Learning On User Held Data
No ratings yet
Practical Secure Aggregation For Federated Learning On User Held Data
5 pages
UNIT-1 Foundations of Deep Learning
100% (1)
UNIT-1 Foundations of Deep Learning
51 pages
A Hybrid Approach To Privacy-Preserving Federated Learning
No ratings yet
A Hybrid Approach To Privacy-Preserving Federated Learning
11 pages
Research Article: Deep Learning Data Privacy Protection Based On Homomorphic Encryption in Aiot
No ratings yet
Research Article: Deep Learning Data Privacy Protection Based On Homomorphic Encryption in Aiot
11 pages
Review Article: A Review of Deep Learning Security and Privacy Defensive Techniques
No ratings yet
Review Article: A Review of Deep Learning Security and Privacy Defensive Techniques
18 pages
Applied Sciences Privacy-Preserving Federated Learning Using Homomorphic Encryption
No ratings yet
Applied Sciences Privacy-Preserving Federated Learning Using Homomorphic Encryption
17 pages
Robust and Communication-Efficient Federated Learning From Non-IID Data
No ratings yet
Robust and Communication-Efficient Federated Learning From Non-IID Data
17 pages
Blockchain For Federated Learning
No ratings yet
Blockchain For Federated Learning
18 pages
Research Paper 8
No ratings yet
Research Paper 8
7 pages
Mobile Information Systems - 2020 - Tariq - A Review of Deep Learning Security and Privacy Defensive Techniques
No ratings yet
Mobile Information Systems - 2020 - Tariq - A Review of Deep Learning Security and Privacy Defensive Techniques
18 pages
Research Paper 7
No ratings yet
Research Paper 7
12 pages
Adaptive Laplace Mechanism: Differential Privacy Preservation in Deep Learning
No ratings yet
Adaptive Laplace Mechanism: Differential Privacy Preservation in Deep Learning
13 pages
Bharati Et Al 2022 Federated Learning Applications Challenges and Future Directions
No ratings yet
Bharati Et Al 2022 Federated Learning Applications Challenges and Future Directions
17 pages
The role of deep learning in ensuring privacy integrity and security-Applications in
No ratings yet
The role of deep learning in ensuring privacy integrity and security-Applications in
13 pages
Privacy
No ratings yet
Privacy
14 pages
Deep Learning With Differential Privacy
No ratings yet
Deep Learning With Differential Privacy
14 pages
Blockchain For Deep Learning: Review and Open Challenges
No ratings yet
Blockchain For Deep Learning: Review and Open Challenges
25 pages
Adversarial ML Survey Paper
No ratings yet
Adversarial ML Survey Paper
23 pages
Unit-3 Notes
No ratings yet
Unit-3 Notes
16 pages
Anomaly detection and defense techniques in federated learning a comprehensive review
No ratings yet
Anomaly detection and defense techniques in federated learning a comprehensive review
34 pages
Differential_Privacy_for_Deep_and_Federated_Learning_A_Survey
No ratings yet
Differential_Privacy_for_Deep_and_Federated_Learning_A_Survey
22 pages
Quality_Inference_in_Federated_Learning_With_Secure_Aggregation
No ratings yet
Quality_Inference_in_Federated_Learning_With_Secure_Aggregation
8 pages
ML Archs
No ratings yet
ML Archs
36 pages
Privacy and Security Issues in Deep Learning A Survey
No ratings yet
Privacy and Security Issues in Deep Learning A Survey
28 pages
Shayan et al._ Biscotti
No ratings yet
Shayan et al._ Biscotti
13 pages
A Trusted Recommendation Scheme for Privacy Protection
No ratings yet
A Trusted Recommendation Scheme for Privacy Protection
11 pages
Mathematics 11 01091 With Cover
No ratings yet
Mathematics 11 01091 With Cover
17 pages
2017 - Privacy-Preserving
No ratings yet
2017 - Privacy-Preserving
20 pages
D P M - L: Ifferentially Rivate ETA Earning
No ratings yet
D P M - L: Ifferentially Rivate ETA Earning
17 pages
Communication-Efficient Learning of Deep Networks From Decentralized Data
No ratings yet
Communication-Efficient Learning of Deep Networks From Decentralized Data
11 pages
Deng 2022 Auction
No ratings yet
Deng 2022 Auction
14 pages
Expert Systems - 2023 - Gopalakrishnan - PriMed Private Federated Training and Encrypted Inference On Medical Images in
No ratings yet
Expert Systems - 2023 - Gopalakrishnan - PriMed Private Federated Training and Encrypted Inference On Medical Images in
14 pages
Essential Federated Learning: AI at the Edge
From Everand
Essential Federated Learning: AI at the Edge
Robert Johnson
No ratings yet
SC To BZA TICKET TRAIN
No ratings yet
SC To BZA TICKET TRAIN
2 pages
Guidance For Industry: Size of Beads in Drug Products Labeled For Sprinkle
No ratings yet
Guidance For Industry: Size of Beads in Drug Products Labeled For Sprinkle
7 pages
1. Cell structure and function answers
No ratings yet
1. Cell structure and function answers
2 pages
VADIVEL
No ratings yet
VADIVEL
43 pages
Further Topics in Industry and Competitive Analysis
No ratings yet
Further Topics in Industry and Competitive Analysis
21 pages
Cerambycidae
No ratings yet
Cerambycidae
12 pages
Pullman - Wa Body Camera Policy
No ratings yet
Pullman - Wa Body Camera Policy
5 pages
3.3 Solving Equations with Word Problems
No ratings yet
3.3 Solving Equations with Word Problems
3 pages
DYN Model Overview
No ratings yet
DYN Model Overview
13 pages
Mathematical Analysis and Its Applications 2015
No ratings yet
Mathematical Analysis and Its Applications 2015
752 pages
NAUGHTY BOY LYRICS - La La La PDF
No ratings yet
NAUGHTY BOY LYRICS - La La La PDF
2 pages
Homework 1 - Legal Aspects
No ratings yet
Homework 1 - Legal Aspects
2 pages
Ecological Indicators: Sciencedirect
No ratings yet
Ecological Indicators: Sciencedirect
13 pages
Stqa Unit4
No ratings yet
Stqa Unit4
71 pages
Modernization: Assistant Professor, Dept. of Sociology, Mahatma Gandhi Central University, Bihar
No ratings yet
Modernization: Assistant Professor, Dept. of Sociology, Mahatma Gandhi Central University, Bihar
10 pages
Action Plan in Campus Journalism
100% (1)
Action Plan in Campus Journalism
7 pages
Form SS-4 For ESTATE ONLY Template
100% (1)
Form SS-4 For ESTATE ONLY Template
2 pages
Carboxylic Acid and Amines Worksheet PDF
No ratings yet
Carboxylic Acid and Amines Worksheet PDF
22 pages
Understanding Light Tutorial 7
No ratings yet
Understanding Light Tutorial 7
42 pages
Đề Minh Họa Tốt Nghiệp 2024 Của BGD Ra Ngày 22-3-2024 - Môn Tiếng Anh (Bản Word Có Đáp Án Tham Khảo) - 1nCSFCpaiX
No ratings yet
Đề Minh Họa Tốt Nghiệp 2024 Của BGD Ra Ngày 22-3-2024 - Môn Tiếng Anh (Bản Word Có Đáp Án Tham Khảo) - 1nCSFCpaiX
7 pages
Dates Format in Sas
No ratings yet
Dates Format in Sas
6 pages
Hose Binding Data Sheet3
No ratings yet
Hose Binding Data Sheet3
2 pages
5 Fun Facts About Fakesgiving
No ratings yet
5 Fun Facts About Fakesgiving
10 pages
Cape Biology Unit 2 Lab PDF Free
No ratings yet
Cape Biology Unit 2 Lab PDF Free
4 pages
TOIEC Grammar - Relative Clauses
100% (1)
TOIEC Grammar - Relative Clauses
6 pages
Jecrc Hackathon PPT (2) - 2
No ratings yet
Jecrc Hackathon PPT (2) - 2
17 pages
Kamlesh Resume
No ratings yet
Kamlesh Resume
4 pages

Deep Chain

Uploaded by

Deep Chain

Uploaded by

This article has been accepted for publication in a future issue of this journal, but has not been

DeepChain: Auditable and Privacy-Preserving

Index Terms—Deep learning, Privacy-preserving training, Blockchain, Incentive

R ECENT advances in deep learning based on artificial

attack on aggregated location data is now available [21].

Notation Description Number of parties Maximum number Threshold

Algorithm 1: Trading(T raniP1 ,...,T raniPN ) Algorithm 2: Processing()

The proof P roofCDi,j is used to verify validity of the de-

$Coins in DeepChain. It is worth mentioning that w is C ← S(1λ , wb ) :

Fig. 6. Training accuracy for the case of four parties.

Fig. 4. Impact of No. of gradients on cipher size.

DeepChain in a multi-party setting by using 4 metrics, that

slightly with N . For tuploadByW orker and tdownloadByP arty ,

7 C ONCLUSION AND F UTURE W ORK

Jilian Zhang received his M.S. degree from

Ming Li received his B.S. in electronic informa-

Yue Zhang received his M.S. degree and B.S.

Weiqi Luo received his B.S. degree and M.S.

You might also like