MongoDB .local Houston 2019: Using Client Side Encryption in MongoDB 4.2

db.coll.insert({
_id: 1,
name: "Doris",
ssn: "457-55-5462"
})

doc = db.coll.find_one({
ssn: "457-55-5462"
})

{
_id: 1
name: "Doris",
ssn: "457-55-5462"
}

db.coll.insert({
name: "Doris",
ssn: "457-55-5462"
})
{
insert: "coll",
documents: [{
name: "Doris",
ssn: BinData(6, "a10x…")
}]
}
You see: MongoDB sees:
Encrypt before sending

{
_id: 1
name: "Doris",
}
Driver receives: You see:
{
_id: 1
name: "Doris",
ssn: "457-55-5462"
}
Decrypt after receiving

How does this differ from…?
•… encryption in-transit (TLS)
•… encryption at-rest (encrypted storage engine)

Attacker
Query
Client
Disk
insert write
MongoDB
Auth
db.coll.insert({ name: "Doris", ssn: "457-55-5462" })

Client
Disk
insert write
MongoDB
Attacker
Snoop
TLS

Client
Disk
insert write
MongoDB
Attacker
insert
TLS

Client
Disk
insert write
MongoDB
Attacker
Steal
ESE

Client
Disk
insert write
MongoDB
Attacker
Login
Client Side Encryption

Client
Disk
insert write
MongoDB
Boundaries of unencrypted data

Client
Disk
insert write
MongoDB
… with Encrypted Storage Engine

Client
Disk
insert write
MongoDB
… and TLS

Client
Disk
insert write
MongoDB
with Client Side Encryption

Client
Disk
insert write
MongoDB

db.coll.update({}, {
$set: { ssn: "457-55-5462" }
})
{
update: "coll",
updates: [{
q:{},
u: {
$set: { ssn: BinData(6, "a10x…") }
}
}]
}
Update that overwrites value

db.coll.aggregate([{
$project: { name_ssn: {$concat: [ "$name", " - ", "$ssn" ] } }
}]
Aggregate acting on the data

Find with equality query
* For deterministic encryption
db.coll.find({ssn: "457-55-5462" }) {
find: "coll",
filter: { ssn: BinData(6, "a10x…") }
}

db.test.find(
{
$and: [
{
$or: [
{ ssn : { $in : [ "457-55-5462", "153-96-2097" ]} },
{ ssn: { $exists: false } }
]
},
{ name: "Doris" }
]
}
)
You see:

MongoDB sees:
{
find: "coll",
filter: {
$and: [
{
$or: [
{ ssn : { $in : [ BinData(6, "a10x…"), BinData(6, "8dk1…") ]} },
{ ssn: { $exists: false } }
]
},
{ name: "Doris" }
]
}
}

Doris
Private stuff in storage

PoliceDoris
Private stuff in storage

Vault key
Held only by you
Vault

Encrypted Data
MongoDB
Encryption Key

{ _id: 1, ssn: BinData(0, "A81…"), name: "Kevin" }
{ _id: 2, ssn: BinData(0, "017…"), name: "Eric" }
{ _id: 3, ssn: BinData(0, "5E1…"), name: "Albert" }
…

Destroy the key
Provably delete all user data.
GDPR "right-to-be-forgotten"

client = MongoClient(
auto_encryption_opts=opts)

{
name: "Doris"
ssn: "457-55-5462",
email: "Doris@gmail.com",
credit_card: "4690-6950-9373-8791",
comments: [ …. ],
avatar: BinData(0, "0fi8…"),
profile: { likes: {…}, dislikes: {…} }
}

Describes JSON
{
bsonType: "object",
properties: {
a: {
bsonType: "int"
maximum: 10
}
b: { bsonType: "string" }
},
required: ["a", "b"]
}
{
a: 5,
b: "hi"
}
{
a: 11,
b: false
}
JSON Schema

{
bsonType: "object",
properties: {
ssn: {
encrypt: { … }
}
},
required: ["ssn"]
}
JSON Schema "encrypt"

encrypt: {
keyId: <UUID[]> or <string>,
algorithm: <string>
bsonType: <string> or <string[]>
}
bsonType indicates the type of underlying data.
algorithm indicates how to encrypt (Random or Deterministic).
keyId indicates the key used to encrypt.

opts = AutoEncryptionOptions(
schema_map = { "db.coll": <schema> }
…)

Remote Schema Fallback
db.createCollection("coll", { validator: { $jsonSchema: … } } )
Misconfigured
Client insert "457-55-5462"
error, that should be
encrypted
MongoDB

What if
… the server lies about the schema?
Misconfigured
Client insert "457-55-5462"
Evil MongoDB
ok :)

Key vault
Key vault key
Held only by you

schema_map = { "db.coll": <schema> },
key_vault_namespace = "db.keyvault"
…)

schema_map
Sub-options
key_vault_namespace

What if
… attacker drops key vault collection?

Keep at home
key_vault_namespace = "db.keyvault",
key_vault_client = <client>
…)

schema_map
Sub-options
key_vault_namespace
key_vault_client

Protects keys Stores keys
KMS
Key vault key
Key vault
Key vault
collection

key_vault_namespace = "db.keys",
kms_providers = <creds>
…)

schema_map
Sub-options
key_vault_namespace
key_vault_client
kms_providers

db.coll.insert({
name: "Doris",
ssn: "457-55-5462"
})
Get encrypted key
Decrypt the key with KMSDecrypt the key with KMS
Encrypt 457-55-5462
Send insert
Compare to JSON schema

Authenticated Encryption with Associated Data using the Advance

AEAD_AES_256_CBC_HMAC_SHA_512
Provides confidentiality + integrity

AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic
AEAD_AES_256_CBC_HMAC_SHA_512-Random

coll.insert({ ssn: "457-55-5462" }) { ssn: BinData(6, "a10x…") }
You see: MongoDB stores:
coll.insert({ ssn: "457-55-5462" }) { ssn: BinData(6, "f991…") }
…Random

You see: MongoDB stores:
…Deterministic

Can be queried
doc = db.coll.find({
ssn: "457-55-5642"
})
{
find: "coll",
filter: { ssn: BinData(0, "a10x…") }
}
Driver sends:
{ ssn: BinData(6, "a10x…") }
MongoDB returns:
…Deterministic

Only for binary comparable types.
db.coll.find({ a: { b: 1.0 } })
{ a: { b: NumberInt(1) } }
{ a: { b: 1.0 } }
{ a: { b: NumberLong(1) } }
MongoDB returns:
…Deterministic

{ a: { b: NumberInt(1) } }
{ a: { b: 1.0 } }
{ a: { b: NumberLong(1) } }
{ a: BinData(6, "19d0…") }
{ a: BinData(6, "b515…") }
{ a: BinData(6, "801f…") }
Encrypted as:

db.coll.find({ a: { b: 1.0 } })
{ a: { b: 1.0 } }
MongoDB returns:
"a" encrypted

{ ssn: BinData(6, "AWNkTYTCw89Ss1DPzV3/2pSRDNGNJ9NB" }
New binary subtype
Older drivers and older MongoDB will treat as a black box.

byte algorithm
byte[16] key_id
byte original_bson_type
byte* payload
Ciphertext

byte algorithm
byte[16] key_id
byte* payload
key_id + algorithm describes how to decrypt.
No JSON Schema necessary!
Ciphertext

byte algorithm
byte[16] key_id
byte* payload
Provides extra server-side validation.
But prohibits single-value types (MinKey, MaxKey, Undefined, Null)
Ciphertext

byte algorithm
byte[16] key_id
byte* payload
Payload includes encoded IV and padding block, and HMAC.
Ciphertext adds between 66 to 82 bytes of overhead.
Ciphertext

opts = ClientEncryptionOptions(…)
ce = ClientEncryption(opts)
opts = DataKeyOpts(kms_provider="aws",
master_key="…")
key_id = ce.create_data_key(opts)
ciphertext = ce.decrypt(ciphertext)
opts = EncryptOpts(algorithm="…", key_id="…")
ciphertext = ce.encrypt("457-55-5462", opts)

db.test.find({
$or: [
{ ssn : { $in : [ "457-55-5462", "153-96-2097" ]} },
{ name: "Doris" }
]
})
db.test.find({
$or: [
{ ssn : { $in : [ BinData(6, "a10x…"), BinData(6, "8dk1…") ]} },
{ name: "Doris" }
]
})

Limitations
If we cannot parse…
or it is impossible…
we err for safety.

{ passport_num: "281-301-348", ssn: "457-55-5462" }
{ passport_num: "390-491-482", ssn: "482-38-5899" }
{ passport_num: "104-201-596" }
passport_num and ssn encrypted with different keys

db.test.aggregate([
])
{ $project: { identifier: { $ifNull: ["$ssn", "$passport_num" ] } } },
{ $match: { identifier: "457-55-5462" } }
How do we encrypt 457-55-5462?

bypass_auto_encryption = True
…)
client = MongoClient(auto_encryption_opts=opts)
(Decryption still occurs)

ce = ClientEncryption(opts)
db.test.aggregate([
])
{ $project: { identifier: { $ifNull: ["$ssn", "$passport_num" ] } } },
{ $match: { identifier: ce.encrypt("457-55-5462", opts) } }

{
_id: UUID(…)
keyAltNames: [ "mykey" ],
keyMaterial: BinData(0, "39aJ…"),
… (some metadata) …
}
> db.keyvault.find()
Identify
(Cached locally)

encryption = ClientEncryption(…)
id = encryption.create_data_key(…)
print (id)
Script prints: "609"

{
_id: 23
email: "Doris@gmail.com",
pwd: "19dg8%"
following: [ 9, 20, 95 ]
}

…
"email": {
"encrypt": {
"keyId": 609,
"algorithm": "…Deterministic"
"bsonType": "string",
}
},
"pwd": {
"encrypt": {
"keyId": 609,
"algorithm": "…Random"
"bsonType": "string"
}
}
…

client = MongoClient(
auto_encryption_opts=opts)
def register(db, email, pwd):
db.users.insert_one({ "email": email, "pwd": pwd })
def login(db, email, pwd):
user = db.users.find_one({ "email": email })
if user and matches(user["pwd"], pwd):
return True
else:
return False

client = MongoClient()
for doc in client.db.users.find():
print doc
{
_id: 23
email: BinData(6, 810f…"),
pwd: BinData(6, "19A0…")
}

{
user_id: 123,
date: Date("6/8/2019"),
title: "My first entry",
body: "Dear diary, … "
}
…
"body": {
"encrypt": {
"keyId": 609,
"algorithm": "…Random",
}
}
…

def create_post(db, user_id, title, body):
db.posts.insert_one({
"user_id": user_id,
"date": datetime.now(),
"title": title,
"body": body
})

def delete_user_data(db, user_id):
db.posts.delete_many({ "user_id": user_id })

encryption = ClientEncryption(…)
def register(encryption, db, email, pwd):
user_id = db.users.insert_one({
"email": email,
"pwd": pwd
}).inserted_id
opts = DataKeyOpts(keyAltNames=[user_id])
encryption.create_data_key("aws", opts)

…
"body": {
"encrypt": {
"keyId": 609,
}
}
…
…
"body": {
"encrypt": {
"keyId": "/user_id",
}
}
…
{
_id: 584,
user_id: 23,
date: Date("6/8/2019"),
title: "My first entry",
body: "Dear diary, … "
}

def delete_user_data(db, user_id):
db.keyvault.delete_one({ "keyAltNames": user_id })

def get_follower_posts(db, user):
cursor = db.posts.find_many(
{ "user_id": { "$in": user["followers"] } },
limit = 20,
sort = { "date": DESCENDING }
)
return list(cursor)

def list_follower_posts(db, user):
cursor = db.posts.find_many(
{ "user_id": { "$in": user["followers"] } },
limit = 20,
sort = { "date": DESCENDING },
projection = { "body": False }
)
return list(cursor)

(used by journalists and teens)

EAST
DICTATORLAND
Users
Global Shards

EAST
DICTATORLAND
{ _id: 1, body: BinData(6, "A81…") }
{ _id: 2, body: BinData(6, "017…") }
{ _id: 3, body: BinData(6, "5E1…") }
…

MongoDB .local Houston 2019: Using Client Side Encryption in MongoDB 4.2

Recommended

More Related Content

What's hot (19)

Similar to MongoDB .local Houston 2019: Using Client Side Encryption in MongoDB 4.2 (20)

More from MongoDB (20)

Recently uploaded (20)

MongoDB .local Houston 2019: Using Client Side Encryption in MongoDB 4.2