blindfold module

Python library for working with encrypted data within nilDB queries and replies.

Bases: dict

Data structure for representing cluster configuration information that at minimum specifies the number of nodes in a cluster (but that may contain other information about cluster nodes).

Parameters:: configuration (dict) – Cluster configuration.

A configuration must satisfy independent requirements that govern its type and what keys and values it must have. The constructor ensures that the provided cluster configuration is valid and creates an instance of the class.

>>> Cluster([{}, {}, {}])
Traceback (most recent call last):
  ...
TypeError: cluster configuration must be a dictionary
>>> Cluster({})
Traceback (most recent call last):
  ...
ValueError: cluster configuration must specify nodes
>>> Cluster({'nodes': 123})
Traceback (most recent call last):
  ...
TypeError: cluster configuration node specification must be a sequence
>>> Cluster({'nodes': []})
Traceback (most recent call last):
  ...
ValueError: cluster configuration must contain at least one node

This function does not check the supplied arguments against requirements for particular key types. Those checks are performed within the methods of specific key classes.

class Operations[source]

Bases: dict

Data structure for representing a specification identifying what operations on ciphertexts a key supports.

Parameters:: specification (dict) – Operations specification.

A specification must satisfy independent requirements that govern its type and what keys and values it must have. The constructor ensures that the provided operations specification is valid and creates an instance of the class.

>>> Operations([])
Traceback (most recent call last):
  ...
TypeError: operations specification must be a dictionary
>>> Operations({'foo': True})
Traceback (most recent call last):
  ...
ValueError: permitted operations are limited to store, match, and sum
>>> Operations({'store': 123})
Traceback (most recent call last):
  ...
TypeError: operations specification values must be boolean
>>> Operations({'store': True, 'sum': True})
Traceback (most recent call last):
  ...
ValueError: operations specification must designate exactly one operation

This function does not check the supplied arguments against requirements for particular key types. Those checks are performed within the methods of specific key classes.

class SecretKey[source]

Bases: dict

Data structure for representing all categories of secret key instances. Instantiation must be performed using the generate method.

>>> SecretKey()
Traceback (most recent call last):
  ...
RuntimeError: keys must be instantiated using the generate method

static generate(cluster, operations, threshold=None, seed=None)[source]

Return a secret key built according to what is specified in the supplied cluster configuration, operations specification, and other parameters.

Parameters:

cluster (dict) – Cluster configuration for this key.
operations (dict) – Specification of supported operations on ciphertexts.
threshold (Optional[int]) – Minimum number of parties required to decrypt a ciphertext.
seed (Union[bytes, bytearray, str, None]) – Seed from which to deterministically derive cryptographic material.

Return type:

SecretKey

The supplied arguments determine which encryption protocol is used when encrypting ciphertexts with this key.

>>> sk = SecretKey.generate({'nodes': [{}]}, {'sum': True})
>>> isinstance(sk, SecretKey)
True

Supplying an invalid combination of configurations and/or parameters raises a corresponding exception.

>>> SecretKey.generate({'nodes': [{}]}, {'sum': True}, seed={})
Traceback (most recent call last):
  ...
TypeError: seed must be a bytes-like object or a string
>>> SecretKey.generate({'nodes': [{}]}, {'sum': True}, threshold='abc')
Traceback (most recent call last):
  ...
TypeError: threshold must be an integer
>>> SecretKey.generate({'nodes': [{}, {}]}, {'match': True}, threshold=1)
Traceback (most recent call last):
  ...
ValueError: thresholds are only supported for the store and sum operations
>>> SecretKey.generate({'nodes': [{}]}, {'sum': True}, threshold=1)
Traceback (most recent call last):
  ...
ValueError: thresholds are only supported for multiple-node clusters
>>> SecretKey.generate({'nodes': [{}, {}]}, {'sum': True}, threshold=-1)
Traceback (most recent call last):
  ...
ValueError: threshold must be a positive integer not larger than the cluster size
>>> SecretKey.generate({'nodes': [{}, {}]}, {'sum': True}, threshold=3)
Traceback (most recent call last):
  ...
ValueError: threshold must be a positive integer not larger than the cluster size
>>> SecretKey.generate({'nodes': [{}]}, {'sum': True}, seed=bytes([123]))
Traceback (most recent call last):
  ...
ValueError: seed-based ... summation-compatible ... not supported for single-node ...

dump()[source]

Return a JSON-compatible dictionary representation of this key instance. This method complements the load method.

Return type:: dict

static load(dictionary)[source]

Return an instance built from a JSON-compatible dictionary representation.

Parameters:: dictionary (dict) – Dictionary representation of a secret key.
Return type:: SecretKey

This method complements the dump method and also makes it possible to work with JSON representations of keys.

>>> sk = SecretKey.generate({'nodes': [{}]}, {'store': True})
>>> import json
>>> sk_json = json.dumps(sk.dump())
>>> sk == SecretKey.load(json.loads(sk_json))
True

Any attempt to supply an invalid input raises an exception.

>>> SecretKey.load('abc')
Traceback (most recent call last):
  ...
TypeError: dictionary expected

class ClusterKey[source]

Bases: dict

Data structure for representing all categories of cluster key instances. Instantiation must be performed using the generate method.

>>> ClusterKey()
Traceback (most recent call last):
  ...
RuntimeError: keys must be instantiated using the generate method

static generate(cluster, operations, threshold=None)[source]

Return a cluster key built according to what is specified in the supplied cluster configuration and operations specification.

Parameters:

cluster (dict) – Cluster configuration for this key.
operations (dict) – Specification of supported operations on ciphertexts.
threshold (Optional[int]) – Minimum number of parties required to decrypt a ciphertext.

Return type:

ClusterKey

The supplied arguments determine which encryption protocol is used when encrypting ciphertexts with this key.

>>> ck = ClusterKey.generate({'nodes': [{}, {}, {}]}, {'sum': True})
>>> isinstance(ck, ClusterKey)
True

Cluster keys can only be created for clusters that have two or more nodes and can only enable encryption for storage and summation compatibility.

>>> ClusterKey.generate({'nodes': [{}]}, {'store': True})
Traceback (most recent call last):
  ...
ValueError: cluster configuration must contain at least two nodes
>>> ClusterKey.generate({'nodes': [{}, {}, {}]}, {'match': True})
Traceback (most recent call last):
  ...
ValueError: cluster keys cannot support matching-compatible encryption

dump()[source]

Return a JSON-compatible dictionary representation of this key instance. This method complements the load method.

Return type:: dict

static load(dictionary)[source]

Return an instance built from a JSON-compatible dictionary representation.

Parameters:: dictionary (dict) – Dictionary representation of a cluster key.
Return type:: ClusterKey

This method complements the dump method and also makes it possible to work with JSON representations of keys.

>>> cluster = {'nodes': [{}, {}, {}]}
>>> ck = ClusterKey.generate(cluster, {'sum': True}, threshold=2)
>>> import json
>>> ck_json = json.dumps(ck.dump())
>>> ck == ClusterKey.load(json.loads(ck_json))
True

Any attempt to supply an invalid input raises an exception.

>>> ClusterKey.load('abc')
Traceback (most recent call last):
  ...
TypeError: dictionary expected
>>> ClusterKey.load({
...    'cluster': {'nodes': [{}, {}]},
...    'operations': {'store': True},
...    'material': 'abc'
... })
Traceback (most recent call last):
  ...
ValueError: cluster keys cannot contain key material

class PublicKey[source]

Bases: dict

Data structure for representing all categories of public key instances. Instantiation must be performed using the generate method.

>>> PublicKey()
Traceback (most recent call last):
  ...
RuntimeError: keys must be instantiated using the generate method

static generate(secret_key)[source]

Return a public key built according to what is specified in the supplied secret key.

Parameters:: secret_key (SecretKey) – Secret key from which to derive this public key.
Return type:: PublicKey

A public key can only be derived from a compatible secret key.

>>> sk = SecretKey.generate({'nodes': [{}]}, {'sum': True})
>>> isinstance(PublicKey.generate(sk), PublicKey)
True
>>> ck = SecretKey.generate({'nodes': [{}, {}]}, {'sum': True})
>>> PublicKey.generate(ck)
Traceback (most recent call last):
  ...
TypeError: secret key material must be of the correct type
>>> ck = ClusterKey.generate({'nodes': [{}, {}]}, {'sum': True})
>>> PublicKey.generate(ck)
Traceback (most recent call last):
  ...
TypeError: secret key expected

dump()[source]

Return a JSON-compatible dictionary representation of this key instance. This method complements the load method.

Return type:: dict

static load(dictionary)[source]

Return an instance built from a JSON-compatible dictionary representation.

Parameters:: dictionary (dict) – Dictionary representation of a public key.
Return type:: PublicKey

This method complements the dump method and also makes it possible to work with JSON representations of keys.

>>> sk = SecretKey.generate({'nodes': [{}]}, {'sum': True})
>>> pk = PublicKey.generate(sk)
>>> import json
>>> pk_json = json.dumps(pk.dump())
>>> pk == PublicKey.load(pk.dump())
True

Any attempt to supply an invalid input raises an exception.

>>> PublicKey.load('abc')
Traceback (most recent call last):
  ...
TypeError: dictionary expected
>>> PublicKey.load({
...    'cluster': {'nodes': [{}]},
...    'operations': {'sum': True},
...    'threshold': 3
... })
Traceback (most recent call last):
  ...
ValueError: public keys cannot specify a threshold

encrypt(key, plaintext)[source]

Return the ciphertext obtained by using the supplied key to encrypt the supplied plaintext.

Parameters:

key (Union[SecretKey, ClusterKey, PublicKey]) – Key to use for performing encryption.
plaintext (Union[int, str, bytes, bytearray]) – Plaintext to encrypt.

Return type:

Union[str, Sequence[str], Sequence[int], Sequence[Sequence[int]]]

The supplied key determines which protocol is used to perform the encryption.

>>> sk = SecretKey.generate({'nodes': [{}]}, {'store': True})
>>> len(encrypt(sk, 'abc'))
60
>>> sk = SecretKey.generate({'nodes': [{}]}, {'match': True}, seed='xyz')
>>> encrypt(sk, 'abc')[:70]
'Y3V9Nm4o3F5cTEy+oy3utP19m8XA1eMQ2zFfQiEdGpkE92g4X7eXy4T1yH4u1aBtw0FUs0'
>>> sk = SecretKey.generate({'nodes': [{}]}, {'sum': True})
>>> pk = PublicKey.generate(sk)
>>> isinstance(encrypt(pk, 123), str)
True
>>> sk = SecretKey.generate({'nodes': [{}, {}]}, {'store': True})
>>> shares = encrypt(sk, 'abc')
>>> len(shares) == 2 and all(isinstance(share, str) for share in shares)
True
>>> ck = ClusterKey.generate({'nodes': [{}, {}, {}]}, {'sum': True})
>>> shares = encrypt(ck, 123)
>>> len(shares) == 3 and all(isinstance(share, int) for share in shares)
True

When encrypting for a single-node cluster in a summation-compatible way, it is possible to supply the secret key. However, this introduces a performance overhead because a public key must be generated in this case.

>>> sk = SecretKey.generate({'nodes': [{}]}, {'sum': True})
>>> isinstance(encrypt(sk, 123), str)
True

Invocations that involve invalid argument values or types may raise an exception. The type of the key argument is checked. Incompatibilities between the key’s attribute values and the supplied plaintext argument are detected. However, the values associated with those attributes (such as the cluster configuration, the cryptographic material associated with the supplied key, interdependencies between these, and so on) are not checked for validity.

>>> encrypt('abc', 123)
Traceback (most recent call last):
  ...
TypeError: secret key, cluster key, or public key expected
>>> key = SecretKey.generate({'nodes': [{}]}, {'sum': True})
>>> encrypt(key, {})
Traceback (most recent call last):
  ...
TypeError: plaintext must be string, integer, or bytes-like object
>>> encrypt(key, 'abc')
Traceback (most recent call last):
  ...
TypeError: summation-compatible encryption requires a numeric plaintext
>>> encrypt(key, 2 ** 64)
Traceback (most recent call last):
  ...
ValueError: numeric plaintext must be a valid 32-bit signed integer
>>> del key['operations']['sum']
>>> encrypt(key, 123)
Traceback (most recent call last):
  ...
ValueError: cannot encrypt the supplied plaintext using the supplied key

decrypt(key, ciphertext)[source]

Return the plaintext obtained by using the supplied key to decrypt the supplied ciphertext.

Parameters:

key (Union[SecretKey, ClusterKey]) – Key to use for performing decryption.
ciphertext (Union[str, Sequence[str], Sequence[int], Sequence[Sequence[int]]]) – Ciphertext to decrypt.

Return type:

Union[int, str, bytes]

The supplied key determines which protocol is used to perform the decryption.

>>> key = SecretKey.generate({'nodes': [{}, {}]}, {'store': True})
>>> decrypt(key, encrypt(key, 123))
123
>>> key = SecretKey.generate({'nodes': [{}, {}]}, {'store': True})
>>> decrypt(key, encrypt(key, -10))
-10
>>> key = SecretKey.generate({'nodes': [{}, {}]}, {'store': True})
>>> decrypt(key, encrypt(key, bytes([1, 2, 3])))
b'\x01\x02\x03'
>>> key = SecretKey.generate({'nodes': [{}]}, {'store': True})
>>> decrypt(key, encrypt(key, 'abc'))
'abc'
>>> key = SecretKey.generate({'nodes': [{}]}, {'store': True})
>>> decrypt(key, encrypt(key, 123))
123
>>> key = SecretKey.generate({'nodes': [{}, {}]}, {'sum': True})
>>> decrypt(key, encrypt(key, 123))
123
>>> key = SecretKey.generate({'nodes': [{}, {}]}, {'sum': True})
>>> decrypt(key, encrypt(key, -10))
-10
>>> key = SecretKey.generate({'nodes': [{}, {}]}, {'sum': True}, threshold=2)
>>> decrypt(key, encrypt(key, 123))
123
>>> key = SecretKey.generate({'nodes': [{}, {}, {}, {}]}, {'sum': True}, threshold=3)
>>> decrypt(key, encrypt(key, 123)[:-1])
123
>>> key = SecretKey.generate({'nodes': [{}, {}, {}, {}]}, {'sum': True}, threshold=2)
>>> decrypt(key, encrypt(key, 123)[2:])
123
>>> key = SecretKey.generate({'nodes': [{}, {}]}, {'sum': True}, threshold=1)
>>> decrypt(key, encrypt(key, 123)[1:])
123
>>> key = SecretKey.generate({'nodes': [{}, {}]}, {'sum': True}, threshold=2)
>>> decrypt(key, encrypt(key, -10))
-10

A decryption threshold of 1 is permitted in order to accommodate seamlessly scenarios in which it may be useful to replicate plaintext or encrypted data across nodes (such as for redundancy).

>>> key = SecretKey.generate({'nodes': [{}, {}, {}]}, {'store': True}, threshold=1)
>>> decrypt(key, encrypt(key, 123)[:1])
123
>>> decrypt(key, encrypt(key, 123)[1:2])
123
>>> decrypt(key, encrypt(key, 123)[2:])
123

However, the use of a threshold of 1 incurs the same representation size overheads (compared to simply keeping copies of the same data on different nodes) as the use of larger threshold values. For example, note below that 80 > 68 and that 28 > 8.

>>> key = SecretKey.generate({'nodes': [{}]}, {'store': True})
>>> len(encrypt(key, 123))
68
>>> key = SecretKey.generate({'nodes': [{}, {}, {}]}, {'store': True}, threshold=1)
>>> len(encrypt(key, 123)[0])
80
>>> key = SecretKey.generate({'nodes': [{}, {}, {}]}, {'store': True}, threshold=3)
>>> len(encrypt(key, 123)[0])
80
>>> import base64
>>> len(base64.b64encode(int(123).to_bytes(4, 'little')))
8
>>> key = ClusterKey.generate({'nodes': [{}, {}, {}]}, {'store': True}, threshold=1)
>>> len(encrypt(key, 123)[0])
28
>>> key = ClusterKey.generate({'nodes': [{}, {}, {}]}, {'store': True}, threshold=3)
>>> len(encrypt(key, 123)[0])
28

An exception is raised if a ciphertext cannot be decrypted using the supplied key (e.g., because one or both are malformed or they are incompatible). The type of the``key`` argument is checked. Incompatibilities between the key’s attribute values and the supplied ciphertext argument are detected. However, the values associated with those attributes (such as the cluster configuration, the cryptographic material associated with the supplied key, interdependencies between these, and so on) are not checked for validity.

>>> decrypt('abc', 123)
Traceback (most recent call last):
  ...
TypeError: secret key or cluster key expected
>>> key = SecretKey.generate({'nodes': [{}, {}]}, {'store': True})
>>> decrypt(key, 'abc')
Traceback (most recent call last):
  ...
ValueError: key requires a valid ciphertext from a multiple-node cluster
>>> key = SecretKey.generate({'nodes': [{}]}, {'store': True})
>>> ciphertext = encrypt(key, 'abc')
>>> key['operations'] = {}
>>> decrypt(key, ciphertext)
Traceback (most recent call last):
  ...
ValueError: cannot decrypt the supplied ciphertext using the supplied key
>>> key = SecretKey.generate({'nodes': [{}, {}]}, {'store': True})
>>> key_alt = SecretKey.generate({'nodes': [{}, {}]}, {'store': True})
>>> decrypt(key_alt, encrypt(key, 123))
Traceback (most recent call last):
  ...
ValueError: cannot decrypt the supplied ciphertext using the supplied key

allot(document)[source]

Convert a document that may contain ciphertexts intended for multiple-node clusters into secret shares of that document.

Parameters:: document (Union[bool, int, float, str, list, dict, None]) – Document to convert into secret shares.
Return type:: Sequence[Union[bool, int, float, str, list, dict, None]]

The output consists of a sequence of documents; the number of documents in the sequence is determined by the number of secret shares that appear in ciphertext values found in the document. Shallow copies are created whenever possible.

>>> d = {
...     'id': 0,
...     'age': {'%allot': [1, 2, 3]},
...     'dat': {'loc': {'%allot': [4, 5, 6]}}
... }
>>> for d in allot(d): print(d)
{'id': 0, 'age': {'%share': 1}, 'dat': {'loc': {'%share': 4}}}
{'id': 0, 'age': {'%share': 2}, 'dat': {'loc': {'%share': 5}}}
{'id': 0, 'age': {'%share': 3}, 'dat': {'loc': {'%share': 6}}}

A document with no ciphertexts intended for decentralized clusters is unmodofied; a list containing this document is returned.

>>> allot({'id': 0, 'age': 23})
[{'id': 0, 'age': 23}]

When performing allotment, None is a valid document share leaf value.

>>> d = {
...     'age': {'%allot': [1, 2]},
...     'name': None
... }
>>> for d in allot(d): print(d)
{'age': {'%share': 1}, 'name': None}
{'age': {'%share': 2}, 'name': None}
>>> allot(None)
[None]

Any attempt to convert a document that has an incorrect structure raises an exception.

>>> allot({1, 2, 3})
Traceback (most recent call last):
  ...
TypeError: boolean, integer, float, string, list, dictionary, or None expected
>>> allot({'id': 0, 'age': {'%allot': [1, 2, 3], 'extra': [1, 2, 3]}})
Traceback (most recent call last):
  ...
ValueError: allotment must only have one key
>>> allot({
...     'id': 0,
...     'age': {'%allot': [1, 2, 3]},
...     'dat': {'loc': {'%allot': [4, 5]}}
... })
Traceback (most recent call last):
  ...
ValueError: number of shares in subdocument is not consistent
>>> allot([
...     0,
...     {'%allot': [1, 2, 3]},
...     {'loc': {'%allot': [4, 5]}}
... ])
Traceback (most recent call last):
  ...
ValueError: number of shares in subdocument is not consistent

unify(key, documents, ignore=None)[source]

Combine a sequence of compatible secret shares of a document into one document.

Parameters:

key (Union[SecretKey, ClusterKey]) – Key with which to reconstruct leaf values from secret shares.
documents (Sequence[Union[bool, int, float, str, list, dict]]) – Sequence of document secret shares to combine.
ignore (Optional[Sequence[str]]) – Sequence of dictionary keys to ignore.

Return type:

Union[bool, int, float, str, list, dict, None]

Corresponding plaintexts acting as leaf values are deduplicated and corresponding secret shares acting as leaf values are used to reconstruct plaintexts that appear in the resulting document.

>>> data = {
...     'a': [True, 'v', 12],
...     'b': [False, 'w', 34],
...     'c': [True, 'x', 56],
...     'd': [False, 'y', 78],
...     'e': [True, 'z', 90],
... }
>>> sk = SecretKey.generate({'nodes': [{}, {}, {}]}, {'store': True})
>>> encrypted = {
...     'a': [True, 'v', {'%allot': encrypt(sk, 12)}],
...     'b': [False, 'w', {'%allot': encrypt(sk, 34)}],
...     'c': [True, 'x', {'%allot': encrypt(sk, 56)}],
...     'd': [False, 'y', {'%allot': encrypt(sk, 78)}],
...     'e': [True, 'z', {'%allot': encrypt(sk, 90)}],
... }
>>> shares = allot(encrypted)
>>> decrypted = unify(sk, shares)
>>> data == decrypted
True

It is possible to wrap nested lists of shares to reduce the overhead associated with the {'%allot': ...} and {'%share': ...} wrappers.

>>> data = {
...     'a': [1, [2, 3]],
...     'b': [4, 5, 6],
...     'c': None,
...     'd': 1.23
... }
>>> sk = SecretKey.generate({'nodes': [{}, {}, {}]}, {'store': True})
>>> encrypted = {
...     'a': {'%allot': [encrypt(sk, 1), [encrypt(sk, 2), encrypt(sk, 3)]]},
...     'b': {'%allot': [encrypt(sk, 4), encrypt(sk, 5), encrypt(sk, 6)]},
...     'c': None,
...     'd': 1.23
... }
>>> shares = allot(encrypted)
>>> decrypted = unify(sk, shares)
>>> data == decrypted
True

When performing unification, None is a valid document share leaf value.

>>> unify(sk, [None, None]) == None
True

The ignore parameter specifies which dictionary keys should be ignored during unification. By default, '_created' and '_updated' are ignored.

>>> shares[0]['_created'] = '123'
>>> shares[1]['_created'] = '456'
>>> shares[2]['_created'] = '789'
>>> shares[0]['_updated'] = 'ABC'
>>> shares[1]['_updated'] = 'DEF'
>>> shares[2]['_updated'] = 'GHI'
>>> decrypted = unify(sk, shares)
>>> data == decrypted
True

Unification returns the sole document when a one-document list is supplied.

>>> 123 == unify(sk, [123])
True

Any attempt to supply incompatible document shares raises an exception.

>>> unify('abc', [])
Traceback (most recent call last):
  ...
TypeError: secret key or cluster key expected
>>> unify(sk, 123)
Traceback (most recent call last):
  ...
TypeError: sequence of documents expected
>>> unify(sk, [123, 'abc'])
Traceback (most recent call last):
  ...
TypeError: sequence of compatible document shares expected
>>> unify(sk, [123, 123], 456)
Traceback (most recent call last):
  ...
TypeError: ignored keys must be supplied as a sequence of strings