Erasure Coding (EC 14+4) for Data Durability
At AnonDrop, we are committed to the security and durability of your files. We use a powerful technique called Erasure Coding (EC) with a 14+4 configuration to ensure your data is safe from hardware failures. This page explains how it works.
1. What is Erasure Coding?
Erasure Coding is a method of data protection where a file is broken into fragments, and redundant "parity" fragments are created. These fragments are then distributed across multiple storage locations. The original file can be reconstructed even if some of the fragments are lost.
Our 14+4 configuration means we split your encrypted file into 14 data fragments and generate 4 additional parity fragments. This gives us a total of 18 fragments, which are stored on different hard drives. We can recover the original file from any 14 of these 18 fragments, meaning up to 4 hard drives can fail simultaneously without any data loss.
2. Encoding a File
First, your file is encrypted on your device. Then, on our servers, the encrypted file is encoded. The following Python code demonstrates the principle of encoding data into 14+4 fragments using the pyeclib
library.
from pyeclib.ec_iface import ECDriver
def encode_file(file_path):
ec_driver = ECDriver(k=14, m=4, ec_type='liberasurecode_rs_vand')
with open(file_path, 'rb') as file:
file_data = file.read()
fragments = ec_driver.encode(file_data)
# In a real system, these fragments would be sent to 18 different storage nodes.
# For this example, we'll just return them.
return fragments
3. Decoding a File (Reconstruction)
When you request a file, we retrieve the fragments from our storage nodes. Even if some are missing, we can reconstruct the original data as long as we have at least 14 fragments.
from pyeclib.ec_iface import ECDriver
def decode_file(fragments):
ec_driver = ECDriver(k=14, m=4, ec_type='liberasurecode_rs_vand')
# Simulate losing 4 fragments
available_fragments = fragments[:14]
decoded_data = ec_driver.decode(available_fragments)
return decoded_data
4. Security Implications
Erasure coding is primarily for data durability, but it also enhances security. Since your file is encrypted *before* being fragmented and distributed, an attacker would need to compromise at least 14 different physical drives to get enough data to even attempt to break the encryption. This makes unauthorized access extremely difficult.