One of the key selling points for data access systems with a huge single namespace is their superior suitability for data aggregation. Once data has been aggregated, it is very valuable, because it allows companies to see the big picture and make important strategic decisions with less risk.

The (Black) Market for Big Data

This also makes systems providing access to such data a desirable target for criminal organizations. Such organizations have been able to develop this interest because old security measures like passwords and MD5 hash algorithms have become very easy to crack with services like AWS. For example, a collision attack against the MD5 algorithm just takes 10 hours and costs only 65 cents on a GPU instance using open source software.

In the U.S. alone, more than 1,000 companies legally offer aggregated personal data, with some claiming more than 2,000 data points for every person living in the United States. These companies sell their data quite aggressively, making it easy for a criminal organization to steal identities through social engineering or using spearphishing to gain access to data centers managing critical information. That file system’s FUSE client running on the CIO’s computer in reality could be running on a yacht at the antipodes of Silicon Valley, halfway between Madagascar and Port-aux-Français. A mai-tai-sipping pirate could be stealing the latest design of a missile to sell it to a competitor, deleting sequencing data from an expensive experiment, or deleting the positioning parameters for a deep sea drilling ship’s positioning system when a bribe is not paid.

In addition to the large legal market for aggregated personal data, there is also a vibrant underground black market for data stolen in security breaches. Both markets enjoy brisk business and the issue is not whether a data access system is penetrated, but when. The best response is to contain the damage and make it hard for the intruders to move around, hoping they will attack a competitor’s easier target.

Making Big Data Access Secure

We made this sound scary, because in IT, a little bit of paranoia is healthy. The important lesson is that the organization has to be vigilant and always ready to take action by creating a culture of security. In addition to firewalls and network traffic analysis, the data access system has to be very secure, because it is a big target.

Aureum has five levels of security measures, which can be chosen as needed, but we recommend that you implement the highest level:

1. No security other than login
2. Public key infrastructure (PKI) and client authentication
3. Integrated Kerberos 5 authentication
4. Command integrity through cryptographic signing
5. Full confidentiality through encryption of all traffic on the wire

It’s like security for your house: the lowest level is just locking your front door, while at the higher level you also lock all doors inside the house. When a burglar cracks the front door, he can’t move around freely and the police are warned by your alarm system.

Any personal data you store should also be encrypted at the application level. Typically, database management systems have encryption at the field level. For files, if an application does not provide encryption, you can use a utility like Pretty Good Privacy (PGP), which has been widely used since 1991 and is very mature.

Almost all storage media sold today offers self-encryption (SED) in hardware. It is extremely difficult to crack, but it does not give you the protection you get with Aureum, because once the drive is unlocked, a hacker who has penetrated the system has access to all data. But some applications require SED, so you should enable it if you can, because it gives you two important features:

1. If your data center is physically cracked and the drives are removed, with SED activated they become bricks.
2. When a drive is decommissioned, you can crypto-erase it in an instant.

Authentication, integrity checking, confidentiality: you cannot live without them.