News/Trends, Tech/Engineering

Shouldn’t all backups be immutable?

February 7, 2020 W. Curtis Preston, Technical Evangelist

Immutable/i(m)ˈmyo͞odəb(ə)l/, adjective

unchanging over time or unable to be changed.

It should go without saying that all backups should be immutable and protected against ransomware attacks. Bad actors should never be able to attack what should be your last line of defense against malware: your backups. I saw this week that a competitor announced immutability support for some backups. It’s only for backups copied to the cloud, and only if the customer enables the immutability feature on a particular copy. While I congratulate our competitor on adding this functionality to its product, this alas counts as an admission that all of the other backups it creates are not immutable.

It’s all about access

Backups stored on-premises offer several ways for a bad actor to gain access to backup images and delete or encrypt them. This is why on-premises backups are never immutable—no matter which vendor tries to claim that they are. In addition, even if a product supports storing some of its backups on immutable storage, the index of the backups is often still stored on-premises and can still be at risk.

Target deduplication appliances are part of the problem. The default installation of deduplication  appliances uses an NFS or SMB share that can potentially be accessed and attacked by a bad actor. If that target deduplication appliance is replicating its backups to another target deduplication appliance (a very expensive option), both copies can be damaged when the ransomware attack is replicated between appliances. (Some point out that properly implemented systems might not be susceptible to such attacks, at which I point out that most ransomware is stoppable by simply being up to date on your patches—and yet here we are.)

Another common ransomware attack vector is Windows-based backup servers, as they are susceptible to the same ransomware attacks that go after the rest of the data center. The risk is higher if the Windows backup server is in the same physical location as the servers you are protecting. This is not just FUD; Google your favorite backup product and the phrase “encrypted backups.” Such customers have no choice but to pay the ransom.

Finally, it is important to understand that anything is possible with physical access. Remember that many ransomware attacks, if not most, are considered to be insider attacks. Someone with physical access to your data center might be the very person you’re trying to protect your backups from. All that it takes for a ransomware attack is a USB stick and a live boot CD from a Linux distribution to be able to directly access these “immutable” backups stored on disk drives in your data center. Those backups could then be deleted or encrypted, depending on what the bad actor intends to do. Or an attacker can simply steal enough disks to make an array and its backups unusable.

The cloud comes to the rescue

Our competitor knew that it could not have immutable backups stored on a backup server in a data center, so the only way that it could protect your backups from ransomware was to copy them to the cloud. For each backup that you want to be immutable, they will automatically copy it to object storage and tell it that these objects can neither be changed nor deleted until the end of the specified immutability period.

Immutable backups done right

We agree with our competitor that if you want your backups protected against ransomware attacks, you need to send them to the cloud. But the way most companies copy customers’ backups to the cloud is neither scalable nor cost effective. The competition does not incorporate source-side global deduplication to reduce the amount of data that needs to be transferred to and stored in the cloud. This means that your storage can only scale so far, and it will typically cost more than what Druva charges.

While I do not miss the days of performance-tuning tape drives, I do miss the days of a good old air gap. Modern-day backup systems can only approximate a true air gap since they are online. However, I believe that the way that Druva designed its backup system is as close to an air gap as you’re going to find. The following are examples of our design principles.

Encryption in transit, and at rest

Backups should be encrypted before they ever leave your site and encrypted when they are stored. This should be done using a key management system where the vendor never has access to the keys.

Role-based administration

Separate various tasks and assign them to different people so that no one person can destroy all of the data. This can limit the “blast radius” of an attack when a given user ID is compromised. Also, investigate multi-factor authentication to help prevent an outside attack from a bad actor using compromised credentials.

Change in protocols

Store backups in a completely different storage system than what you use on-premises. That is the beauty of using object storage, as it has not yet been the target of ransomware attacks. This is because bad actors typically go the path of least resistance.

Separate data and metadata

Storing data and metadata in separate systems makes it much harder to attack the entire system, especially if the goal of an attack is to steal data.

Change in ownership

Object storage might at some point get attacked by ransomware. This is why it is essential that you use completely different credentials for your cloud backups. Since Druva’s service runs in our account, the customer never sees the credentials needed to access their data outside of our user interface.

Use available security features

Unfortunately, many customers have discovered they had an open S3 bucket or compromised credentials. But S3 offers features so that a given bucket can only be read to or written from a certain process, and Druva automatically enacts those features for the buckets where we store customer data. Even if a bad actor somehow broke every other level of security and was able to gain the access credentials to an S3 bucket that Druva controls, they would not be able to read or write the data because they are not using the appropriate process to do so.

Thanks for proving our point

It’s always nice when a competitor, whether knowingly or not, validates our strategy. Druva has been storing immutable backups in the cloud for several years. It’s good to know that we had the right idea.

Download your copy of the CIO’s guide to cloud-first data protection and learn more about how Druva is providing cloud-native and backup recovery.