It happened. Over the 2020 Thanksgiving weekend, we saw a customer reducing consumption and deleting some data. At first, everything on the customer side seemed fine. The customer had just renewed a week prior and we encouraged them to right-size their deployment and leverage our consumption model.
But something seemed a little out of place — our internal security and monitoring system kept alerting us about this particular deployment.
We tried to contact the customer and got an email response confirming they were simply right-sizing. We again contacted the customer after the weekend over the phone and got the same response. But something still didn’t seem to connect.
We were informed Tuesday that all the while we were actually communicating with a hacker who had taken over the customer’s identity system, and hence also the email and phone systems. The hacker had created a new backup admin, deleted the old one, and was deleting all the virtual machine (VM) backups and associated infrastructure in the Druva system.
But thankfully, the customer and hacker did not know a little secret about Druva. The customer environment was fully recovered due to a secret feature we have been testing and have kept under wraps until now. This powerful feature very simply allows customers to easily and quickly roll back unauthorized or accidental deletion activity.
Another secret feature that we have been testing and have kept under wraps will be unveiled at the Druva Cyber Resiliency Summit. Stay tuned.
Backup Software and Ransomware
Almost every backup software claims to protect against ransomware — and yet here we are.
- Installing yet another software (or even hardware) on an already compromised network is no good.
- Air-gapping data and immutability are not enough. The backup application, and underlying infrastructure, could itself be compromised.
- The 3-2-1 rule (three copies of everything, two off the main server, and one off-site) — was good when customers centralized into one data center. But now, three copies of every app in every location has created a major security (and storage) nightmare.
The problem is not about the software itself, but the fundamental approach. We need to think about the overall security posture. Let’s break down the problem — key fundamental flaws in most available backup software — and possible solutions.
Zero Trust
There is a lot of marketing buzz around elements of Zero Trust, and most of it doesn’t make any sense. Zero Trust ultimately boils down to making sure that different layers of security work independently — they need to overlap and create a sufficient air gap, but a security compromise at one layer should not be able to affect the other.
In many ways, the layers always assume that the layer above is already compromised, and to simply trust nothing. Let’s take an example:
For backup, these layers are:
- Data / storage
- Processing / tasks / configuration / logic
- Authentication / IDP / access
- Data export / APIs
So, if someone breaks into the IDP system (which happened in the Thanksgiving example), they still should be blocked from firing off a bunch of delete commands. Even if they somehow succeed — the eventual data deleted (storage) needs to somehow protect itself.
Most software simply stops at data immutability but does nothing to avoid the application being compromised, and as a result, there is a risk configuration and related infrastructure (proxies, agents, server entities) can be deleted — making the recovery process very painful.
Blast Radius
Another interesting point to understand is blast radius — which means how large the collateral damage can be when bad things happen. For example, if a developer manages to check-in malicious code, how do you actually make sure that the seemingly minor software update does not end up becoming a ransomware incident?
What checks and balances does your provider employ within the software to:
- Contain such issues (layered security model)?
- Provide adequate sophistication to avoid this being exploited large-scale across customers?
With Druva, these threats are very well contained by limiting the scope and impact of every feature, every engineer, and deployment.
Security Operations
Eventually, everything comes down to how well you operationalize the security around software. Regardless of what backup software you deploy, the key question is: who in the organization is responsible for the following tasks?
- Authorization and access governance
- Monitoring logs
- Penetration testing
- Security patches
The fundamental difference between buying a software or a service comes down to air-gapping operations (even more than the data). Going back to the Zero Trust principles — the environment and team responsible for your primary environment and its security, isn’t probably the right choice for the secondary/backup as well. And even if they are, security isn’t something most backup teams are good at anyways. You need a SaaS solution, not a software — just like you did for email, CRM, IDP and many other enterprise software stacks.
Accelerated Recovery
When you get compromised (and it’s when, not if) the most critical factor will be how fast your organization can recover. Here are some key considerations for you to think about:
- Do you have well-defined SLAs on recovery by tiers of applications?
- Is your runbook well defined?
- Does your backup and recovery solution integrate with security (EDR, SIEM, SOAR) products?
- Do you have capabilities and/or APIs to perform bulk recovery at scale?
At Druva, we thought hard about these problems and have introduced an Accelerated Ransomware Recovery module, which solves critical gaps and helps reduce data loss via intelligent automation and orchestration. It’s unmatched in our industry and brings customers:
- Access Insights to understand location and identity for all access attempts
- Anomaly detection and alerts for data entropy
- Quick quarantine for infected systems and snapshots
- Recovery Scans for known malware and customer-provided indicators of compromise
- Curated Recovery to automatically recover the most recent clean version of every file
Responsibility and Ethics
Ransomware attacks are impacting lives across the globe. Almost 50% of all hospitals were impacted in the last several years. With so much marketing and misleading information, it’s hyper critical vendors come clean with what they can and cannot do.
Most backup software/hardware solutions are ransomware themselves. They ask five years and payment upfront for something they may not be able to resolve. Almost every vendor claims to solve for ransomware, and yet the rate of ransomware attacks is only accelerating.
I implore all businesses to be critical of their vendors, ask the hard questions, and hold them accountable about their capabilities versus their marketing speak.
Druva’s Cyber Resilience Virtual Summit — and a Shameless Plug
To learn more about how a data protection solution can help your organization strengthen its data resilience, join us at the Druva Cyber Resilience Virtual Summit on October 13.
You will have the opportunity to hear from thought leaders and peers about how organizations are fighting back against increasingly sophisticated cyberattacks, and will leave with actionable information on emerging best practices that span data protection and recovery, as well as how to integrate modern security and backup technologies.