4 Key Tips to Secure Your DevOps Process & Protect Customers from Backdoor Malware

By Jared Hrabak, Cybersecurity Engineer, cStor

Maybe you heard about the 2020 SolarWinds debacle where that company unknowingly sent malicious code to more than 18,000 customers while updating their Orion software. If not, I’ll briefly fill you in. According to the U.S. Department of Homeland Security’s advisory on the breach, the threat actors gained access to SolarWinds’ network in September of 2019, were testing malicious injections in October, by February of 2020 the ‘Sunburst’ malicious code was fully injected, and by March, Orion updates were widely distributed, complete with the malicious code.

Customer recipients of the malicious code included the likes of the U.S. Department of Homeland Security themselves, other government agencies, and private entities such as Microsoft, Intel, Cisco, Deloitte and more. YIKES!! I think this definitively refutes the old Hollywood saying that ‘any press is good press,’ huh?

While six months may seem like plenty of time in the email security world to detect, stop and remediate a broad-scale attack, in the world of custom applications, it’s clearly not. That’s why I’m focusing this blog on four key tips to help secure your DevOps processes so you can minimize risk to your company and your customers, and hopefully not end up making the headlines as the next SolarWinds.

Tip #1: Make Security an Inherent Part of DevOps

DevOps requires speed and agility and is far too important to leave out of the security mix. Moreover, DevOps teams have some very unique needs and are often under pressure to release code updates, so it’s not uncommon for engineers to shortcut and sidestep typical security processes to meet deadlines.

According to Forbes, 57% of operations teams don’t follow security best practices, while 67% of cybersecurity pros are demanded to do everything possible *not* to slow business down.

DevOps teams are essentially ‘caught in the crossfire’ between a demanding and relentless release schedule, and trying to follow security best practices that are often perceived to slow them down. You can understand why this kind of culture and mind-shift is necessary. Some organizations, however, are seeing the forest through the trees and are actually re-labeling DevOps to “DevSecOps,” in a deliberate effort to embed security in their processes and culture.

Tip #2: Avoid Common Security Shortcuts Across the Board

Given the stats in tip #1, it’s easy to understand why DevOps teams often sidestep and shortcut common security protocols. I recommend spending some critical thinking time on how to avoid security missteps such as:

• hardcoding credentials;
• pairing auto-scaling servers with secrets management;
• short-cutting privileged access management, or “PAM,”;
• avoiding encryption methodologies; and
• not building security into every phase of your process and culture.

The truth is, security needs to be baked into the process, not bolted on as an afterthought.

Tip #3: Leverage Advanced Application Security Tools

According to a recent Forrester report, “The State of Application Security, 2021,” applications are a top attack vector and researchers are finding more and more security flaws not only in traditional web applications but also in APIs and in open source software (OSS) that is often used to accelerate time to market. Forrester reports that the percent of codebases that contain OSS has doubled in the last five years, and remediation efforts following a breach take over a week longer than single source code.

That may all sound daunting, but fortunately, there’s good news. As the security world evolves, new categories are emerging to answer the call, including Application Security (AppSec), and Application Security Testing (AST). Check them out online and in Gartner’s Application Security Testing Reviews and Ratings released in May 2021, which details some of the leaders including Contrast Security, Snyk, Checkmarx and Veracode.

While adoption of these types of tools and platforms is inconsistent so far, they are likely to emerge as a new baseline standard in DevSecOps, and I’m sure will continue to mature over time. The big message here is that you’re no longer on your own when it comes to application security, and IT can ease the burden by facilitating a tool evaluation based on your unique environment and needs.

Tip #4: It’s Time to Shift-Left

The Shift-Left model means moving your security protocols into the early development phase of your software development life cycle. To my earlier point about baking it in, rather than bolting it on, it’s imperative to deploy pre-release testing tools and embrace the value of early remediation.

Despite the pandemic and the rise of cyber-attacks, application security adoption remains spotty at best. Some industries are expected to be far more aggressive about it than others, such as retail and wholesale. Financial services and the public sector are heavily focused on building security walls to prevent mobile attacks and bot breaches. In my estimation, your industry may not matter. Nearly every company is a technology company because they build applications in order to support their products, services and customers. No one is immune.

It’s time to get your DevSecOps game together!

In the End, Lessons Learned

So what happened with SolarWinds you might ask? Well, a private company that was one of the original Orion victims finally identified the domain used to inject the malware and reverse-engineered it into a ‘kill switch.’ While this in essence ‘stopped the bleeding,’ the unfortunate news is that in June of this year (over a year later!) the malicious code was still affecting customers and their end customers. OUCH!

If you need help working through an evaluation of your DevOps security process and protocols, we’ve got a team of experts ready to dive in and help, so don’t hesitate to drop us a line.