As the number of hacks and ransomware attacks continue to increase, so has the pressure felt by cybersecurity teams tasked with helping businesses and other organizations remediate events capable of causing catastrophic damage to businesses.
When an MSSP is called into to help remediate a ransomware attack, it’s not unheard of for security pros to work up to 24-hour shifts, days at a time, until the remediation is complete. And criminals don’t take holidays or vacations either, leading your cyber staff to possibly miss big events and valuable time with family and friends. One MSSP executive lamented earlier this year that members of the incident response team only last about 18 months before they ask to be taken off the team—or worse, they just leave the company. That means having to find and train more cyber staff, no easy task in a market already in need of more cyber-skilled professionals. It’s a vicious circle, but there are ways to keep your cyber staff, customers, and bottom line all happy.
We asked two cybersecurity leaders and members of the CompTIA ISAO to share their tips on how to keep your security teams satisfied and successful. Here’s what they had to say:
Ask for Help When It’s Needed
Billable hours for cybersecurity staff—especially incident response teams—can be a bit unpredictable. You don’t know when a prospect will come to you asking for help and you don’t know how much resources will be needed, nor for how long. You could have stretches with no cyber incidents, then get two or more on top of each other. With the latter, it behooves MSSPs and other security companies to have access to a bench of trusted external resources when needed. Find a partner, or partners, to share cybersecurity pros with and balance out the feast or famine periods.
“Ransomware can require as little as one engineer to as many as 20 or more all at once. The unpredictability of work, stacking of many cases at once and other factors still require stretching all involved," said Kevin McDonald, COO and CISO at Alvaka Networks and member of the CompTIA ISAO’s SME Champions Council. “With ransomware, you cannot delay, spread out, or cross assign staff to do the work. They need to be available and ready at a moment’s notice and often for weeks straight. We are also now using highly trusted and closely vetted partners to stretch as we hire more staff. This, along with a referral pool, help us to rarely say no,” McDonald said.
Find a Trusted Partner
Asking for help is one thing but finding a trusted partner you can count on is quite another. Accomplishing that is critical to fill gaps in your security coverage. In the best of cases, your partnership is invisible to your customer, who still views you as their primary protector. Find that partner and you don’t have to worry about being all things to all people, said Eric Weast, president of ECW Network & IT Solutions, Deerfield Beach, Fla., and member of the CompTIA ISAO’s Executive Advisory Council.
“We know we’re not great at PEN testing and incident response. But we don’t want to be in that market either,” Weast said.
ECW offloads that function to other companies, allowing the company—and its cyber staff—to focus on its core competencies. That model keeps employees engaged and not stressed about failing a customer for something they’re not skilled at. “Understanding and doing are two different animals that people too often don’t separate themselves from. Ultimately, we know that we cannot take these things on alone,” Weast said.
Weast noted one recent example that occurred after Microsoft released an Exchange Vulnerability in March. One of ECW’s tech partners, Huntress Labs, detected a web hook caused by the vulnerability in one ECW customer, and let the solution provider know—around midnight.
“We have a mature 24x7 response capability, so while I was asleep, one of our engineers took the call and immediately responded,” Weast said. “He got on a late-night conference call with Huntress and Microsoft. They helped us remediate the immediate threat and restore systems. This was a heavy compliance customer, so there was quite a bit on the line. I called the customer early the next morning and explained that we thought our rapid triage stopped the attack during the night, but in these cases, running a detailed analysis and incident response was needed; we partnered with one of our valued partners, Dark Cubed, to work though those details over the following 24 to 48 hours.”
Share Your Successes with Your Team
While no one would be thrilled to hear a web hook had been detected, the client appreciated the quick action taken by the MSP and its partners, especially when the company later discovered several other branches had been compromised and were not quickly remediated by other solution providers.
“We told them that by responding quickly and remediating we took what could have been a very bad situation and turned it into a much better one. I’m sure that if we would have waited another few days, there would have been an exploit detonated or data exfiltrated.”
Weast shared that success with his team, news that was very warmly received by those that worked the long and late hours.
“It does reduce agita of the staff. Whether it’s a server warranty or firewall support or cloud provider, we’ve always wanted to have extra support and we’ve built good relationships to do that. It reduces our team’s anxiety because they know we’re not combatting a security problem alone,” Weast said. “We know we have good help and extended capabilities if we need to tap into.”
Recognize that Cyber Incidents Impact Everyone
A cyber incident response can tax the entire company: sales, accounting, legal, executives, engineering, and more. In most cases, everyone plays a role, and everyone chips in to help solve the problem. It’s important know when your whole staff might need some breaks or incentives, not just the technicians on the front line.
“It is a right-now proposition that frequently involves midnight calls, holidays, and requires teams of people to respond,” McDonald said. “Even if a deal falls through, it has interrupted the lives of several people before that decision has been reached. Once a mid-size or enterprise deal is reached, it requires the non-stop efforts of engineers, weekend and night support from security consultants, sales, project management and executives. The engineers and the project managers definitely take the brunt, but you have to be on compliance, legal and executive calls and watch how everyone is handling it.”
If Necessary, Just Say ‘No’
If a business comes to you and says they’ve had a cyber incident and they need help, but you’re already overwhelmed with activity and don’t have the resources, it’s OK to pass on the business. That’s not something you’d want to do to an existing client, but for a company you don’t know, the ramp-up and information-gathering process could only further tax an already stretched staff. To help save face, you could always recommend the prospect to another security company and hope that both companies remember you played matchmaker and return the favor someday, according to Weast.
“It’s difficult to look at someone who legitimately is seeing their business crippled. You feel bad for them. When people are in dire circumstances, you naturally want to help them out of it,” Weast said.
In many cases, a victim of a cyber incident got there because they didn’t take their security seriously enough and remediating the current problem may not change that either, warned Weast.
“They think that what you do will fix it forever. You cleaned up ransomware, so they’ll never get it again. Security is not a solved problem,” he said. “I resided myself to the fact that we can’t help everybody. We can’t fix the world. They only want that fire to go out so they can go about their day. They’re not interested in the next phase of conversation. Those are people we can’t help.”
Remember the Highs Outweigh the Lows
While ransomware is emotionally and physically exhausting, successfully completing a project often leaves security pros proud and euphoric. It’s a great feeling not necessarily present after a traditional IT job.
“We know that we are helping a business return to a livelihood that was stolen from it,” said McDonald. “We value always being there for those who need us and being able to improve the lives of those who are either already clients or about to become one. It is incredibly rare that one of our current clients gets hit with ransomware. Whether they are a 20-server company or a thousand, we treat every victim as though they are already with us. These are values we seek in our entire company and particularly our incident responders, ransomware engineers and support staff. For those that have decided to take on this challenge, they and everyone in the company involved feel a great sense of personal reward.”
Training Investments Pay Big Dividends
As Weast mentioned, security is not a ‘solved problem.’ As a result, it behooves security companies—any tech company, really—to continually invest in cyber-related training. Employees like to know that the company values their skills and wants them to advance in their careers. Company-paid education and certifications go a long way to keep people happy.
“We have two rotations, one for people to be on-call for incidents and one for training,” said Weast. “Every year we look back and say here are the security products we use, what can be improved. We also trust our team and listen to their feedback on what they’re seeing in the market.”
Be Honest, Transparent with Employees
There’s no easy fix on the horizon and ransomware and other cyber-attacks aren’t going away. Keeping your security staff going—especially during difficult moments—can be a challenge. It’s always best to be open and honest with your teams, let them know what’s expected and see who responds to the challenge.
“The ransomware field is dirty, stressful and a brutally difficult business,” McDonald said. “Having enough help, allowing for downtime between gigs and saying no to the next rescue when staff needs a break are really the best ways to deal with burnout issues. Being able to pass on the revenue and frankly saying no to the company asking for help is never easy, but consideration of staff and always doing a stellar job are critical to long term success. Clients and employees deserve no less.”
Keep Calm and Carry On
Many security and business leaders need to remember that they’re also professional managers. How they behave and act under pressure can have a big impact on a team.
“If you’re freaking out internally, you could be causing the burnout yourself. If you keep yelling at employees in high-pressure situations, people are going to quit. It’s that simple,” said Weast.
Regular communication with everyone helps managers have a better understanding of what’s going with their staff and how they might respond to minimize any potential problems before they escalate.
“Talk to me. What’s on your mind. We always tell people criticism of the company is OK as long as they offer constructive criticism. We’re willing to drop the veil and talk about anything,” Weast said. “If someone is working hard, sometimes it’s not about money and training, they just need time off to feel like a person again. Knowing that and providing that can make a big difference.”
Add CompTIA to your favorite RSS reader
