Tabletop exercises are a great way to help prepare your incident response plan. More than that, these exercises can help push information technology (IT)/operational technology (OT) convergence forward because they force the two sides of the coin (plus the C-suite) to have conversations about securing their industrial environments.
For more context on tabletop exercises and IT/OT convergence, we spoke with Dino Busalachi, principal partner and co-founder of Velta Technology, a system integrator that works with OT/industrial control system (ICS) security. To listen to the whole interview, you can find it on our ICS Pulse Podcast page, and you can find the first installment of this article series here.
The following has been edited for clarity
ICS Pulse: How do tabletop exercises play into IT/OT convergence?
Dino Busalachi: On the convergence side, I think IT can help vet the tools. They have the ability to policy and govern the roles that they participate in. Sometimes you’ll hear that I’m a firm believer in if IT was going to have a significant role in running a plant, then a CIO would become the plant manager. We’re working with the OT guys.
The problem with that is a mid-tier manufacturer might have a dozen plants, 30 plants, 40 plants. How many people do you think that takes in order to grasp the amount of assets that you have scattered throughout those facilities? It’s a bigger task than any of them realize right now, in my view. On IT/OT convergence, I prefer the method of taking IT-skilled individuals that know networking well, know intrusion prevention systems, intrusion detection systems that they’re familiar with very well, and move them into the OT organization.
They don’t work for the CIO anymore. They work for the business unit. They are the reason why you’re in business, to work with that organization and work with their partners, sit down with the Rockwell teams and the Siemens teams and the Schneider Electric teams and GE and Honeywell and all their SIs (system integrators) and OEMs (original equipment manufacturers) when it comes to trying to figure this out. You can’t sit over there and work with a Microsoft entirely to figure this out, or a Cisco entirely to figure this out, per se, because they don’t necessarily have the skillsets to work on that side of the fence either. You have to start working with those automation technology partners and building relationships with them.
If I’m the CIO of a large manufacturing plant and 70% of my automation technologies are Rockwell Automation, I’d want to have a relationship with Rockwell. I’d want to know about our services support contracts. I’d want to be able to meet with the account management teams that are responsible for servicing us, the distributors, the OEMs. I’d want to build a relationship with them. Part of a tabletop exercise should be that.
I’ve asked that question of CIOs in the past, and it doesn’t always go over very well. How can you take the role of securing these if you don’t even know who to call if you have a problem with ControlLogix’s PLCs in an exploit with zero data that just came out yesterday, and you’re asking Rockwell what they are doing about it? Who are you going to call? Because the OT guys aren’t looking for it. They’re not sitting there paying attention to those bulletins as they come out.
It’s something as simple as taking backups and having backups, but then being able to say, “OK, here we are going to do incident response. I want to see if I’ve got the backups. If we have them, will they work to restore and get back up and running? Do I do part of the incident response? What are my options here? Do I have good backups?” Some of these control systems have been sitting out there for a dozen years or 20 years. Trying to find everything you need to build a machine from the ground back up could take you a long time.
How do you prepare for that? Why are you preparing? Are you practicing the same amount of due diligence for your plant floor as you do your enterprise? They have backups. They’ve got disaster recovery sites. It’s not like you can disaster recover your plant, per se. If you lose a plant, it’s down. You just can’t say, “Well, I’m just going to make my stuff somewhere else.” However, there are things that you can do with defensible architectures, with backups, with continuity planning. There’s things that you could do to be prepared to get back up and running as quickly as possible. Otherwise, you could be down for months.
ICSP: So why do many businesses find themselves unprepared when an attack hits?
Busalachi: I think a lot of them rely on cybersecurity insurance from that risk factor. We’ve seen what’s been going on with that with the recent announcement of what went on with Mondelez and their insurance carriers. One insurer came out a couple months ago wanting to put the war exclusions in there going to next year. So they can say, “Look, we’re not going to defend you if you get attacked, and it’s been tied to a nation-state.”
We’re seeing insurance companies being dropped. We’re finding that their policies are going up. The insurance companies have underfunded what they’ve been selling. There are some groups out there that do a lot of homework on this. I think you’re going to see the insurance industry get very smart about how they’re going to assign this if you’re even going to get into this business, if you want cybersecurity insurance. Organizations rely on that until you need it, and all of a sudden, you’re not getting what you want because they’re not paying for business interruption or liability and property damage or anything along those lines that can happen when control systems get out of control.
ICSP: Is there a role for penetration testing when you’re talking about ICS or OT cybersecurity?
Busalachi: Yes, but it has to be passive in nature. These intrusion detection systems that are passive are a better way to start versus trying to pierce your way in from the outside and trying to get all the way down to the filler on my packaging line and trying to be disruptive to it. Even though you may not recognize the fact that you’re not trying to be harmful to that machine, the fact that you’re probing and scanning and asking questions into that environment can be disruptive to my filler running my packaging line. If it’s not running, it’s costing me $80 grand an hour in downtime. The pen testing needs to be inside out, and it needs to be passive is how you start.
You want to do it while the environment’s running. It doesn’t do any good to do a pen test because it’s been scheduled and planned. The plant’s like, “Well, yeah, you can do it while we’re shut down for the next two weeks because they’ve got all this other work to do. You can do your pen testing then.” What’s the value in that? To check a box and say, “Oh, we got a pen test done.” You got PLCs out there that might be trying to get on the internet, but you don’t know that because they’re offline right now. They’re not running.
You may not be able to see some of those control systems down just because of the architecture of that environment. It takes a lot of knowledge to be able to go through a plant. They may have 100 panels out, and that’s 100 different points of interest that are endpoints that you need to figure out what’s in there. If you’re not seeing any of that stuff, then you’re missing out on huge chunks of your environment. They’re just invisible to you.
Do you have experience and expertise with the topics mentioned in this article? You should consider contributing content to our CFE Media editorial team and getting the recognition you and your company deserve. Click here to start this process.
