The Security Assessment Was the Easy Part (Like Catching a Magikarp)

Jori VanAntwerp

CEO and Founder at EmberOT || Web

For over two decades, Jori has enabled industrial and IT organizations to be successful in reducing risk, increasing compliance, and improving their overall security efforts. He has had the pleasure of working with companies such as Gravwell, Dragos, CrowdStrike, FireEye, McAfee, and is now CEO & Founder at EmberOT, a cybersecurity startup focused on making security a reality for critical infrastructure.

A security assessment fills a report with findings the way a Pokedex fills with entries. Neither one wins a fight. Here’s why so many OT programs stall the day after the assessment, and how to build one that actually sticks.

Anyone who played Pokemon knows the trap, even if they never called it that. Catching a Pokemon feels like the accomplishment. The ball clicks shut, the screen says caught, the Pokedex ticks up one. It’s satisfying, it’s measurable, and it’s almost completely beside the point. A box stuffed with level-five Pokemon you never trained loses every battle that matters. The catching was the easy part. The grind, leveling them, evolving them, building a team that can actually take a gym, is the entire game.

OT security has a catching problem. We’re very good at the assessment. We bring in the experts, we scan and interview and map, and we produce a thick report full of findings. The Pokedex fills up. Everyone feels productive and pats each other on the back. And then, in a discouraging number of cases, almost nothing happens. The report goes in a drawer, the findings age, and a year later, the next assessment catches most of the same things again.

The catch was never the hard part. The team-building is. A security assessment is only useful when its findings turn into controls, ownership, prioritization, and a repeatable follow-up process.

Why the Program Stalls After a Security Assessment

This isn’t a story about lazy teams or bad assessors. The stall has a structural cause, and it’s worth naming plainly because it is so common.

In OT, the security decisions are shared, and the people who own the risk are usually not the people who carry the cost. As one industry leader put it, security teams are accountable for cyber risk, but operations and engineering typically carry the cost and the burden of sustaining the controls, and those costs land in tight maintenance budgets tied to reliability (see the original quote from Victor Atkins at 1898 & Co in Industrial Cyber). If a proposed control adds maintenance effort, increases outage risk, creates operational friction, or makes recovery harder, it won’t stick, no matter how compelling the risk story sounds.

Read that again, because it’s the whole problem. A finding is not a fix. A recommendation that operations can’t afford to sustain, in dollars or downtime or risk to the process, is a recommendation that quietly dies. The assessment caught the issue. Nobody could field it. Before a finding becomes a project, it needs an owner, an operational impact review, a realistic maintenance path, and a fallback control if remediation has to wait.

The vulnerability data makes the same point from another angle. In our own analysis of recent disclosures, a large share of advisories pointed to a hardware upgrade as the remedy, and for a small operator, a hardware upgrade is a multi-year capital project, not a remediation. The assessment can flag a finding all day. If the only offered fix is one the environment can’t execute, the finding just sits there, caught and useless, a level-five sitting in the box. Poor little Magikarp.

Build a Team, Not a Pokedex

The trainers who win aren’t the ones with the fullest Pokedex. They’re the ones with a small, well-leveled team built for the fights they actually face. The same is true of an OT security program after a security assessment: the work only matters if it survives contact with operations. 

A few things separate a team from a box of untrained findings:

Co-own the decision from the start

The operators and engineers who carry the cost of a control need to be in the room when it’s chosen, not handed a verdict after the fact. They know the maintenance windows, the edge cases, and the failure modes you won’t find in any report. I’ve said before that operators are engineers and their expertise is invaluable, and nowhere is that more true than in deciding which controls can actually live in the environment long term.

Favor controls that don’t fight the process

The reason passive monitoring has become the backbone of so many OT programs is exactly this. It adds visibility without adding outage risk, without scanning that can knock over fragile devices, without agents on equipment that was never built to run them. A control that respects availability is a control that gets to stay. A control that threatens uptime gets declined the first time it’s inconvenient.

Prioritize the few that move real risk

You cannot, and should not, act on every finding at once. Sort them by what actually matters in your environment, what’s exploitable, reachable, and consequential, and build from there. We laid out a full way to do that in a recent piece on prioritization, and the principle is the same one every good trainer learns: a leveled team of six beats a box of sixty you never trained.

Document and revisit

A control you chose with operations, wrote down with a clear reason, and re-check on a real cadence is a control that holds up when leadership asks and when the next assessment rolls around. The point isn’t to catch the same things again next year. It is to show the team got stronger.

The Report is the Starting Line

The most encouraging part of all this is what it means for the small operator who feels outgunned. You dont need an enormous roster. You do not need to act on every finding in the report. A co-op with three people can run a genuinely strong program if the controls they choose fit their environment and actually get sustained. A small, leveled team that fits how you operate will beat a giant box of untrained findings every single time.

So treat the assessment as what it is. The catch. A good one tells you whats out there and hands you a real starting point. But the badge does not come from the Pokedex. It comes from the team you build afterward, and the discipline to keep leveling it. That part is on you, and it is absolutely within reach.

No noise. Just signal.

~Jori 🤘🔥