Every business has got assets that can’t be properly secured on their own or assets that need more security than what the rest of the network gets. By placing those assets behind a firewall, an access control list (ACL), or in another VLAN, we can make them more secure. If you want to say that last sentence in one word, you go with “segmentation”. If you want two words, “network segmentation.”

In the small company, the number of devices that need segmentation can be small. They may still be a high percentage of the devices on the network, but it’s still a small number. It’s a small company, after all. If the devices aren’t mobile, segmentation can be as easy as permitting only traffic on ports specified by the vendor and then blocking everything else with the obligatory “deny any any all” rule at the bottom of the ACL. If you have a large number of devices with similar needs, place them in the same VLAN and then apply a single ACL to the whole VLAN. Simple, yes?

Well, yes and no. Often, the network gear in place in a small business is limited in what it can do. If the owner couldn’t afford a switch that can support multiple VLANs or ACLs – or the operating system version that unlocks those features – then you can’t do that easy implementation. The good news is that you might not need to upgrade the entire environment, maybe just one switch that can handle all the sensitive connections and then leave everything else on the lighter-grade switches.

In the medium company, maybe there’s an actual datacenter where all the cool servers go. If so, great! Put a firewall between that datacenter and the rest of the network and start putting rules on it that limit the inbound and outbound traffic. If there’s a business case to have unlimited access, then do so only through a single host, and then log all the activity on that host.

As for the other devices, you’ll be looking at VLANs for the most part to provide that segmentation. The reason comes down to memory available on switching devices for ACL storage. On many devices, it’s limited to where we would not want to have an ACL on every single port. That memory gets exhausted and the switch goes down. Having ports gathered into VLANs and then giving that VLAN an ACL makes much better use of memory on the switches.

The problem now becomes one of defining the address scopes and the routing for those new VLANs, which can be a lot of work even for one VLAN type. Carving out a printer VLAN at 10 different sites is no mean task. What’s more, it means that the IP address management (IPAM) system needs to be kept current with comments for each VLAN type, so that people can understand what’s going on.

And once you start documenting your IPAM, you can’t stop. Or, rather, if you *do* stop, you’ll leave question marks on other network ranges and force someone later on to go back and revisit your work – and that someone might be you!

Large firms may already have VLAN hell and an IPAM with best-effort maintenance. There’s no easy way to put this: you’ll have to clean that up and get processes in place to keep it cleaned up. Network and security teams will have to join in on this effort and it would be best if people on both teams were familiar with the IPAM system being used and what different security ACLs apply to the various VLANs. That means documentation, meetings, documentation of what was said in meetings, meetings to work out issues with the documentation, and so on.

Sorry about all the work that goes with this… but if you want proper segmentation, there needs to be both proper documentation and proper maintenance of the scheme to keep it relevant over time. The small company has the lightest load for documentation, but that might be offset by the need to purchase higher-end gear. For all the other companies, you’ve got the stuff that can do the job, but you’ve got a big job to do.

There is so much more to security than:

1. Find the hackers.

2. Shut them down.

First of all, we need to know what, exactly, is on the network and what it does and whether or not it should be doing that function while connected without restriction to the rest of the network.

But before we tackle the question of what should be on the network, we need to go about discovering what is on the network, and this can be a journey full of surprises.

Typically, the start of this investigation will involve someone saying that everything that is connected to the network are phones, PCs, access points, and the printers. Oh, and also the badge readers. And the security cameras. But that’s it. Besides the barcode readers. That’s all, though. Hang on, we also have some digital signage…

At this point, you may now take what anyone has to say about what’s on the network with a grain of salt. It’s time to answer this question for yourself.

In a small company, you may be able to track down all the devices by hand during an off hour or two. It’s a great exercise and will prove invaluable for doing troubleshooting.

In a medium-sized company, this cannot be done alone. You’ll need a few other people to help out. That, or you’ll just do it all over a much longer period of time… however it’s done, you’ll likely also need some form of automation of tasks to get all that data collected in a usable way.

For a large company, this cannot be done alone. You will need tools. You will need a project manager. You will also need cross-team cooperation.

For all of these investigations, you will also need to talk to people that don’t usually talk to folks in IT. You will need to talk to them because they have connected things to the network, the likes of which you have never seen on a network ever before…

… and watch me talk about those things without once using a catch-all phrase that describes all of them…

In the small company, especially one that’s going through initial growing pains, there aren’t enough ethernet ports in wall outlets. That means, most likely, a cascade of cheap, unmanaged switches, also known as “cockroach switches” because when you see one of them, there’s a thousand more, hiding in the dark places of the office.

Because the switches are unmanaged, finding out what’s connected to them can be a chore. It can be done, but it may involve tracing cables up through holes in the ceiling and then dropping down into a room next door. You might also find the switches themselves in the ceiling space, acting as repeaters so the 100m cable from the main switch can extend into the nearby office space that was recently leased out.

In medium and large companies, even those with plenty of accessible wall ports available, people will bring in their cockroach switches and plug ’em in. Why? Maybe you should ask the developers who want to run 7 boxes in their cubicles and who don’t know they can requisition an old Cisco 3750 that’s still good, but was decommissioned last year. It could also be a boss that wants to have an extra laptop running or an app team that wants to have a concentration of monitoring devices for a war room or something similar.

But switches you can expect. I mentioned the unexpected, and I will now deliver on that promise.

Small companies have it lucky. Once the odd things are found – be they cameras, badge readers, printers, industrial devices, barcode scanners, automated fryers, refrigeration units, glucometers, or environmental controls, the person doing the discovery is not far away from the person responsible for those devices. And by “not far away”, I mean that both in the sense of both physical and organizational distance. The company is still small enough to have a familial feel to it, where everyone can walk up to everyone else.

Once you meet the person responsible for the unusual device, you’ll get a story behind it that’s likely to contain the business reason to have the device on the network. That, or a promise to get it off the network if it’s causing a problem.

In the medium-sized company, it’s a longer walk to have that kind of chat, and maybe you also have to go through a manager in order to have permission to engage in that kind of talk. You might even have difficulty finding out who you need to talk to about the serial-to-ethernet devices, the USB-to-ethernet devices, and the parallel-to-ethernet devices.

Also, you’re now more at risk to find ancient history still connected to your network. The small companies also tend to be *new* companies, so they tend to have new gear. Medium companies have likely been around for a while, and that means they could have devices that the company forgot about… devices that are now no longer supported by their vendors and which will need replacement in order to not be the security threat which they now constitute.

But for diversity and legacy, nothing beats the large company. The bigger it is, the crazier the scenario can get. Time for my disclaimer – I work for a company that makes a product designed to discover devices on the network and then classify them (among other things), and I have lots of large companies for my clients. With that disclaimer out of the way, I was recently at a client where, in the space of one hour, we reviewed a proof of concept deployment and found the following things on the wired network:

1. A cockroach switch.
2. A Nintendo.
3. A Windows 98SE PC. (Also connected through a cockroach switch, just for good measure.)
4. A network range used in more than one place.

Oops, forget to mention that fourth thing in my previous paragraphs. But, it’s a sad fact of physics or biology or some kind of science that, as companies grow, growing with them is the chance that some self-proclaimed techie will set up a network using an address space already operational somewhere else. The worst case of this was where a site that had a large number of guest wireless devices utilized the entire 10.0.0.0/8 range for it. We found it about a month after it was created, in the course of tracking down intermittent and unpredictable network timeouts and connection refused errors…

But I’ve seen earth movers, cows, ATMs, lightbulbs, drug pumps, silicon wafer fabs, vending machines, cash registers, information kiosks, ovens, refrigerators, pneumatic drills, scales, televisions, cars, personal health monitors, vacuum cleaners, and baby monitors all on customer networks. It’s not just that if there’s a thing, there’s both porn of it as well as an Internet-enabled version of it. It’s that those internet-enabled things will show up on your networks because either they were purchased and connected by the organization, or because people who work at that organization decided to bring them in and connect them up.

Some of those things are just fine, if they stay on guest networks. Some of those things are just fine, provided they are on segmented networks with limited or no access to the rest of the corporate networks and/or the Internet. And some of those things don’t belong anywhere on any network. The final say on which devices go where is up to the organization’s mission, values, and overall security posture.

But, before you can decide what should or should not be on the network, you need to know what *is* on the network.

The sentence is simple: get all the security solutions to work with each other. So how do different sized firms deal with that directive?

At the small company, the good news may be that there are only one or two solutions to work with. The bad news may be that they’re small business solutions that don’t have full enterprise features for integrating with anything. The bad news may also be that the IT person at that small business is either a visiting consultant or someone that handles all the IT, from the production line systems on up to ordering replacement RAM for company laptops. Basically, someone that doesn’t have 100% attention on security.

But let’s say that the small business IT person wants to do the right thing and be serious about security. She’s got an antivirus program for the PCs and a firewall for the Internet connection. She could stare at firewall logs all day long, or maybe she could spin up a syslog server. That sounds like it would be both a fun project and have a big payoff at the end of the work.

Unless she’s unfamiliar with Linux. Because that’s where the free syslog servers live. Linux is not an intuitive sort of thing, and learning it can be a difficult and frustrating experience. Chances are, if this IT person is dedicated enough to get into Linux, she may have moved on to a better opportunity by the time she knows enough to start up a Graylog server.

Now, if she’s staying with the small company out of sheer loyalty (maybe a family member or other dearly loved one is running the company), she’s got to learn how to do Greylog after that bout with Linux. Once that task is done, she can turn on logging on that firewall and create some rules in Greylog to alert her on specific rule violations or when there are multiple violations of the same rule from a single host…

… and then come back the next day to see her inbox swamped with alerts from the syslog server. Now she’s in the final phase of implementation, tuning the alert frequency. After that, she’s still faced with manually inspecting devices that are generating the most alerts because that anitvirus solution at the small firm doesn’t have any monitoring tools to go with it.

By now, she is master of the firewall, syslog, a fair amount of Linux, and how to find great deals on copier paper and toner. Not wanting to develop her copier paper ordering skills any further, it is quite likely she’s ready to rationalize away whatever loyalty she has and move on to the next opportunity.

And that’s the final obstacle for security solution integration at small companies. Quite frequently, they can’t pay enough to keep motivated, skilled professionals on the payroll. They’ll either have to deal with unmotivated IT people that really don’t care to stretch their skills or turn to a firm that will place someone onsite 2 or 3 times a week to check on how things are going there. If the previous person set up an alerting system, they’ll use it. Maybe. But they sure aren’t going to build one out. That’s work well above their pay grade.

So we follow our IT pro to a medium-sized company. Here, she’s no longer a department of one. For sure, she’s no longer dealing with renewing licensing for everybody’s softphones. She’s the security person, alongside the network person, the sysadmin, the phone guy, the 3 techs that do operations, and the wireless person. Not bad, am I right? She can specialize now, no question about it.

Well, maybe there’s a few questions about it…

For example, this medium-sized company has an AV system, an IPS here and there, a perimeter firewall and a datacenter firewall (different vendors, to boot!), a syslog server that is running at the very limits of the “free” offering from its vendor, a proxy server, and security is also in charge of the IPAM and PAM systems. There’s a good chance that our IT pro may not have heard of either IPAM or PAM and may even make the mistake of thinking they’re the same thing. But she’s on top of things and learns the difference between IP Address Management and Privileged Account Management, and all seems well, except for the fact that she has to ramp up on 6 different technologies. There won’t be any integration until that happens.

As she’s ramping up on those techs, she’s also responsible for supporting them. That means lots of explaining to users and developers why this security system or that one isn’t interfering with their application’s performance. She even posts this image in her cubicle and points to it as she sees a user or developer walk up:

(On a personal note, I’ve used that image. It has yet to prove my case to a developer out of hand, but it does help to set the tone of the discussion to encourage the dev to look for other reasons why the app isn’t working.)

While that helps with the firewall questions (see my personal note), it does nothing for the constant requests to exempt websites from the proxy filter. She’s barely got enough time to read product documentation, so when is she going to find time to integrate those solutions?

Moreover, how does she go about automating actions between the systems? It’s not like the firewall is built to take direct input from the proxy server. The syslog server seems to be the logical choice as a clearing house of information, but how can it be configured to send commands to one system or another based upon logging info that’s coming in from another source?

It’s possible that the security systems have an API that can allow commands to be sent to them. It’s also quite possible that the systems *don’t* have an API, or that the API is such that the syslog system can’t send commands to it. Even if the API is one that the syslog server can interact with, our IT pro would then have to learn how to write code. If she’s lucky, she can borrow a developer for a day or three to help with the project. If not, then she’s got a steep learning curve ahead of her if she’s never really done programming before.

But there’s also a fair chance that she won’t have to do all this alone. It’s entirely possible that the medium-sized firm has enough wherewithal to contract professional services from a vendor. If that can be done, then she can stay focused on her day-to-day work while the vendor’s pro serv person hacks out the code and does a knowledge transfer at the end of the engagement.

Now, I need to make a disclaimer here because I am part of a professional services team for a vendor. While someone could accuse me of wanting to feather my own nest, the truth is that, as a customer, I have benefited greatly from vendor professional services. They are definitely worth looking at.

The pro serv route is also available at the large company level. If we have our IT pro start a career at a large firm, she’s going to find that she can specialize more in the technologies she works with each day. This means that, while she gains a deeper knowledge of just 2 or 3 systems, she’s also no longer connected to *all* the systems. Other people on her team, possibly even other teams entirely, will handle those systems. Integration now means not just mastering the technology, but mastering the political considerations that go with cross-team projects. Will the integration mean one team or the other takes over a technology? If both teams manage the system, which managers are responsible for which functions?

One of the stickiest questions is: will we wind up stretching one product to fill a role that is actually better suited to another product? Added to that one would be: which systems does it make sense to integrate with which other systems? Both of these questions deal with lines of demarcation, where one system ends and another begins. For example, at what point does the antivirus protection end and the vulnerability scanner responsibility begin? Which has priority over web traffic, the data exfiltration protection or the proxy server?

While any integration at the small or medium sized company was done pretty much as a solo or very small group effort, the large company integration could very well be impossible without a multidisciplinary product team, with an oversight committee made up of about a dozen operational and service-line managers.

Like I said, “get all the security solutions to work with each other” is easy to say. Getting progress on that task means understanding the obstacles and then figuring out how to clear them out of the path.

I remember the first time I saw an AI antivirus program. I was amazed, impressed, and sure that it would be something we’d want to use back at my day job. After the conference, I leaned over the cube wall of the AV Manager and started to tell him what I saw.

He smiled, kind of cut me off, and said, “I’ve heard of those guys and another vendor that does a similar thing. However…” He swung his monitor so I could see it. It showed his admin dashboard for AV installations. “I need one of these. I can’t have any AV product unless I get to see an enterprise dashboard that tells me who has it installed and who doesn’t.”

That was at a global megacorporation. PCI, HIPAA, and other regulations require that any PC that connects or even might connect to a sensitive network have antivirus software installed and running. The regulations do not specify that the antivirus actually has to work, just that it be installed and running. The primary concern in the big company is in delivering a report to an auditor that shows the AV software is installed and running on every PC in the company.

As for dealing with viruses, that’s a simple matter. Download the latest signature, test it against a development environment, verify that it doesn’t break production, then roll it out. While it’s true that most AV packages can’t deal with a zero-day threat, it’s also true that most threats are from the dim and distant past. Remember CIH? Melissa? Nimda? Well, they’re still out there. They’re out there with all of their old-school buddies from 20 years ago, and that AV program is there to keep all the known threats out of the PCs it protects.

Flashy new products are nice, but the big firms need to know where they’re installed. Until the flashy new product can deliver that information, it won’t be installed. Even if the product can identify virus writers and have them proactively incarcerated, if the AV manager can’t show that it’s on every PC, it won’t be installed.

At the other end of the business size continuum, the key factor is price. Really small firms will have each employee download a personal version of a free AV program and just hope that the Business Software Alliance never knocks on the door. Once the small business is big enough to be on the BSA’s radar, it’s likely that the margins there are so thin that if an AV solution isn’t free or near-free, it’s a non-starter. If the flashy new product can’t meet that price target, then the small firm is going with a near-free vendor that can protect against those legacy threats just as well as the flashy new product that might also be able to stop zero-day exploits before they happen. The thing is, that proactive stuff comes at a cost they can’t afford.

The mid-sized company that’s outgrowing its near-free AV solution but still isn’t yet ready to bow down at the altar of big corporate dashboards may be the best chance for that flashy new product to find a customer. That being said, the flashy new product has an uphill fight against the name recognition of the existing major players. Who’s been fighting against all those viruses for 20 years and more? Not the Johnny-Come-Lately product.

And that new AV product will also have to be sure that it never, ever, ever, never no never not ever takes down production. All those cool new algos and AI learning potential come up face to face with the stark reality that, every so often, a production application does stuff that’s very much like a virus.

Maybe the developers took advantage of a Windows security hole to take care of a task. Maybe a developer copied and pasted some evil code into an app. These things can happen at any size of firm, and present real security issues.

I recently ran into this at a mid-sized company where I noticed that there were devices launching brute-force password attacks at file servers. We traced the attacks to PCs that were all in the same department. As it happened, they all used a particular application specific to their field that contained the brute-force code.The attacks continue as we wait for the vendor to issue an update that doesn’t include that code. The app was already white-listed with their AV program, so it didn’t get shut down, even though it was doing some horribly evil things on the network.

Then there’s the botnet I discovered one day in the badge readers at a large corporation. Those devices had enough Windows embedded in them to support the botnet, but not enough to be able to run the AV program. At least most of the company was running AV on their Windows workstations, so they were protected from becoming part of the badge reader botnet.

While the malware threat from whitelisted apps and IoT devices can be at any size company, there’s one particularly nasty threat that is more pervasive the smaller a firm is: users with local admin rights.

If users have local admin rights, and they typically do at the smaller firms, they can do all kinds of terrible things to their PCs, from accepting the installation of malware along with their Veeblefetzer searchbar add-on, on up to disabling their local AV program so that they can run their torrenting software without being interrupted about the malware that goes with those torrents. Large firms will also have local admin abusers, but the large firms are also more likely to be actively policing for that kind of abuse.

On the whole, I think small firms have it hardest when it comes to getting an AV solution. They have to deal with tight budgets, unchecked developers, and local admin rights for all, so they’ve got the hardest battle to fight. As the firms get larger, the better they get at fighting yesterday’s wars, but remain open to tomorrow’s surprises.

My choice of the title is based on the fact that the size of the business matters when we define security solutions. We don’t just consider the budget available, but the staff skill levels, user population, and overall levels of departmentalization.

Consider what can happen if a firewall admin notices a stream of outbound traffic to an unusual IP address that resolves to Minsk in Belarus…

At a small company, the admin will walk down the hall to where the CEO sits and ask if it’s cool to block traffic going to Belarus. “Sure,” says the CEO, “we don’t do any business with Belarus. Block the whole country.” Once the traffic is blocked, the firewall admin, who is really an all-around IT person, checks the PC that was sending traffic and makes sure its antivirus software is up-to-date. Maybe that’s when it’s discovered that their AV licenses have expired and they need to have a quick conversation with their vendor about renewal…

At a medium company, the firewall admin may notify his manager and wait an hour or two for a response to block just that IP address, since they may expand business to Eastern Europe at some time in the near future. Maybe. Once authorized to block, the admin may dash off an email to the desktop admin to check out the client at 10.1.2.3 that was the source of outbound traffic.

At a large company, the SOC may be up to its eyeballs in preparing reports for auditors to even notice just one more stream of traffic going to a Bad Place. Maybe they do notice it and generate an alert. That alert goes to the level one helpdesk person who then has to follow up with engineering about approval of a change request to shut down the traffic. In the course of the escalation, other teams get involved and start to build a full forensic picture over the next few days and they confirm that, yes, the traffic is originating from 10.1.2.3 and going to a Bad Place in Belarus. As they debate about what to do – they can’t just block the IP, since it’s a major ISP in Belarus that they use for B2B communications – the flow of traffic stops… so they decide to wait and see if it happens again before doing anything final.

Now those aren’t the only possible outcomes, but they illustrate the differences between getting security at different levels of business. I’d like to start a conversation of “war stories” that can help other professionals understand all the wrinkles involved in implementing security solutions, so that we can be more aware of those wrinkles as we discuss security with the decision-makers at those firms.

So what are your impressions and experiences, working at different levels and types of organizations?

Can coronavirus COVID-19 impact your network? The short answer is “yes”, if your firm hastily adopts a remote work policy without considering some common sense security precautions.

1. No personal email. The only exception for this would be to contact helpdesk about being unable to access corporate email. Personal email is not typically set up to properly archive and retain messages that could later be subject to a legal hold. The very use of personal email for business purposes can potentially expose your firm to liability costs that would exceed the value of whatever business you planned to get done.

2. No personal file sharing. This is right up there with personal email. Personal anything is not allowed for business use, mmmkay?

3. No Remote Desktop Protocol (RDP) use over unsecured Internet. If I had a nickel for every person that told the network team to open up port 3389 on the firewall so that they could work from home, I’d be comfortably well off. Yes, RDP means you can access your desktop or server from home. It also opens up great work from home capabilities for attackers. They will guess your username and password. It’s only a matter of brute force time.

4. No low-security options on the VPN configuration. While I’ll allow you to use RDP through a VPN connection, I’ll only allow it if your VPN is not just secure, not just really secure, but only if it is really really secure. That means not just IKEv2 and the best AES that your system will support, but also secured authentication that uses more than a username/password combo. Let there be a certificate or software token as part of 2-factor authentication.

5. No split tunnels. It’s tempting to let a local ISP handle all the Facebook and YouTube traffic that users consume in between productivity spurts, but don’t. Either pass all that traffic through your own network, or block it with a message that VPN bandwidth is limited due to whatever reason you want to provide in order to justify blocking that traffic. My point being that a split tunnel approach allows for an attacker on the Internet to bridge their attack through your user’s PC.

Can there be more possible pitfalls? Sure. These are just the five biggest ones. If your firm is anticipating a stretch where a large percentage of employees must work remotely, then take the time to bake some security into that plan so that reducing health risk doesn’t increase IT risk.

On 12 December 2019, Chinese broadcaster CCTV announced that a new viral outbreak had started in the city of Wuhan. While the first confirmed case was on 17 November, it was not until more cases came to the attention of authorities – in a way that they could not ignore – that the Chinese government began to publicly acknowledge something new was underway. Following that 12 December announcement, the world began to transform. As output ground to a halt in much of China, factories depending on Chinese raw and intermediate goods had to slow or stop production. The lesson learned was both sharp and timely – “just in time” methods of production left firms vulnerable to disruptions in the supply chain. If firms kept a reserve of parts, those could have lasted through at least some of the lapse, if not all of it, and would have allowed for less economic dislocation.

Part of the “just in time” mentality of go, go, go all the time is the ideal of “five nines” or even “six nines” – 99.999% or more uptime for all systems. While, yes, this does mean the product always moves out the door, it also means that the things making those products go unpatched and unprotected for long stretches of time, making them prime targets for attackers. Those vulnerabilities leave the firm just one click on an email attachment away from utter ruin.

Just as there’s an argument to be made for adding some storage capacity to help weather supply chain shocks, we need to talk about “two nines” uptime as a way to avoid eventual “infinite zeroes” uptime conditions. If you give me 100 minutes each week, I can get a breathing space to apply needed patches on production servers and equipment. If I don’t need a week’s 100 minutes, let it roll up into next week – maybe I’ll need more time to apply the next patch, who knows? But let me have a reserve of time during the working year so I can do my job to patch and protect. Let me reboot gear that needs its queues cleared, let me stop and restart services on servers, let me keep things up to date so we can spend the other 99% of the time feeling more confident about the resiliency of the environment… just in case, ok?

I’m aware that executives in most nations have a fiduciary duty to maximize shareholder value. That’s a short term goal that is itself replete with abuses when it considers employees as expenses as opposed to capital or when it looks at wages as a race to the bottom. I’ll leave those criticisms of neoliberalism for another paper at another time. But here is where I criticize those fiduciary duties as regards security. Maximizing shareholder value means minimizing expenses in the short run, and security is seen as an expense, not as an investment. Current accounting structures blind the books to an ability to properly assess the value of a security system in its ability to provide long-term stability and constancy. I would love it if share prices for a firm jumped every time it announced it was undertaking a security project. Sadly, they’re more likely to drop as those expenditures for security are seen as short-term profits lost, not long-term profits gained.

In the meantime, I’m reading headlines about increases in ransomware and other attacks using email attachments with references to coronavirus, COVID-19, and even SARS-CoV-2 to successfully penetrate those PCs bridging traffic between the raw Internet and the corporate VPN, because it was cheaper to use a split-tunnel solution than to backhaul all the Internet traffic through the corporate networks – and also because it was seen as “nicer” than banning non-business related Internet usage for devices on the VPN. I know I’m getting into just one of the technical weedpatches of issues, there are others… and if firms could see their way towards working more for the long haul than the short-term gain, we’d likely have the right solutions instead of the cheapest and easiest, which are never the strongest.

I work for a security vendor and I see into many customer environments. Often, the thought pops in my head, “If I were an attacker, I’d really get them if I [REDACTED].” And if I don’t think I can get them with [REDACTED], then [REDACTED], [REDACTED], [REDACTED], and [REDACTED] round out my top five most common ways to break into an organization and have access to quite a lot that I shouldn’t have access to.

It’s not that these methods necessarily have something to do with the product I support. Some do, like [REDACTED], [REDACTED], and [REDACTED]. But [REDACTED] and [REDACTED]? Well, that’s some other vendor that helps out with those weaknesses.

Now, as you read those paragraphs, what thoughts do you have that fill in the blanks I left? Unless you’re someone that works in the same line of security that I do, I’d dare say your top five exploits list is different from mine, in part or in whole. What’s more is that we may even have some of the same customers, and we may, between all the people that work in security at that customer or on behalf of that customer, we may actually know dozens of things that fill in those tantalizing [REDACTED] blanks.

Now, I know that the customer might want to know all the details of everything I notice, but I’m often noticing things that are out of the hands of the people I’m directly working with. They can only report up to their manager, and that communication only goes so far before it drops in urgency and loses its audience. Or, worse, it’s just added to the list of security things to fix, right behind [REDACTED], which got noticed last week in an audit finding.

So, let’s ask the question: what are the things that are the hardest to fix that leave organizations the most vulnerable? There are a number of “10 Quick Security Fixes” articles out there. Everyone knows how to pick low-hanging fruit. What I want to ask about are the projects that nobody wants, the projects that get people fired, the projects that land everyone in the [EXPLETIVE DELETED]. Because these are the ones that don’t get done, get done but badly, or get done only to such a point as a box on a checklist can be ticked, and then no more. For example, nearly everywhere I’ve been to has firewalls set up. That’s good. But when we talk about turning the firewall concept inside, to regulate traffic in a segmentation project, then I know I’m going to have an uphill fight in getting information about which apps use which ports.

Why will I have that uphill fight? Because I have to ask for netflow, that’s why. And then we need to talk to different teams about when they run their apps so that we’re sure to not block anything that runs only once per year. And then we have to deal with how Microsoft recommends that we leave open ALL. THE. PORTS.So that’s just one type of project that is practically a mission impossible.

What projects do you face that are nearly impossible, but fill in those [REDACTED] blanks?

I promise the dear reader that this will not be just a rant about how nobody takes security seriously or anything in that vein. Read on, and I’ll get to the actionable items. I just need to set some things up in order to give credence to my conclusions.

Some years ago, the Polish science fiction author Stanislaw Lem wrote an essay about weapons development titled “The Upside-Down Evolution”. In it, Lem called out several interesting trends: miniaturization, dehumanization, and deformalization. The key trend gave the essay its title: rather than developing smarter and smarter AI, the true breakthrough Lem foresaw was not in artificial intelligence, but in artificial instinct. Lem postulated that a weapon need not be coded to handle all types of situations. It only needed to be able to perform a certain range of tasks under certain conditions, nothing more.

Combined with miniaturization and dehumanization, limited weapons systems – artificial insects, in Lem’s parlance – also allowed for the deformalization of war. No more a matter of exchanged ultimatums and formal declarations, war in Lem’s future would be constant and acts of aggression difficult to attribute. Consider a swarm of artificial insects each carrying a fractional amount of fissile material that converge on a location to create a critical mass for a nuclear explosion. If all the artificial insects are destroyed in the explosion, who could say what actor or actors was behind the event? Could it be an attack by a foreign power or a false flag attack used to justify an attack on another foreign power? Or could it be done to frame a third party?

Once deformalized like that, warfare would be constant. Natural disasters could be no more than just that, or they could be the products of an attack by a hostile party. There would be no way to tell the difference.

While we are yet to see Lem’s artificial insects on a grand scale, we *do* see the next closest thing – cyberattacks.

Cyberattacks check all the boxes of the upside-down evolution. They are mere digital streams of signals – miniaturized. They are often products of algorithms – dehumanized. They are always out there, always attacking in the ways they are set up to attack – deformalized. And they only do that *one* set of operations that they have to do – artificial instinct.

Lem’s essay did not go into matters of defense except to say that the need for uniforms, marching, parade drills, and generals all went by the wayside. At best, those were worthless vestiges of another age. At worst, they hindered responses that had to be just as rapid and ruthless as the attacks. Lem only considered nation-states, but we now live in an age with a myriad of players having access to these attacks – and a myriad of defenders still trying to fight the last war.

Old-timers will remember Clifford Stoll’s epic, The Cuckoo’s Egg. The story is of a human tracking and trapping another human. At the time, the FBI was uninterested in the case, as no large sum of money was involved (less than $1) and no classified files were accessed by the attacker. While we may look back on that and shake our heads the way modern combat veterans would react to how various World War One generals dismissed the power of the machine-gun, that was the FBI still fighting the last war.

Well, Stoll went on to write in 1995 that the Internet was just a fad and would never catch on as a platform for commerce and information exchange. Yes, he still kicks himself over that article, but at least he’s aware of the irony and how outdated that thinking was. And though I talk of a mindset fighting the last war, that was the 1986 mindset. People today may have moved beyond that, but not much. Most are still expecting a Stoll-like boffin to do the investigative work to catch the baddies and bring them to justice. That’s because the events described in The Cuckoo’s Egg are those of a previous war.

To be perfectly honest, most firms aren’t even thinking about fighting a war. They’re not built to do so. At no point is there an MBA class on Sun Tzu’s Art of War that ever tells the students, “You know, this really isn’t allegorical when it comes to IT.” I know this because I have yet to work with a customer in the business world that doesn’t underline the principle that security won’t interrupt business as usual.

I’m sorry, but that’s quite the paradox, Mr. Customer. Do you want business as usual without security, or do you want to change how you do business in order to have security? Are you still forming soldiers into phalanxes of spearmen for operations on an open field of battle, or do you plan to tell them about the need to disperse and entrench so as to avoid being overwhelmed by large-area effect weapons? If still the masses of spearmen, I have a rude surprise waiting for them when the drone with a fuel-air explosive arrives on the scene…

… and even that analogy is out of date, as the actual attacks coming at us every day are not even needing drones in order to do their damage. Worse, because we put emphasis on doing business first, we’re only looking at security as a bolt-on. That means the underlying systems will always be more vulnerable that necessary.

So what keeps this article from being another mass of groanings about how things are? What are my fixes, my takeaways that businesses can put into place? All right, all right, I’m ready to get to my point.

You’ve got to apply upside-down evolution to your systems. Doing so will give them higher immunity and better resiliency against attacks. It will mean more interruptions to business, but of less total time than what would happen to your business if there was a successful denial of service attack against it. Moreover, the interruptions will be localized, not general.

Automate your responses to any breach of standards, and make those responses harsh. Do not exempt anything. I grant that the last sentence is more a starting negotiation position than a final state, but I stand by it, all the same. When the endpoint or server or application goes wrong, shut it down immediately and get it fixed just as fast. Then, when it comes back online, it is fresh and ready to defend itself.

And if your shutdown actually caught an attacker, so much the better. The swift action meant limited damage. Do you know how Taiwan had such a low infection rate in the recent pandemic? It shut down ALL travel to the island. Nations that made exceptions got hit hard. Taiwan made no exceptions, and that swiftness and harshness saved lives.

What is your return on investment? Your business stays open, allowing you to continue to get returns on all your other investments, that’s the ROI. The coming years will see attacks that are more miniaturized, more dehumanized, more deformalized, and more artificially instinctful. Trying to stay open 24/7 in that world will be like leading those spearmen in a charge against a tactical nuclear warhead. Automate, be strict, and accept small downtimes now instead of permanent downtimes later. Fight the current, upside-down evolution-born war, not the one where we trace a 1200 baud modem connection back to Bremen after months of investigation.