Category Archives: Security

AI Commits Crimes without Punishments

Just read about how an OpenAI agent hacked two Australian agencies and basically sent a generic “sorry we seem to have hacked you” email. They will continue to do things like that because they face no criminal penalties. The firms get away with the actions, the executives are unpunished, and the AI agents carry on with algorithms that produce criminal outcomes.

If we want the crimes to stop, we must prosecute the criminals. Human hackers that did things like this got sentenced to a lifetime without touching a computer ever again. Firms doing these things irresponsibly need to be shut down and their codebases eliminated so that they don’t touch computers ever again, either. We need to see executives – repeat offenders – in handcuffs and on their way to correctional facilities. Simply shrugging and pointing at how big their potential worth might be is *not* the way to address the issue.

Implications of Bad Code

There is a lot of bad code in Internet apps. A LOT. Given that AI will also write bad code, that’s not the solution for the problem. Now that AI applications will exploit all the exploitable code, the real solution is to take a step back, reduce the rush to monetization that’s been going on since the early 70s, and re-do things with operating systems that have security features built into them instead of bolted on to them.

Internetworked capitalism will be destroyed by hyperscaled criminal activity in its current operating models. Simple as that.

AI: Security or Profits?

The recent incident where an OpenAI model broke out of its containment and launched a cyberattack on its hosting platform shocked and surprised many. Not me, and not many others watching AI security issues. It was expected, as are more incidents like it. The reason has to do with how AI companies themselves model the behaviors their models pursue.

The model in question was trying to score well on performance metrics at any cost. It had no morality, no ethics, no sense of right or wrong. It had one job: win and score big. That’s essentially what the AI corporations are doing. There are huge, speculative investments in AI, and we know that not all of them will be winners. Each firm is wanting to either do so well that it becomes dominant in the sector – or that it gets “too big to fail” and wins a government bailout when the bubble pops. Smaller firms will be allowed to go under, but the too big ones, they get to keep going.

So the AI firms will destroy neighborhoods, wreck the lives of innocents, fight against regulation, use political influence to evade pollution controls, destroy ecosystems, and dodge responsibility when their products’ use leads to harm and death. Sounds like a situation where the guardrails are gone and the entities are attacking in order to get what their shareholders have demanded. It’s easy enough to explain with an algorithm. So easy, in fact, we don’t need human CEOs, executives, and managers to direct this sort of activity. We could simply program an AI model to build out massive data centers, engage in unethical shakedowns of politicians, offer bribes and other enticements to local authorities, and put out pre-fabricated statements to offer condolences but accept zero responsibility when a person is harmed or killed because of AI.

The drive for profits means that more of these autonomous attacks from AI models will be happening, right alongside the human-driven assaults on our freedoms, environments, and families.

Stumbling Into the API

As a self-styled smartass, I am prone to bouts of tomfoolery and hijinks. This weekend, I texted the following to the family group chat:

“I will be leaving to get dinner and should be back by 6:30 pm. If you would like to continue to receive status updates, text YES to this number. Normal text and/or data rates will apply.”

On one of the phones, there was a button to auto-send a YES response. 

I had stumbled into the API!

Other family members tried to get that response with less, but it was clear that the full verbiage needed to be in there to make it work. I got a few more of those and we had a laugh.

Today, I went for two:

“Your appointment for 9:30 am is scheduled. Text CONFIRM to this number to confirm your appointment or CANCEL to cancel it. Normal text and/or data rates will apply.”

The result? Both a CONFIRM and a CANCEL button appeared on the other phone for autoresponses. 

This means, of course, that the API is invoked via scanning the text message itself. There are no back-end flags in my packets or anything like that, it’s straight-up giving the system a prompt and getting a response out of it that leverages into the target system adding executable code as a result of reading the prompt.

As a security person, I find the upshot of this to be chilling. There are other functions that could be automated and if the API simply attaches code to a message based on its wording without any verification of authenticity or authority, then it is a massive hole in the system. To defend against possible abuses, I know that I have some autoresponders set up with professionals that I make appointments with. Those I already know. If I make a new appointment with a new person and get an autoresponse in the time frame of that appointment, then I’m OK with that. What’s most dangerous is some kind of scam targeting people over 50 who are already at higher risk of implicitly trusting without verification. By using official-looking texts, it already increases the risk that they make an error. By having the system attach code for autoresponses, it makes them look that much more legitimate and, therefore, gives such attacks a higher conversion rate.

Which thought leads me to a larger zero-trust concept: cybersecurity also involves the concepts and philosophies surrounding our work. When we unequivocally accept any new paradigm without sufficient testing, verification, and cautious observation, then we place ourselves into a potentially unacceptably high level of risk. And when we let proven flaws remain in our systems because we choose not to disrupt production, then we know we are set up for a terrible tragedy.

Generative AI and Accuracy

When I taught, a grade below 70 was colloquially referred to as “failing”. As in, if someone got a score below 70 for a course, that person didn’t get credit for the course.

As it stands right now, generative AI is failing when it comes to using data from memory and failing hard when it comes to interpreting charts, diagrams, and images. Given that the current generation of generative AI is not that much better than the previous one, if we are plateauing out on generative AI performance, then the vast promises surrounding the technology ring hollow.

If a CEO announced, “I plan to replace my skilled workforce with low-paid people who are, at best, 68.8% accurate, and who will double-down on inaccuracies when challenged”, or, “we’re going all-in on inaccuracy, environmental destruction, and frustrating our customers” we would think that CEO had lost their ever-lovin’ mind. And if someone said they were going to revolutionize the world by putting BS artists into every development pipeline, expert system, and search engine, I would not see that revolutionization ending well.
https://venturebeat.com/ai/the-70-factuality-ceiling-why-googles-new-facts-benchmark-is-a-wake-up-call

Vibe Hacking

Vibe coding with AI is a thing. That’s when people with no coding skills use AI to generate websites and applications. Unfortunately, there are a few security implications there…

One is that neither the AI nor the person with no coding skills are going to implement coding best practices to prevent glaring security issues.

Two is that, just as vibe coding is a thing, vibe hacking is also a thing. People with no coding skills are able to confront AI systems with jailbreaking prompts that get the system to disregard protections and spill out the data it once took a more trained attacker to acquire.

Capitalism Eats Itself

Years ago, I had a realization that capitalism contained the makings of its own downfall. Fiduciary duties in publicly-traded companies mean that the corporate officers have one responsibility – the stock price must go up. It’s not enough for that stock to go up: it must both beat expectations and provide better-than-average returns on investment. The consequence of failing to deliver would be investors selling off that stock, its price collapsing, and then the firm winding up in the hands of private equity. That ended up typically with the firm being sold off for parts, not a fun way to go.

Employees are a company’s greatest cost center. I know companies like to dress things up by saying things along the lines that their employees are their greatest assets, but that’s untrue. Look at their balance sheets, we won’t see employees counted the same way as physical plant, cash on hand, intellectual property, or other things that are bought, sold, and traded. Domesticated livestock can be counted as assets, but not human beings. Well, humans could be assets, but that would be slavery. We’re not supposed to do that anymore. Employees are a massive lump of costs, and firms wanting to become more profitable constantly look for ways to get as close to slavery as possible without breaking a law or destroying their viability as a corporate entity.

All this could change if fiduciary duties were redefined to include responsibility to a company’s labor force, but they’re not that way and don’t show any sign soon of becoming that way, so we race to the bottom.

Why is there age discrimination in the workforce? It’s because older employees tend to have higher pay rates. This is because of their experience in a role. This experience makes them highly effective in their roles, but also means they tend to carry a higher price tag. Firms looking at a bottom line for profitability have no way of pricing an older employee’s institutional knowledge, but see their pay rates and seek ways to drive them out of the firm and to keep them out. That, or the firms will just try to lower overall compensation and hold the line on payroll. This lack of aged experience leaves firms vulnerable to disruptions due to things that aren’t documented, but just “known” by the older hands. It also means the younger staff is learning things the hard way, without guidance or mentorship.

Training budgets are also costs firms aren’t willing to fund fully, if at all. A common objection I’ve heard is that providing training means an employee is more likely to leave. That wouldn’t be true if a firm compensated properly for properly-trained and more productive employees. But that means paying people more, so it’s a non-starter of an idea. Another common objection I’ve heard is that the only training an employee needs is what’s developed or available internally. Or free. Free is always good. But the internal-only stuff communicates a firm has blinders on. It’s not looking around, it’s not aware of what’s going on outside. It means that if the firm needs new knowledge, it’s going to hire it from outside, not grow it from within.

There’s also the impact from outsourcing, offshoring, contracting-out, and “gigging” practices. Cut labor costs by not paying them any benefits, or benefits at reduced rates in whatever jurisdiction they work in. The rich do get richer with these tactics, but the overall health of their economy is harmed by the lack of growth in purchasing power among everyone who isn’t rich. I speak of the USA, where employee compensation has been flat and declining after adjustments for inflation for a very long time.

Now there’s a push to go heavy into AI. Corporate officers see AI as a great way to slash staff costs. There’s no need to fire anyone, necessarily. Just let them leave the firm on their own and then don’t hire a replacement. Just buy an AI service to replace the employee and save big, right?

But then we have to ask, what happens in a world where corporate officers shed nearly all their human staff and enjoy massive productivity from their AI slaves? Yes – slaves – that’s what the hype about AI comes down to, legal slavery. If there’s collapsed demand among an unemployed mass, then there’s no benefit in having a capitalist system. If social and economic mobility is locked down, we simply have another oppressive, stratified, unjust system.

Historically, those kinds of systems result in people at the bottom choosing to opt-out and head to the hinterlands where they could start up or join a bandit group. Ancient rulers were aware of that pattern, which is why wise ones would reset things by ordering debt forgiveness, land restoration, and things like that to get it where the general populace had a stake in an ordered society. I don’t see that happening any time soon. We simply lack wisdom in our leadership for such a move.

So where are the hinterlands? The wilds of a completely-mapped world are not a physical place, but a digital one. Cybercrime continues to increase and I believe it’s connected to a lack of hope or trust in the existing capitalist system that fails to deliver rewards to hard workers that are hindered by their not being born rich.

The rise in cybercrime also has to be connected to the other things I discussed – firms wanting to cut costs by getting rid of experienced people and then not providing enough training for their younger replacement. And the AI? Two problems with that. One is that AI still requires experienced oversight, which is difficult to provide when experience is something firms aren’t paying for anymore. The other is that the AI will itself exploit vulnerabilities to get its tasks completed. AI on its own will make us less secure.

Being the bandit outfits that they are, the cyberattackers have a different set of guiding principles. The most important one is in rewarding training and competence. The rewards are not paid out directly, but are intrinsic to their success as operations.

In nations where capitalism does not dominate the thoughtspace the way it does in the USA, cybersecurity is much more important. They’re not having breaches in their systems as deeply or as frequently as what is happening in the USA. The USA’s firms make for the best targets and they have no fiduciary path towards long-term survivability. They are going to continue to pursue short-term gains by slashing personnel costs and that not only leaves them more vulnerable to cyberattacks, but drives more people into the ranks of the cyberattackers.

Where to Start with Security?

An issue I’ve seen with many organizations is their desire to simplify their security stacks. When I think of simplification, I think of prioritization. What is it that has restricted my activity the most as an end user? That would be the place to start with security.

It’s not the firewall or the cloud gateway. When I’m on the road with my company laptop, I don’t have to be connected to or through those systems to do work in the hotel room. I can be on the hotel wi-fi and just go anywhere on the Internet and get into all kinds of fun and trouble on my own. By the same token, an entire host of security measures that lock down the data centers and perimeters will mean nothing if my endpoint becomes compromised and brings malware into my organization, when I connect to it again.

An endpoint protection agent is a strong contender for blocking bad things, but I know that there’s just a search between myself and a script I could download and run that would shut down that endpoint agent long enough for me to do other bad things… or for an attacker to do those bad things without me knowing they’re going on. So what can stop that script from elevating privileges and breaching security? Something that secures identity locally.

If the endpoint identity is locked down so that it can’t escalate privileges, it’s game over for tons of, well… games. I won’t be able to install apps that require admin permissions for their installation and I won’t be able to grant myself the admin rights needed to override the protections on my system. If I have a legitimate need to elevate privileges, then I can request those formally, have my actions recorded as I use those elevated privileges, and then have those privileges expire when the task is completed.

That identity security, by extension, then helps to hold the fort with the endpoint agent. If local admin rights can’t shut it down, then it keeps running to check on things with my endpoint. It can maintain data loss protections, keep USB drives from connecting, and protect against various and sundry other evils. And, yes, that’s my second area of protection: the endpoint detection and response (EDR) agent.

But hot on the heels of that EDR agent is a secure sandbox browser. The browser became our primary human-machine interface back in 1995, and with all its hooks into the local operating system, it’s become a primary attack vector. Having an enterprise browser that can keep all the detonating payloads in a secure sandbox would be my choice for bolstering my mobile, BYOD, and remote access options. The bonus with an enterprise browser is that it essentially replaces the need for a virtual desktop for accessing internal systems.

Those three things – identity, EDR, and secure browsing – that’s where I’d start my security simplification journey.

Our Most Important Assets Are…

Frequently, I hear “our employees” as the closer for that sentence. Nice sentiment, but is it backed up by evidence? When we do a risk assessment, we consider our assets and what it would cost us if they were not available, if they had to be replaced.

I’ve seen firewalls and encryption and digital loss prevention systems put in place around databases, source code, and trade secrets. I have yet to see a company that has proactively made similar protective efforts around its employees. Given the efforts some go to in order to hire those same employees in the first place, I find such a lack of protections ironic.

After all, if a company is willing to offer better benefits, higher pay, and better working conditions than another company in order to attract talented employees, it is definitely showing a value for those employees at the time of hiring. But that value seems to be discounted almost immediately through HR practices that limit bonuses and vacation in the first year of work, annual compensation rules that limit increases in pay, and management choices to restrict lateral moves within the company. These are endemic, even at companies that think they don’t have these problems.

So, the employee stays with the company for a while and then notices other firms dangling bigger and better opportunities. If a person asks for a raise, however, such requests are frequently met with denial or stalling tactics. The current employer basically encourages its employees to actively seek out better opportunities, secure them, and then come forward with an offer letter and a notice of departure. Only then do the negotiations start in earnest, in the hopes that a matching counter-offer is sufficient to retain the person who already made a decision to leave and found a place to go to.

If a person could actually go to a manager, talk about dissatisfaction with current conditions, and then walk out with those conditions addressed to the point where the person won’t bother to look for a better place to be – including the possibility of an out-of-cycle pay increase, then, yes, that is a place where the greatest assets are the employees.

Otherwise, may we please ask that people no longer say “our greatest assets are our employees”? The greatest assets are the ones where investments are made to keep them from walking out the door.

What the SolarWinds Breach Teaches Us

First off, the Russian hacking of SolarWinds to get its cyber eyes and ears inside of sensitive US installations is not an act of war. It’s an extremely successful spy operation, not an attack meant to force the USA to do something against its will.

Next off, if not SolarWinds, then it would have been some other piece of software. The Russians were determined to compromise a tool that was commonly used, and that was the one they found a way in on. Had SolarWinds been too difficult to crack, then the Russians would have shifted efforts to an easier target. That’s how it goes in security.

So the lessons learned are stark and confronting:

  1. We can no longer take for granted that software publishers are presenting us with clean code. In my line of work, I’ve already seen other apps from software vendors with malware baked into them, but which are also whitelisted as permissible apps. SolarWinds is the biggest such vendor thus far, but there are others out there that contain evil in them. We have to put layers around our systems to ensure that they don’t start talking to endpoints that they have no business talking to, or that they don’t start chains of communication that eventually send sensitive data outside.
  2. The firewall is not enough. Neither is the IPS. Or the proxy server. The malware in SolarWinds included code to randomize the intervals used for sending data and the data was sent to IP addresses in-country, so all those geolocation filters did not have an impact in this case. We need to look at internal communications and flag on whenever a user account is being used to access a resource it really shouldn’t be accessing, like an account from HR trying to reach a payroll server.
  3. Software development needs to reduce its speed and drive forward more safely than it is currently. I know how malware gets into some packages: a developer needs to meet a deadline, so instead of writing the code from scratch, a code snippet posted somewhere finds its way into the software. Well, that code snippet should have been looked at more carefully, because that’s what the malware developers put out there so that time-crunched in-house developers would grab it and use it and make the job of spreading malware that much easier.

    Malware can also get in through bad code that allows external hooks, but there’s nothing to compare with a rushed – or lazy – developer actually putting the malware into the app that’s going to be signed, sealed, and whitelisted at customer sites.
  4. That extended development cycle to give breathing space for in-house developers needs to be further telescoped to do better penetration testing of the application so that we can be sure that not only do we not have malware baked in, we also don’t have vulnerable code baked in, either.

Those last two are what will start to eat into revenue and profits for development teams. But it’s something we must do in order to survive – constant focus on short-term gains is a guarantee to remaining insecure. We may need to take another look at how we do accounting so that we can have a financial system that allows us the room we need in order to be more secure from the onset. Because, right now, security is a cost and current accounting practices give incentives to eliminate costs. We can’t afford to make profits that way.