Author Archives: deanwebb

Good Morning America How Are You?

Leave a reply

The city of New Orleans just got attacked and that made me think of the song about a train by the same name, whose chorus opens with that line… but this time, the question lacks the soft charm and slow nostalgia of Steve Goodman’s folk song. This time, the question is cold, jarring, unnerving. It’s not the first major US city to be attacked and made to be dark and it won’t be the last. The cities and other local governments of the USA simply aren’t going to be able to deal with cyberattacks on their own, so they’re going to be target-rich environments for state actors and the criminals they hire to detonate hand grenades to cover their tracks… or just the criminals who blow things up, you never can tell.

We can tell the cities and counties and states of the USA all we want about security and be met with the tired, nodding heads and empty eyes of IT staff that tried to tell the same message to their higher-ups. They know. They’re not idiots. They’re just faced with small budgets and political imperatives to get stuff done, no matter what. They know that when their town / county / state experiences a major breach, it will lead to the first time that entity seriously considered spending time and money on security measures. It will lead to the first time IT is allowed to do what it knows needs to be done, even if it’s done on top of the rubble and ruin of the past.

Do they have a perimeter firewall? Sure, but there was the time somebody high up got mad about traffic being blocked, so it’s set to permit all traffic by default. Do they have a datacenter firewall? Yes, indeed, right here in this box in the storeroom. It is fresh and ready to go. Do they have antivirus running on every PC? Absolutely. Well, we can only tell for sure on PCs that have antivirus running on them… we don’t know about the ones that have fallen out of communication with our software maintenance platforms.

Need I continue? Some of you are already at the point where you can bear the horror no more, but I must press on! You must see more, that you know the depths of their helplessness! Do you see the unsecured Internet line in that office, terminating on a Windows server with RDP running, no limit on logon attempts? Do you see the flat network, with telnet still running on switches and routers? Do you see massive file shares with no permissions set to halt normal users from deleting or changing files? Do you see the backup server that constantly fails its nightly backups, with the backup operator simply clicking through the errors on his shift because he was told long ago to just ignore them? Do you see the gear that all respond to the SNMP community “public”?

And there is more horror in there, I say. I didn’t even get to the Windows NT 4.0 server that’s still on the network. Why? Well, the payroll application couldn’t upgrade to run on Windows 2000, so we keep it going on that server over there… and there is yet more, deeper and deeper into hell.

Who knows what static routes lurk deep within the network, routes that bypass the firewall entirely for special IP addresses in faraway lands where US lacks extradition rights? And are there programs on unsuspected and unsuspecting systems that are just counting down the days until the dust settles, things revert to normal, and the problems of the past make themselves available for mayhem once again? Clean up all you want, but what do you do if that payroll server on NT 4.0 is infected? The only person who can rebuild that system died 3 years ago. If it’s infected, maybe we can just put it behind a firewall and only open the ports needed for Windows and Active Directory. Oh wait, that’s all of them…

So what is the solution? Is this where the federal government steps in and supplements the IT budgets of local government entities? Or would that lead only to swollen management salaries with pittances spent on actual new technical hires? Is this where the feds create a system of firewalls to filter all traffic entering and leaving the nation, such as the Chinese do?

Actually, that might be what we need. It wouldn’t do anything for completely domestic attacks, but it could do at least something to halt attacks from outside the USA, right?

Except… how do we know the difference between legitimate traffic from abroad and traffic with malicious intent? Encryption doesn’t allow one to peek into the packets very easily. Banning known bad source IP addresses just leads to attackers compromising systems with other IP addresses and then launching attacks from there.

But maybe the protection is on the outbound side, with a massive proxy server cutting communications with scam sites and other evil online in other countries. But for how long would the proxy server be protecting us only from malware and fraud? Wouldn’t law enforcement argue that we need to be protected from terrorist propaganda? How broad is that classification? Wouldn’t entertainment firms want to protect us from download sites? Would they also want to “protect” us from foreign entertainment outlets that didn’t allow them to act as middlemen brokers for their content? Would we also be “protected” from foreign news sources that didn’t go along with the administration’s views? Blocking Russian state news propaganda I wouldn’t mind, but I sure would mind if a CBC or BBC investigative journalism programme that was critical of a US firm or governmental policy was blocked.

I hate to suggest this, as it’s highly exploitative, but we could allow recent grads to learn IT and then work for pathetic, near-volunteer wages for local government entities in order to pay off their student debts. I hesitate to introduce a scheme to offer pardons for nonviolent offenders that do pro bono IT work, since fraud and cyberattacks are, themselves, nonviolent crimes…

The City of New Orleans owns Louis Armstrong International Airport. Did this recent attack penetrate into the airport? Or was the firewall that is supposed to sequester it also permitting all traffic because there’s a full trust between its AD domain and the City’s? Or for some other reason, I don’t care. It’s all a nightmare, and when I wake up, there’s some shadow moving across my screen, saying, “g00d m0rn1ng 4m3r1c4, h0w r u?”

I don’t know how to answer that question. I normally don’t want to curse the darkness without lighting a candle, but I’m at a loss for answers to all the questions I asked. Cyberattacks can produce near-nuclear results, if done on a sufficient scale and with intent to destroy, not just encrypt and demand ransom. Perhaps lasers and hypersonic missiles can defend the USA from sudden attacks launched from bombers, ICBM silos, or nuclear submarines. What good are those against cyberattacks that target our highly vulnerable small government entities?

2019-11-11 As a Cold Front Approaches

Leave a reply

The sun yields the floor to the clouds
Temperature falls, wind and drizzle
Remind the nose and ears there are seasons other than summer
The hemisphere tilts towards winter, towards snow,
Towards a quiet, dark blanket
Towards a stillness of thought

Time for a song to play while stepping on the damp leaves underfoot
A song about thinking about the year rolling to a close
A song about the life to spring forth in the future from the descending quiet
A quiet song, with motion underneath it all
A stirring beneath the bark as the hemisphere has its afternoon nap

It’s raining a little, so why not cry a few tears of thanksgiving?
Why not smile beneath the scarf?
Why not oil the heart with gratitude as the cheeks get wet?
It is cold, but I have warmth
I have love
I have forgiveness
I have hope
These are worth tears, worth the thanksgiving
These are worth a humble accounting
Here as the hemisphere spins ’round a darkened pole
There is a light within, sustaining

The cold outside is part of life
Therefore, I am thankful for that cold, that pain
Life is life
The lichen under the rock
The bear in the cave
The frog in the mud
Time for that song, the damp leaves song
The thankful song
The quiet, peaceful, grateful song.

Do You Rate Use Cases For Maturity?

Leave a reply
https://www.peerlyst.com/posts/do-you-rate-use-cases-for-maturity-dean-webb

More than once, I’ve been in the meeting where someone is questioning whether or not to get a particular security system. This someone asks, “OK, so if someone has the CEO at gunpoint and forces him to log in to his PC and then takes pictures of the documents visible on his screen, then blackmails the CEO to say nothing to the local police as he slips away into the shadows and to a foreign nation where extradition is difficult, will you be able to stop that data exfiltration?”

“Uh, no…”

And then that someone crosses arms and boldly states, “Then why bother with all this trouble if it’s useless against a *real* hacker?”

Now, maybe it’s not exactly that scenario. But whatever’s offered up is an advanced use case that even the tightest of security nets would have trouble catching. And if the current state of the IT environment is where someone could bring a PC from home and copy all the files off the main server, maybe that group of advanced use cases isn’t what anyone should be worrying about right now.

Which is why it’s important to consider such exotic cases, but rate them for what they are – exotic. When someone brings up a basic use case that is well within the capabilities of the security product to restrict, rate that as a basic case that will be among the first to be dealt with as the system is introduced. As the system matures, then the more mature cases can be considered.

I deal with NAC in my role, so I see the range of use cases all the time in my meetings with customers. Block a PC that isn’t part of your firm? This is not difficult to do. Block someone spoofing the MAC address of a printer? Well, that’s more than a basic task. I have to ask how we can tell a legitimate printer apart from a spoofed device. If there is no way to tell, then we have to ask if it’s possible to treat all printers as outsiders and restrict their access. This is where maturity comes into consideration.

Maybe we just proceed forward with the PC use case and think some more about that printer issue. Perhaps once we have the PC use case dealt with, there may have been time enough to set up an SNMPv3 credential to use to log on to legitimate printers. Maybe there was enough time to determine how to set up printer VLANs and restrict them. If so, then we’re ready to deal with that printer issue. While we’re doing that, we could be thinking about how to handle the security camera issue, or something like that.

Each environment will have different levels of maturity for their use cases. Perhaps at one firm, it is easier to deal with securing PCs than it is with MacOSs. At the next one, they could have a better handle on their MacOS management than they do with PCs. Maturity could simply be deciding between equally-difficult tasks about which one will be done first.

Maturity can also be seen in calling out when a use case goes beyond the capabilities of the product under consideration. A proxy server does not provide its own physical security system, for example. So, if we entertain scenarios in which physical security is defeated, we should be tabling those until we’re looking at a physical security system. By the same token, if for a scenario to be plausible another security system has to be defeated, then that begs an argument about the safeguards and durability of the system that has to be defeated, not the one under current consideration.

We also see maturity in getting different systems to work together. Being able to automate responses from one system to another gives firms the ability to deal with increasingly advanced threats. All the while, as long as we keep a perspective on how mature our security systems are, we know what level of threat we can deal with.

Auditing Firewalls

Leave a reply

There’s an old Robert Frost poem, ‘Mending Wall’, that I’d like to pirate draw inspiration from and make a few adaptations to, if you don’t mind…

Auditing Firewalls

Something there is that doesn’t love firewalls,
That opens the ports, many and varied,
And spews out the code in plain text in prod;
And makes gaps even two can pass abreast.
The developers’ work’s another thing:
I have come after them and made repair
Where they have left not one single port blocked,
But they would have the code loaded straight to prod,
To please the yelping dogs. The gaps I mean,
No one has seen them made or heard them made,
But at spring audit-time we find them there.
I let my neighbor know in the next cube;
And on a day we meet to read configs
And set firewalls between us once again.
We keep firewalls between us as we go.
To each open ports that have opened to each.
And some are ranges and some are in groups
We have to use a spell to keep them all closed:
‘Stay where you are until our backs are turned!’
We wear our fingers rough with scrolling down.
Oh, just another dull video game,
I call out the new insecurities
There where it is we all need those firewalls:
Where contractors connect to prod boxes
Where file servers sit, shares all exposed
To outsiders’ eyes. And we accept risk.
He just says, ‘Good firewalls make good neighbors.’
Spring is the mischief in me, and I wonder
If I could put a notion in his head:
‘Why do they make good neighbors? Isn’t it
Where they segment traffic?’ But no segments,
No zones define our flat, inner network
Contractors here mixed with outsourcers there,
Aren’t firewalls and segments for those neighbors?
Something there is that doesn’t love firewalls,
That wants it down. I could say ‘Scrums’ to him,
But it’s not scrums exactly, and I’d rather
He said it for himself. I see him there
Auditing a rule that’s permit all all
The CISO told him to accept the risk.
He moves in darkness as it seems to me,
Not of woods only and the shade of trees.
He will not go behind his CISO’s saying,
And he likes having thought of it so well
Once again, ‘Good firewalls make good neighbors.’

Saturday Morning Music

Leave a reply

Sometimes, my wife sleeps in on Saturday mornings. I know that she can have trouble sleeping some nights, so I like to do quiet things that allow her to keep sleeping. This is one of those times.

Today, I’ve got my headphones on and I’m playing through songs that remind me of her. It’s not hard to do, since just about any love song can make me think of her, even if the words have nothing to do with our situation. It’s the passion in the music, I guess.

But I do love her. I woke this morning and gave thanks for a catalog of wonderment she’s brought to my life, and I know it’s not a complete list.

I’m going to listen to another song, and she’ll be in it, smiling back at me between the notes.

Dragnetwork

Leave a reply

The story I’m about to tell you is true. The names and incident specifics have been changed to protect me from violating my NDA agreements.

This is the network: the RFC 1918 ranges. I work here. I’m a security vendor.

It was a cold November day at the customer site when I walked in for the workshop. I met the security architect in the lobby. Nice enough guy, I guess. His name was Ram Gopal. We exchanged pleasantries and headed to the conference room. 

Once we were all plugged in and Ram fired up my product’s GUI, we got underway. I was there to do one thing, and that was to answer the question, “What is this?” for every device on the network. Network visibility, that’s my stock in trade, and it’s an endless, glamourless, thankless job that’s gotta be done.

I know Jack Webb said that about being a policeman. Well, I’m Dean Webb, so I can say that about being a security professional.

The Windows devices were easy enough to figure out. Thousands of endpoints with TCP 135, 139, and 445 open. We passed over those. We also skipped the TCP 515 and 9100 devices: printers or print servers, for the most part. Ram’s eyebrow went up when he saw switches and routers with Telnet still open, but we knew what those things were. He’d write the email to the network team later on. 

He’d write that email later because we were now at the end of the line, the Skid Row of the network. All the IoT devices plugged in by every Tom, Dick, and Harry at the company. The devices dangling off of D-link hubs, just like ID cards hanging off of branded merchandise. The gear left behind by long-gone consultants. The things people plug in without ever thinking. And the heartbreak that comes from not thinking. No thinking at all about security, about personal information, about known vulnerabilities, about default passwords. They’re just plugged in, given a server IP address, and then forgotten about, left for someone else to worry about. 

The first one we looked at had a normal, unassuming IP address. 10.2.44.63. Nobody ever expects trouble from 10.2.44.63. It’s the IP address next door, the all-American kid with a freckled face and a country smile. We look at that IP address and think nothing of it. Well, I’ve got a news flash for you, friend: you never trust an IP address that has port 80 open. It could turn out to be who knows what – a botnet control server, a pivot to the rest of the network, an exposed database, a key to the kingdom, your kingdom, and you won’t be king much longer with devices at 10.2.44.63 having that port 80 open, for anyone to stop by and look at.

I said to Ram, “Let’s put that IP into the browser. See what comes up.”

Ram had to ask, “Hey, I thought you were on the blue team, Dean?”

“Let me set you straight, Ram. I’m on the blue team, through and through. I’m not a penetration tester – coding was never my bag. But when it comes to devices that are serving up port 80 like a dealer offering that first, free hit of dope, that makes my blood boil. My red blood, if you catch my drift. And what kind of blue team player would I be if I didn’t know how the red team was going to come at me? What if I didn’t know about my blind spots, where some punk with a buffer overflow could give me and everyone on this site a really bad day?”

“OK, OK, we’ll see what comes up.”

A colorful page with a vendor logo is what came up, complete with a pair of boxes where a username and password go. Like a reflex action, I typed in that vendor name and “default password” into a search engine. Did I feel lucky? 13 years as an IT guy, do you think I was going to feel lucky? With search engines serving up sponsored pages ahead of the results I really wanted? No, I didn’t feel lucky. I felt smart, and went on to a page of results.

I saw the link to the quick start guide – may as well have been called the lazy hacker’s cheat sheet. And there it was on page 3, the default credentials. Admin/admin.

Ram typed those in and hit enter. The browser wheel spun, and he was in. “So, this looks like the admin page for some system.”

I pointed at the link on the left that read Badge Reader Status. “Looks like your badge reader system.” Then I pointed at the link to Employee Access Database. “And that looks like where the fun starts.”

Ram clicked the link to the database. We saw employee names, phone numbers, usernames, and their access behavior for the last 30 days. I may have said this was where the fun starts, but Ram’s face told a story of pain, disappointment, and betrayal. He said, “Hey, Dean, I need to put a pause on this for about 20 minutes.”

“You need to have a quick meeting with those badge reader people.”

“Yeah, you got that right.”

“Do what you need to do, Ram. I’ll be here.” I wasn’t going to leave his side. It may not have been his first wide-open system, but that didn’t matter. All the years I’ve been a security professional, it’s never been easy. We laugh, we act tough outside, but deep down inside, we’re all feeling that pit in our stomach open up as we wonder how badly that access has been abused in the past. Worse, we know that somebody in operations somewhere is using an app a developer threw together that uses that very vulnerability we just found in order to get his work done, work that makes money for the company. And when it comes down to shutting down a vulnerability or making money, who do you think is going to win out, a lone security architect or a whole operations department? 

That’s why I stand with my customers. That’s why I document my findings. I may only type 40 words per minute, but those are 40 more words every minute that make this world a little bit safer, a little bit more worth living in.

Ram came back from chewing out the badge reader team and I had another IP address with that HTTP port open. This one was the very important-looking 10.122.37.1. Ram put the IP into his browser and said, “That’s supposed to be a perimeter router IP address. That’s our Rancho Cucamonga location.

“I didn’t know you had port 80 open on your routers.”

“We don’t. We turned off HTTP on every one of them.”

“Do you use the same vendor for all your routers?”

“Yeah, we’re a dedicated shop.”

By this time, the web page for the device had come up. “So you guys are a wall-to-wall Netgear shop?”

Ram glared at the Netgear home router login page. I was on the search page, typing in Netgear home router. The second autofill line offered up the other two keywords I needed. The next page gave me all the info I needed without needing to go to a quick start guide. “Try admin/password, Ram.”

One admin/password later, and Ram was on the Rancho Cucamonga perimeter router.

“You need another 20 minute break, Ram?”

“Please?”

“Sure thing, pal. I’ll be here.”

“Wait, before I go, can you tell me how many more Netgear boxes I have on my network?”

“Sure thing.” I applied a filter for Netgear MAC addresses. “You got 21, all with .1 addresses.”

“Can you email me that list?”

“You betcha.” Ram got his meeting together and I sent off a spreadsheet export from my product’s GUI.

There may have been 21 home routers on that list, but Ram only needed 10 minutes to tell a very interested network team the information they needed to know to shut down a Netgear ring that had been a thorn in their side for years. Every one of those IP addresses was one they’d try to get to work in their RMM tool, but their network credentials never worked on them. Now they knew why: they weren’t going with the first or second password everyone guesses when trying to pop a box for the first time.

I was glad that the network team was on Ram’s side, but I didn’t envy the arguments ahead of them. I was betting that these routers hadn’t been a problem before, and that was going to be a problem for convincing concerned parties that they were going to be a problem right now, or an even bigger problem in the future.

Ram came back from his short meeting and said, “You know, comedy works best in threes.”

“Well, maybe the laugh we get from this one will make up for what we see on the next two.”

Ram laughed uneasily. I had already set up a view with plenty of bad news in it. He asked, “What’s that you have there?”

“Well, Ram, these are Windows devices that are members of your domain.”

“OK, that’s not a shock.”

“These have RDP open.”

“RDP?”

“Port TCP 3389 itself.”

“Oh yeah. Sorry, but RDP is also used as an acronym here. I got confused.”

“I understand. Anyway, these stood out because I needed to ask if you have any offices in China.”

“No, we’re strictly in the USA.”

“Nothing in Belarus or Russia?”

“No.”

“Republic of Vietnam? India? Turkey?”

“No, none of those. What are you getting at?”

“Those are just some of the nations with source IP addresses hitting these boxes on port TCP 3389.” I showed him the network traffic view that told the whole sordid tale. 

“I gotta shut down the firewall on that port.”

“Try also the commercial ISP connection at those sites. And then look for the /32 routing statements that send traffic bound to those other nations through the dual-homed Windows boxes with RDP open and exposed to the Internet.”

Ram left the room and I knew he had another impromptu meeting to conduct. I did a little click work in my product and found the IP cameras for this building. Every one of them was open on port 80. On the fifth one I tried, I got the live feed of what Ram was doing in the other conference room with the Windows team. I had kept the other browser windows open so Ram would see that I didn’t even need a default credential to tap into every security camera in his enterprise.

What else did I find? The usual suspects. Xboxes and Playstations. Unpatched web-connected television sets. Printers that responded to the “public” SNMP community. Every iDRAC port that answered to “Calvin”. Nearly every other customer of mine had these devices on their network, and nearly every other customer of mine had a workshop where I called these out as security risks. And even though there was plenty of gore on the network, they thanked me for what I did, because I was on their side. I was fighting the good fight, right there with them, and I was damn glad that they were fighting right alongside me.

But on every network, there’s something new, a little adventure you never wanted to be on, a dragon you haven’t seen before that you nevertheless had to slay. This time the sucker punch came from a little PC on the network with the unassuming name “BURGER_WAGON”.

“So, Ram, what can you tell me about Burger Wagon?”

“Um, that’s a food truck that comes by about 3 times a week. They set up near the cafeteria.”

“So would it be reasonable to assume that a PC named BURGER_WAGON would be theirs?”

“They left a PC plugged into our network?”

“It’s online right now.”

Ram checked his watch. “How about we go get some lunch now, Dean?”

“That’s a great idea, Ram.” We grabbed our jackets and headed out to the cafeteria. If we were lucky, we’d grab an unauthorized device before we grabbed something to eat.

We went up to the Burger Wagon table. I said, “I’m Dean Webb from $VENDOR and this is Ram Gopal, security architect here at $COMPANY. We’d like to ask you a few questions, if that would be all right.”

The Burger Wagon lady said, “Sure, I don’t mind.”

“We noticed that there was a PC named BURGER_WAGON connected to the network. Would you know anything about that?”

“Oh sure, I leave that here so it’s easier to set up when we come in.”

“Uh-huh. And this PC, what is it used for.”

The Burger Wagon lady answered like what she was saying was no big deal. “We process credit card payments on it.”

Poor Ram nearly buckled at the knees with that statement.

The Burger Wagon lady asked, “What’s wrong with Ram, there?”

“He just found out that his cafeteria network is subject to PCI-DSS regulations, that’s what’s wrong.”

“What does that mean?”

If only the cafeteria staff knew what that meant, they wouldn’t have let BURGER_WAGON connect to the LAN. Lecturing the uninformed user wasn’t going to make my job any easier, so I laid it out plain and simple, without judging. “It means that we have to treat this place like a bank processing credit cards. It’s a sensitive environment, with your PC plugged in like that.”

“Oh! I’m sorry! I didn’t know!”

“We’re not angry ma’am. We just want to get the word out so that we can get things on the straight and level around here. If there’s another way for you to connect to the Internet that doesn’t involve using this network, I’d advise you to do so. We are going to start blocking access to devices like these in the very near future. We don’t want to stop you from doing business, just to stop doing it in a way that fails to comply with corporate regulations here.”

The Burger Wagon lady understood and switched over to a guest wireless connection, then and there. She fired up a VPN and Ram got the starch back in his legs. And, you know, we went back to slogging through the unsecured devices on that network after lunch, but we had an upbeat feeling about it. There was a big mountain to climb, but at least there were good people like his network team and that Burger Wagon lady that wanted to do the right thing. That didn’t just make our job easier. It made our job doable.

Racist Words

Leave a reply

Not all racist words have to specifically invoke racist terms. They can be simple, everyday words, but used in a context and with a tone that makes them hateful. If a large group of people takes offense at simple, everyday words, then those words were racist. If you don’t think those words were racist, then you will want to check yourself.