Monthly Archives: September 2016

Open for Business

Vernon Washington punched the call button for a fleet car. Per regulations, he set his watch for atmospheric sampling. Planes were on fire, fuel dumps had been hit, who knows what else was fouling up the air? External drives in the pockets, camera in the contact lens, radio in the earpiece, everything else was ready for gathering information.

Vernon stepped out of Terminal D and into the waiting fleet car. “Datacenter, evasive.” Debris everywhere, smoke hovering over the eastern terminals, psyops staff walking around with man-portable loudspeakers, alarms sounding, fire and emergency crews everywhere… the only thing missing from the scene were the screams of the mourners. Vernon wasn’t in that response crew, though. Those sights were for someone else’s nightmares.

The car made its way deliberately to the datacenter building. It was almost totally new, shining in its energy-efficient, up-do-date architecture. Vernon made a silent bet with himself about how many old problems were simply moved from the old DC into the new that were involved in this breach. He was pretty sure there were thousands of problems, but how many were involved in today’s disaster? Vernon counted on his fingers… five.

The car pulled up to the curb. Vernon got out and the car went to go park itself. A guy with a DFW staff badge was there to greet Vernon. “You the guy with [REDACTED]?”

Vernon tapped the badge above his left shirt pocket. “I’m a federal agent. Are you my escort?”

The guy went from cocky to sheepish in a flash. His name badge read “Edwin Lu”. He badged in and held the door for Vernon. Vernon rolled his shoulders and walked up to the reception desk. “Do you need me to sign in?”

“No, we’re just coming up to my office.”

Wrong answer, Edwin. Vernon stayed by the desk. “I wasn’t really asking. Where’s the visitor ledger?”

Edwin smirked in puzzlement as he produced a ledger. “You’re not auditing us, are you?”

“No, I’m not. But you probably should expect one very soon in light of today’s events. Security is all the rules, all the time, documenting when they’re bent or broken.”

Edwin’s expression indicated that the business culture here hadn’t been stressing security for at least some time…

As they approached Edwin’s cube, Edwin grabbed a chair out of a conference room. “This is more comfortable.” Vernon was thankful for the comfy chair, but felt a little uneasy about how the “Do not remove chairs from conference rooms” sign was ignored. Still, he only expected five problems for this breach.

“OK, Edwin, do you use a RADIUS server for authenticating your wireless devices?”


“Let’s take a look at the configurations. See if there are any new entries on the MAC bypass list.”

“OK…” Edwin started up a console to look at the RADIUS server. “Uhm… how will I be able to tell if the entries are new? They’re all sorted alphabetically.”

“How about a change log?”

“Um, OK…” Edwin clicked on Tools > Security > Admin Log.

The screen filled up with times, dates, usernames, and changes. Edwin and Vernon leaned forward and squinted. As they read, another log entry popped up at the top of the screen. Vernon asked, “Do you have circular logging enabled?”

“Ah… well, I dunno.”

Vernon assumed that meant yes. “Copy all the admin log files to a backup directory. Now.”

“Well, we do backups every night at 3 AM.”

“This is different. Copy them now. As in now.” Vernon didn’t want to say NOW: it was better for the working relationship if he didn’t go all caps on the guy. “It’s for forensics.” Vernon felt better when he added the why.

“OK then, just a sec.” Edwin went to the directory on the RADIUS server where the logfiles were kept and did a CTRL+A CTRL+C move and then did a CTRL+V to copy them to his local PC. “Yeesh. This is gonna be a while.”

“True. But now we have a copy of them from this time.” Vernon looked at the three newest entries in the logfile. They were identical, each 90 seconds apart. Unable to reach device at Most likely a switch or wireless controller that had been deactivated long, long ago and nobody bothered to tell the RADIUS server. “Edwin, any way we can filter those out?”

“Well… I only know how to find stuff in this interface, not unfind them.”

“All right then, page down. We gotta read this over until we know what we’re looking for.”

“Why not check the SOC for unauthorized access events?”

“Because I’m betting dollars to donuts this is authorized access.”

“What, one of us did it?”

“Keep it down, Edwin. I’m not accusing anyone. I have no data, for starters.”

Page down. Page down. Page down. Page down. Those 90-second intervals really pile up, don’t they?

Hang on… “OK, highlight that.” Vernon pointed at a line on the screen that had nothing to do with Edwin clicked on it, putting a nice blue tint on the text. The text noted that WANNA.SAMUE added a few addresses to the MAC bypass list.

The voice said in Vernon’s ear, “We’re getting it just fine. Maintain distance.” Good, the camera was working.

Edwin asked, “Sam did this?”

“Who’s Sam?”

“One of the security admins. Sam Wannamaker. That’s his account.”

“OK, noted. But let’s not jump to conclusions. That’s his account, probably wasn’t him. Look at the timestamps on those events.” Those addresses were added around 6:15 AM, last Saturday. “This guy Sam, when does he usually work?”

“9 to 6, like most of us. We didn’t have any changes scheduled for Saturday.”

“Is he in today?”

“Yeah, you want him?”

“Not yet, what’s the IP of where Sam logged in from?”

Edwin scrolled to the right on the logfile display. “That’s our jump box for DC access.”

“OK, we need to check the event log on that box for where someone logged in with Sam’s account.”

“You want to do that now?”

“Yes, now. Can you hit that box from here?”

“Sure, just a sec.” Edwin fired up an RDP session to A little while later, he had the event viewer up and filtered for logon events. 6:15 on Saturday showed that WANNA.SAMUE logged in from

“Hold the screen there, sir.” Vernon awaited the voice in his receiver.

“That’s the University of Zagreb Computing Center.” Thank you, voice.

Chances were, Sam wasn’t in Croatia over the weekend. And whoever was in Zagreb or connected to a device in Zagreb, that was for the people next to the voice in the earpiece to resolve. Vernon was here to document what had gone on at DFW. For that, he asked Edwin, “Do you guys remote in to this jump box normally?”

“Yeah. Makes it easy for us.”

“Do you VPN in for it?”

“Well, no, not always. Our VPN’s been really unstable for the last, like, year… and we don’t always want to have to drive in to do work.”


“So it’s opened up on the firewall.”

That was one. Sam’s account was two, dollars to donuts. “Let’s go see Sam. He sit near here?”

“He’s two rows over.” Edwin led the way. When they arrived, “Hey, Sam, this is…”

“Vernon Washington.” Let Edwin give the rest of the info.

“Vernon Washington, a federal agent. He’s here investigating, the, uh, thing today.”

Vernon smiled. “Hi Sam. I want to get directly to the point. Can we take a look in your email?”

Sam was too confused to be scared about that question. “Umm, OK.” Sam brought up his email client. “What do you want to look for?”

“Can you search for emails with links in them?”

“Ummmmmm… yeeaaaaaah… yeah. Here we go.” Sam typed the filter into the search box. Tons of marketing emails popped up in the results.

“We need to look at all of these, from before this last Saturday morning. Say before 7am.

“OK.” Sam’s cooperation was pretty natural, not typical for a suspect. Which made sense, since Vernon didn’t suspect Sam the man. Just Sam the account.

The procedure was straightforward: look at the link in the email. Ask Sam if he clicked on it. Hover over the link and see if it goes to where the email claimed it would go. If nothing noteworthy came up, move on to the next email. As it turned out, Sam ignored almost all of the marketing stuff. Lots of looking, lots of scrolling…

Then there was the email from Rhonda, the group coordinator. Sam had clicked on the link and the hovering mouse said it was to an IP address that was nowhere inside the company.

The voice in the earpiece said, “Nothing there now, but it was in Argentina.”

Vernon counted the third problem. No spear phishing training. Or if there had been training, Sam here was in the 1% of computer users that training had no effect on. Sam had clicked on the link, provided a credential, someone used it to try the RDP box open to the Internet, got in and set up the MAC addresses of the grenade launchers to be permitted on the wireless network… and this jump box would also be a likely point of origin for the signals sent to the passenger vans and grenade launchers alike.

Two more openings to find.

First, Vernon collected pertinent files on his external drive. As he made the copies, he asked, “Who’s in charge of the passenger vans?”

Sam and Edwin looked at each other. Sam said, “Facilities?”

That wasn’t going to get anywhere. “How about the IP range for the vans?”

Sam clicked around and brought up the IP management interface. A few more clicks and he had the answer. “”

Vernon asked, “How about doing an SSH to an address in that range?”

Sam tried. He got a connection refused error message.

Vernon groaned inside. “Try telnet.”

When that made a connection, Vernon asked Sam, “Do you know the username and password to use?”


“Try admin/admin.”

Sam typed and got in. Everyone felt ashamed that it had worked, and on the insecure telnet protocol, to boot. Vernon figured whoever was able to send commands to the vans didn’t even have to try – just being in the area would allow anyone to get an unsecured copy of everything sent to the vans. Not just the default, unchanged username and password, but also the commands used to maneuver the vehicles. Pretty darn handy.

And that default credential set was problem number four. One more to go, and that would be no limitation on what devices could send commands to the vans. Obviously, that was wide open.

There wasn’t much more Vernon could do. He made some small talk with Sam and Edwin, handed out cards, asked them to contact him if they had any more informa- say, the lights were flickering.

Then they went out. The air conditioning also cut out. But the computers and monitors didn’t. Vernon made a guess that the power wasn’t cut – something else was getting messed up.

Edwin asked, “What the hell’s going on?”

Vernon made a guess. Given the state of security there, it was a pretty good guess to make. “You guys got licensed hardware?”


“Well, check your licenses. Betcha someone’s zeroed them out. You really need to change those default admin passwords.” Vernon figured he’d gather some more data while he was here. It wasn’t his first license blasting case to investigate, that was for sure…


The man opened his laptop and entered his password. His hard drive spun and programs flicked back on. The laptop re-established its network connection – wired only, the man didn’t trust wireless – and packets began to flow between his PC and the rest of the world. One consequence of that traffic was a notification that he had new email. The man noted that, while he had 12 new emails in his inbox, he had 2 in his “Action Items” folder.

As he was about to open the folder, he heard a crash of dishes from the kitchen. Without getting up, he demanded, “What is going on in there? Is anything broken?”

“Maddie opened the dishwasher too hard!”



The girls continued to argue as the man minimized his email and went into the kitchen. His voice was probably too stern for the occasion, but the man was under pressure. He had action items to address. “Get the dishes done, and get them done quietly. I am very busy and I don’t want any noise. Maddie, be more careful when you open the dishwasher. Laney, you are the older sister, so you should keep a better eye on Maddie and help her more.” The girls were about to cry. The man’s heart softened. “I’m sorry, I shouldn’t have yelled like that. I love you all. Let’s hug.”

And so, they hugged. Maddie, Laney, and the man resolved their issues through reassuring human contact and then went back to work. The girls on their dishes, and the man on his action items.

He first opened a text file. Then he opened the action item emails. In turn, he copied the contents of the emails and pasted them into the text file. Then he deleted the emails. Then he emptied out his deleted items folder. The man knew that this wasn’t a complete deletion of that information, since a digital ghost of it existed on his local hard drive, in addition to whatever the [REDACTED] picked up in its [REDACTED] program. And, since the emails came from Minsk, there were other agencies besides [REDACTED] that would have their copies.

But the data in motion on the Internet and the data at rest was encrypted, so the man knew that nobody would try to break into it unless it was on somebody’s radar, and that wouldn’t be until someone put the pieces together to a very difficult puzzle. After all, it wasn’t against the law to receive emails from Minsk.

That was the fun part about the United States legal system. The whole thing was built around either catching someone in the act of committing a crime, or amassing enough evidence to prove that a criminal act had been committed by a particular criminal. Just as corporations were more efficient at doing business than single proprietors or partnerships, they were also more efficient at committing crimes: no single person did anything that, of itself, was a crime. Instead, the actions of dozens of people had to be connected in order to demonstrate a pattern of behaviors that produced criminal activities. But could the law catch those people? Or did it want to keep to the easier crimes?

The man laughed to himself. Wall Street got the King’s Pass to perpetrate financial crimes on a grand scale, while those mom-and-pop operations, be they corner grocers or corner meth labs, got crushed by legal regulations and the big boys alike.

The man knew he was part of a big operation. He just didn’t know what it was. He liked doing the work. Criminal operations tended to be very libertarian and very agile. He felt empowered to make decisions, was glad his compensation was 100% salary, and had access to the best tools money could buy. The man didn’t need to submit expense reports but did so, anyway, as part of his cover story. The best part was that the cover story was no cover at all – he really [i]was[/i] an IT security consultant that worked from home.

There was the matter of who, exactly, the employer was. The man did not know and did not care. It was like the Algerian FLN. The man got messages from one source and sent his messages to another source. Given the level of obfuscation between the sources, the man felt it highly unlikely that he would meet the same fate as the FLN in Algiers after the French forces broke into the movement’s structure and methodically tracked down each cell.

Time was money. The girls had finished their post-lunch chore and were watching purple dinosaurs engaging in situational ethical discourses with red furry monsters or something like that. The man returned to his task.

The text file showed a list of IP addresses with notations beside them, a handy comma in between the addresses and the comments, in case he needed to view the information as a spreadsheet. The man just liked the text file because it loaded faster.

The information came from the boys in Minsk that scanned and probed IP address ranges. They asked no questions and desired no answers. They just ran their NMAP scans and followed up where they found interesting things, like open RDP ports or SMTP relays, both of which were of interest to the people that had employed the man to use that information.

The man was involved because some people were interested in employing someone with very good English language skills to send emails to some native English speakers. Since the man was both a native English speaker and in possession of an email client, he was a perfect fit for the job. The man also knew a thing or three about how to customize search strings and gathering intel from social media networks.

The man started to scroll through faces and resumes of men and women that worked at the two airports mentioned in the action items. Open RDP ports at DFW and LAX meant his employers would gain remote access to IT systems at those airports if they knew the accounts and passwords to use. Brute force attacks would fail, generate alerts, and generally lead to undesired consequences. The man disparaged such methods, as his were far more elegant and productive.

And that’s where the SMTP relay came into play. Thanks to small businesses constantly starting up, there was an infinitely regenerating supply of unsecured email servers that would allow anyone accessing them to impersonate anyone else with only a minimal knowledge of how to configure an email client. Yes, it could also be done from a command-line interface, but the man needed to send rich content with links and documents – it was a total pain to try and cobble those together in a command-line environment. The man hated programming and wanted to be as far from it as possible, preferring to send his carefully-worded emails from a GUI. It was simply more elegant that way.

As the girls shifted from animated philosophy to that damn game with the irritating soundtrack, the man tried to block the annoying tune from his consciousness as he looked over org charts for DFW staff assignments. The link to those PDFs had been deleted, but not before Google found it, indexed it, and indexed the document so linked, which was still open to the Internet even if the page that once linked it was now a 404 page not found…

And there she was! The man had the name he needed. He highlighted it, pressed CTRL+C, went back to the text document and –

– he saw his wife pulling into the carport. She was back from the grocery store, so the man knew he only had seconds. He clicked at the end of the text block, hit CTRL+V, then comma, and then “admin asst dfw”. CTRL+S saved the info and Windows+L locked his laptop.

The man got up to open the door for his wife, who had two handfuls of plastic bags. She said, “There’s ice cream on the back seat. And milk. Get that first.”

Once the groceries were in, the man went back to his PC while his wife put away groceries and got the girls started on sorting laundry. A password later, and he was ready to get started on his background research for his first email. Rhonda Emerson had a number of promising interests, wine tasting the most promising of them all. It was most promising because the man already had a bogus wine tasting club website set up, along with websites that dealt with beers, cigars, whiskeys, chocolates, travel, running, golf – all the vices. Funny thing was, a username and password to get into one would get into all of them, since they all had the same database driving them. The man didn’t mind. They were only there to gather usernames and passwords.

The best part was the follow-up email. That potentially gave him one of the most important pieces of information: the business email signature of his target. If the target didn’t put a sig on replies, he had another ruse to get the target to send a new email, but most people had a sig on every email.

Rhonda Emerson must have been thinking about the weekend, because that email and account info showed up awfully fast. The man copied and pasted the sig into a draft email that was going to bounce off the relay and into the inbox of one of the people that she served as an administrative assistant/coordinator to.

Hello Ryan,
Harvey Wright would like us to update his SharePoint with all the accounts we use for access to jump boxes, network gear, servers, etc. This is part of the Integrated Account Management initiative. The link to the SharePoint is here.

Kind regards,
Rhonda Emerson
IT Group Coordinator

The man had composed this email in a second email client. In it, he specified the SMTP relay as his server and Rhonda’s address as the “to”. The man didn’t care about the replies. He just wanted the info to be sent to the SharePoint that was set up on a typosquatting website, and hoped that the admin would fall victim to the spear phishing.

Just to make sure, the man copied and pasted the email body to several other emails, each going to a member of the team that Rhonda supported – taking care to edit the name after “Hello”. The first one to submit the info would be the winner.

As things turned out, Samuel Wannamaker was the most prompt at supplying the information. He just posted the spreadsheet that he kept with all the system names, IP addresses, and shared accounts for getting into them. Thank you, Samuel.

The man got that info at 4:23. The wife leaned into the open doorframe and asked, “You almost done in there?”

“Just wrapping up a few things, hon.”

“You want to pick up something for supper? I’m tired.”

“Sure, what do you want?”

“Food. I don’t care. I’m going to go lay down for a while.”

“OK, I’d like to snooze a little myself before going out.”

“How about Chinese?”

“Sounds good.” The man started to drift back to work.

The wife moved into the foyer. “I’ll go ask Laney and Maddie what they want.”

As the wife asked the kids, the man already knew the answer. Chicken fried rice for Laney and beef lo mein for Maddie. He copied and pasted Samuel Wannamaker’s spreadsheet into an email from his first email client and sent it to someone who was interested in usernames and passwords for systems at DFW airport. The man didn’t know what exactly what was going to happen with that information, or the information he’d already collected for Atlanta or for the information he was about to collect for LAX. The man just planned on not flying anywhere for any reason for a few months.

The man responded to a few more emails and then watched a cat video on YouTube. Life was good, working for people that liked to collect usernames and passwords.