Layering On the Security Cheese

In the world of cybersecurity, we like to harp about defense in depth. In this blog post, I’d like to offer a more memorable phrase.

Don’t get me wrong, defense-in-depth is a perfectly fine phrase: it is precise, it’s not too buzzword-y, and it encapsulates a lot of security best practices. You should absolutely be practicing defense in depth.

But it’s also a bit abstract. And you know what isn’t abstract (for me, anyway)? Cheese.

On a recent pentest, I presented a report with many overlapping mitigations, to a team that had not been through a security assessment before. In software, the focus is on often on efficient and elegant solutions, which doesn’t quite jive with how cybersecurity practitioners think.

I wanted to frame the rest of the report from the perspective of defense-in-depth, so I included the cheese analogy.

I’m far from the first person to make this cheese comparison, but months later, one of the team members brought it back up (unprompted), so it must have stuck.

In short: “layering on the security cheese” means that you should have multiple, overlapping protections and mitigations for anything you are trying to protect.

Why So Many Mitigations, Though?

Before we get to the cheese, let’s talk about why one singular mitigation is (almost always) not enough in cybersecurity.

Look: we all want certainty in our lives, or at least in our engineering projects. We don’t want to go for a solution that’s guaranteed to work 60% of the time, every time. Some of us have actual budgets and schedules to adhere to.

So why can’t cybersecurity professionals just offer a complete solution to any given security issue and be done with it?

Compared to engineering or software development, defending these systems is a much different challenge.

Rather than building to a spec, you are trying to defend against attackers who may or may not exist, and who may or may not have the skills, tooling, or resources you expect.

You might have some semblance of what type of attacker would be after your system: financially motivated ones go after companies with money, politically motivated ones go after organizations with personal data or idealogical leanings. And of course the really basic attackers are just pulling endless crypto scams and unexpectedly bold social engineering attacks against seniors.

But past that, what do you know about them? Are they skilled? How much time do they have on their hands? Is this their day job or are they just putzing around for the lulz?

Even where you do have visibility, like logs, there’s still uncertainty:

• Are there shadow IT assets we don’t know about?
• Are we monitoring everything we need to, or are we constrained by memory limits (such as on embedded systems products) or money (everyone)?
• If someone recovered keys from our IoT product and made requests, could we tell the difference between that and a legit device?
• Are those payloads in our logs ineffective AI slop, or do we just not have the latest intel on $vendorName exploits?

No one knows! Or at least no one willing to hand over that information in a timely manner for free (threat feeds cost a very pretty penny).

In a perfect world, yes, we could have perfect visibility, but not in this one with limited budgets and project deadlines.

Even in a world where you did know all the details, software packages go out of date, or get updated (to new issues), or new techniques are published.

Everything’s computer, and everything hurts.

The average time from CVE to exploitation is 5 days, with the fastest (publicly known) times being a couple of hours. AI isn’t helping the situation for attackers, either.

So yeah, you gotta layer on those mitigations.

Visualizing Defense In Depth With Cheese

That finally brings us to cheese.

Each layer of Swiss cheese represents an imperfect protection or mitigation we can add to our system. The holes represent remaining opportunities for an attacker to exploit a given feature or vulnerability.

Let’s look at a few different examples and picture how defending-in-depth helps us defend against constantly evolving (yet still not-quite-known) threats.

Protecting a Login Form

First up, login forms. Sensitive company data, PII, account takeovers, and more sits on the other side of your app’s authentication mechanism, so it needs to be good.

You’ve heard about SQL injection (SQLi), so you make sure the login SQL queries use sanitization and/or parameterization. Maybe your library or framework even does it (auto-magically) for you!

We see some failed SQLi attempts show up in the logs, and feel pretty good about ourselves.

But what’s this?! Our attackers, realizing that the ’00s have come and gone, are employing other methods to guess, bypass, or brute-force logins.

Here’s some more layers we can add in response to attackers trying to guess or brute-force logins:

• Rate-limiting in the form of CAPTCHAs or a web application firewall
• Increasing password complexity so that it takes more time to successfully brute-force a password
• Providing generic error messages to prevent username enumeration: moving from “no such user found” and “wrong password” to “the username/password combo is incorrect”
• Maybe some attacks are still slipping through, thanks to reused passwords or other issues, so we add multi-factor authentication

Even to the best of our cheese-layering abilities, some attacks might still get through, as attackers get even craftier, possibly using toolkits or social engineering techniques to bypass MFA.

Now what? Believe it or not: more cheese. Just different cheese this time.

NIST has the concept of “5 functions of cybersecurity”: Identify, Protect, Detect, Respond, and Recover.

When we think of protections and mitigations, we’re often focused on the “Protect” portion. But we can also layer on defenses in the form of detecting and responding.

We might want to add logging and monitoring that gives insight into what hacking attempts are happening, and if there’s any anomalous activity to investigate.

We can also add email alerts to this example, which allows users to respond to unexpected logins, and reset their passwords.

By layering on a lot of defenses, we’ve upped the difficulty level significantly for our potential hackers, and we’ve got mechanisms to respond to evolving threats (even ones we didn’t initially anticipate).

Remediating XSS

We can also use the security cheese idea when the attack method is known and is specific.

In some cases, a single fix is enough to fully patch the vulnerability. But even when the reported vulnerability had a single root cause, it’s still a good idea to implement overlapping protections in response.

Imagine that you receive a pentest report informing you that one of your webpages is vulnerable to cross-site scripting (XSS). The report then tells you: sanitize user input (good advice by the way, but not sufficiently cheesy).

You add XSS filtering, hopefully via an established library. But what if the (undoubtedly dedicated but overworked) open source maintainers of filtering libraries missed something? What if you missed other parts of the app where user input fields are vulnerable to the same issue?

Here we might add:

• Content Security Policy headers, which dictate which types of content (like scripts) can be executed or displayed, and from which sources. If a XSS payload manages to get past your defenses, it would also need to bypass the CSP to work.
• Escaping any tags before displaying content to the user. Yes, we’re already filtering on the way in. Let’s filter on the way out too, to be doubly sure that XSS payloads aren’t being added from other parts of the app.
• A web application firewall (WAF) which can prevent attackers from attempting to submit XSS payloads in the first place.

Protecting Your IoT/Telematics Firmware

Let’s switch gears for a bit. Say you’re developing an embedded product, and you’re worried about hackers (or competitors) getting a hold of your device firmware.

They might reverse engineer it to develop an exploit, embarrass you in a Defcon presentation, or steal your intellectual property. Yikes.

Making your device difficult to purchase, such as in the case of specialized equipment, isn’t enough.

Not unlike the web examples, there’s many overlapping protections you can put in place to raise the difficulty level for an attacker, and lower the usefulness of accomplishing their goal.

You might also:

• Close debug ports (USB, UART, JTAG, etc) that were used by engineers during development
• Enable memory protections to prevent firmware from being read out
• Encrypt firmware at rest, such as on external flash chips
• Use encryption in transit: make sure that firmware updates are downloaded over secure channels, so that it’s not as simple as watching web traffic on the same network.
• Limit the window of time that a firmware update can be retrieved, such as with time-limited pre-signed URLs and/or do updates automatically so the timing is not known in advance to attackers.
• Encrypt firmware downloads so if they are intercepted (or are freely available on your website), they aren’t much good without the keys.
• Limit the blast radius (a topic for an upcoming blog post!) by avoiding hardcoded passwords and keeping keys unique to each device. If an attacker recovers a password/secret in the process of retrieving firmware, limiting the blast radius means they can only use it to impersonate that one device.

Securing Protecting FOTA Updates

The previous section has overlap with this topic, including encryption in transit and limiting time windows, allowing us to “double dip” on some previous efforts.

But when we look at the firmware updates through the lens of integrity instead of confidentiality (from the CIA triad), there are even more protections to add.

We don’t want attackers to have the ability to control firmware updates, because it could let them modify device functionality for themselves or others, in a malicious way.

Some layers for this task include:

• Encryption in transit and time-limited windows as mentioned before.
• Using signed updates that are validated by the microprocessor (preferably in hardware) before being installed.
• Securing the OTA portal or mechanism by which those updates are deployed. Changing default passwords, enabling 2FA, and limiting the group of people with access.
• Regularly updating libraries that are support the above functions (for encryption in transit, etc.).
• A robust system of peer review and QA testing within the company, before the FOTA update gets deployed.

As you can imagine, there’s many other areas within embedded systems to protect. As we’ll discuss later in this blog post, you can identify and prioritize these areas via threat modeling.

Threat modeling is how we can more easily see all attack angles for a given asset, and how you might expand from “keep the firmware out of attacker’s hands” to other scenarios like FOTA, manufacturing, and so on.

Preventing Social Engineering

Layering on the security cheese works in other domains too. Within the corporate world, layered defenses against social engineering attacks might look like:

• Email filters to keep out suspicious emails, and a system for employees to report phishing attempts
• Training for employees
• A company culture that prioritizes slowing down and double-checking weird requests
• Least privilege on company devices and accounts in case someone does get hacked
• Backups and policies to recover from a successful phishing attack

Same with personal security:

• Unique passwords for each account, so you aren’t vulnerable to credential stuffing attacks sourced with credentials from the various data breaches happening every week
• Storing those passwords in a password manager that has good security posture
• 2FA
• Keeping your devices physically secure, and updated
• Freezing your credit
• Your credit card company monitoring your activity for fraud (and you regularly reviewing bank statements, too)

Much like literal cheese, thinking in layered defenses can help in a variety of domains.

The Goal of Adding More Cheese

All of these cheesy defense-in-depth examples are trying to achieve the same two ideas:

1. Prevent any one single point of failure, and
2. Raise the bar of difficulty against unknown attackers

We still don’t know everything there is to know about our product’s potential attackers, but we’ve done enough to make exploitation difficult, and we also set up alerting mechanisms in case something does slip by our defenses.

By adding “more layers of cheese”, your development team is raising the difficulty of exploitation such that lower-skilled attackers will be ineffective and higher-skilled attackers may be deterred by the difficulty or level of effort required.

Defense in depth is a different frame from the building side of engineering or software. Rather than a singular elegant solution, it looks more like overkill, as we try to cover all the known edge cases, as well as the unknown ones.

How Do I Determine Which Assets or Scenarios to Defend?

While any amount of security (aka defense in depth) will help make your product more secure, it’s best to frame it relative to the protection of assets, or workflows, or the mitigation of specific reported vulnerabilities.

That’s why the web examples were focused on specific areas, and not just “secure the damn web app”.

Likewise, the embedded examples were specific to different portions of how an attacker might get access to firmware. That’s one crucial part of a much bigger picture of “securing embedded devices”.

Otherwise, if you take a 10,000 foot view, you’ll likely come up with a list of many security… but it won’t give you a comprehensive view of how secure the system is overall.

If I were to extend the cheese metaphor a bit too far, this is like not knowing how many burgers or sandwiches you have to put cheese on.

So how do you identify those assets, workflows, attack paths, and so on?

The single best answer is threat modeling.

Threat modeling is a systematic way of looking at what you’re trying to protect, how it might be hacked, and determining the fallout of each scenario.

Note for the project managers reading this post: yes, there are budget and time trade-offs involved, preventing us from indiscriminately cheese-ifying our products. Threat modeling helps here too, giving you a ranking of scenario likelihood and damage, so you can sprinkle your limited cheese in the most useful places.

But that’s a topic for another blog post, and I think this cheese metaphor has been stretched far enough.

In the meantime, whether you’re designing a new feature or patching a vuln (or just making dinner), remember to add more cheese.