It's time to start writing about my personal challenge this year: telling
#SecurityStories. My goal is to help people gain new insights when it comes to all things security; a very essential topic that's unfortunately often dismissed in favor of our own convenience. I have to admit, I dismissed security concerns way too often myself, and unfortunately I still catch myself doing so. I want to change this. Behavior change starts with awareness, and that's a big part of this story as well.
Let me share my experiences creating a threat model for the very first time on my own. It was
Dan Billing who introduced me to threat modeling in his "Web Application Security" tutorial at
Agile Testing Days 2018. The next time I heard about this approach was only last year at
TestBash Manchester.
Saskia Coplans gave a great talk about it (check it out my
sketchnote or even better the
full recording on the Dojo if you have a pro license by any chance) and we also did an example together during open space.
The idea to start my challenge with a story about threat modeling came from one of the security testing sessions I had together with
Peter Kofler beginning of January. He asked me if I knew anything about threat modeling and I shared with him what I learned at the conferences. To paraphrase him: "I already learned something from you: threat modeling and why it's important, why testers would like to learn it." Remember, the desired outcome for my personal challenge is that ten people have confirmed that they learned something new from me in the area of information security. You can imagine how happy I was to hear that feedback from Peter even before really starting out on this journey.
All this gave me the idea for my first #SecurityStories experiment. It took me some time until I could finally start it in March, yet it was a perfect match with an opportunity I had at work.
I believe that creating a threat model for our own product will result in applied knowledge and surprising findings.
I know I'll have succeeded when I discovered an unknown attack vector for our own product.
For a long time developing my team's product, security was not our biggest concern. After all, we are building an internal application and the little security testing we did was mainly focused around access control and permissions. Now I took the chance to do a more structured risk analysis when it comes to security by creating a threat model. Better late than never!
Building on the knowledge I gathered at conferences, I started with research and reading up on several resources. Here are the ones that helped me most when trying to understand the main steps of creating a threat model.
According to these resources, there are two important things to consider when starting out. First, it's strongly suggested to create the threat model with a group of diverse people to end up with a holistic picture: the whole development team, a business analyst, an architect, whoever adds a new perspective. My team, however, was suddenly finding themselves in crisis mode and fully focused on other topics. Therefore, I decided to start this learning journey on my own, creating a first model version, then involving a few people to refine it, presenting our findings to the whole team and continue refining as we go. Better starting imperfectly than not at all.
Second, the resources agreed that no tool itself is required to create a threat model, and a whiteboard might be the best medium for this group discussion. Over the last weeks, my team also had to learn how to work fully remote, full time. Therefore, I considered creating a digital version already in the first step. This would also help me adapt the model as I learned more without having to redraw it each time. I was curious if there would be tools specifically designed for threat modeling. I found several, yet most of them are not maintained anymore. Besides
OWASP's Threat Dragon, which is still under development yet already offers a native client for both MacOS and Windows. I decided to give it a try and see if it would fulfill my needs. I wasn't disappointed. Using Threat Dragon did prove worth it. This application is great as an idea and already provides lots of features. Granted, it's still under development, and I could feel it. For example, usability is not yet too great. I'd love to have the option to add comments or other descriptive text fields to the model. I'd love to be able to select multiple diagram items and move their position all together. Still, I haven't regretted this, and I especially appreciate the nice report this tool provides.
Let me be clear about it. I created a threat model for the very first time, and I very well might have gotten things wrong, or done in a way that's not recommended. As it still proved to be an exercise very well worth its time, I'm just going to tell you what I did and how things went. Due to the nature of the thing, I cannot show you my results - yet OWASP provides several examples in their
Threat Model Cookbook so you can get an idea of what I'm talking about.
Why threat modeling and how does it work?
Threat modeling is a structured way to brainstorm about threats. It's important to consider as many factors as we can think of from diverse perspectives to get a holistic view. For example, we also need to consider malicious acts from within the company, or simple human error. Keep in mind that a model is never correct as it does not represent the full reality, yet models help us think - in this case about security.
Security is a quality attribute that needs to be built into the system from the start. We want to improve and include security concerns early on into how we develop, test and run our services. On the one hand it's about making our systems less prone to human error, on the other hand it's about not leaving the door wide open and make things hard enough for attackers to become less attractive as target. When it comes to the latter, some might think that our internal product does not provide valuable information, that there are more attractive targets in the company; yet we simply cannot tell for sure what is valuable and what not. For example, we could grant people access to other systems in the company through our services without being aware of it, our server resources could be misused for other activities, or we could lose all our data by mistake.
From what I learned, the threat modeling process is basically about the following questions and steps.
- What are we working on?
- What can go wrong?
- What are we going to do about it?
- Did we do a good job?
What are we working on?
As we cannot easily tell how attackers think or what's valuable to protect, the most common approach is to have a look at what we are building and identify the data flow between the different parts. To create such a data flow diagram, we used the following common notation.
- Process (circle): Our services and code.
- Store (two parallel horizontal lines): Data stores (e.g. files, databases, shared memory, message queues).
- Actor (rectangle): External entities, everything but our code and data. This includes people and cloud software.
- Data flow (pointed arrow): Connects processes to other elements.
- Trust boundary (dotted line): Indicates where the trust level changes (e.g. from internet to intranet through a firewall, from web server to database, from our application to an external third party service). There is always a trust boundary when your code goes to someone else’s, or their data comes to your code. We perform security checks only inside our trust boundary.
Before starting my first version of the diagram, I looked up earlier architecture drawings we had created to get inspired. I then started listing assets like stores, actors and processes. Doing so, I had to become clear about above standard notation. What is used for what again, which category would fall that one in? Looking at the threat model examples helped. I quickly noticed that this is very iterative work. I jotted down everything that came into my mind: the services we own, the integrations we have to other systems, the trust boundaries we operate in.
As with all kind of visualization techniques, I realized once more the power of modeling. It really helps you think! Just doing this was a great exercise also for testing and quality in general, not only security focused. It really helped to get the overall picture again; especially as our product landscape and its complexity grew heavily over the last five years. Just a little internal application? Far from that with all infrastructure included and so many things to consider. I realized a thorough model would really take time, yet each step on this endeavor was so much worth it. I got to know our system better again, realized more clearly how things are connected and where they could break, and created a better understanding of our architecture components and how traffic actually goes through them. It was great to have that model not only in my head, but visually "on paper" so I could align it with my teammates and discover any discrepancies or unknown risks.
With more and more components added, the diagram grew and grew. It slowly felt like an overwhelming task. I wondered how much detail should be added to the data flow? Where was it okay to simplify the model, where would exactness help? It's a model after all and it's main purpose is not to depict absolute reality but help us think. Still, it's not easy to decide whether I should add all kind of requests and the used protocols, or abstract this. Should I add the kind of data flowing, down to single entities? That would let the diagram explode. Maybe rather keep the flow generic? In the end, I just decided on one way as a proposal and left the rest for future iterations.
What can go wrong?
Based on the diagram, I started to identify threats. To do so, STRIDE is the most common approach for threat modeling. I've found a good
overview on STRIDE that I'll copy here to get us on a shared page.
- Spoofing
- Property Violated: Authentication
- Definition: Impersonating something or someone else.
- Example: Pretending to be any of Bill Gates, Paypal.com or ntdll.dll
- Tampering
- Property Violated: Integrity
- Definition: Modifying data or code
- Example: Modifying a DLL on disk or DVD, or a packet as it traverses the network
- Repudiation
- Property Violated: Non-repudiation
- Definition: Claiming to have not performed an action.
- Example: “I didn’t send that email,” “I didn’t modify that file,” “I certainly didn’t visit that web site, dear!”
- Information disclosure
- Property Violated: Confidentiality
- Definition: Exposing information to someone not authorized to see it
- Example: Allowing someone to read the Windows source code; publishing a list of customers to a web site.
- Denial of service
- Property Violated: Availability
- Definition: Deny or degrade service to users
- Example: Crashing Windows or a web site, sending a packet and absorbing seconds of CPU time, or routing packets into a black hole.
- Elevation of privilege
- Property Violated: Authorization
- Definition: Gain capabilities without proper authorization
- Example: Allowing a remote internet user to run commands is the classic example, but going from a limited user to admin is also EoP.
While I was still extending the data flow diagram, I was already taking note of any threats that came to my mind. All this was quite an iterative process, while continuously learning more. I heard several people describe threat modeling as dull - yet I found it quite interesting, there's a lot to discover using this structured way to think about these kinds of risks! When pondering on the diverse potential threats and ways to mitigate them I learned a lot about them. My biggest question in this step was how many threats to list? Only those I know are critical, or all I can think of? In the end I decided to go for the latter in favor of a more holistic picture, while being well aware there would be a lot more that I am even not aware of yet.
Now was the time to involve my team and get initial feedback on the model. I invited my colleague who knows our infrastructure best for a call and walked him through what I've done so far. I asked him to double-check the diagram and the initially brainstormed threats. Admittedly, I was quite anxious to hear his feedback. You can imagine my relief when he confirmed that I had created a model that fit to both our shared mental models, with only a few minor adjustments that I gladly worked into the diagram. He shared with me that he has never seen a threat model before and found it very useful to think about risks. What a great entry point for risk discussions indeed!
Step by step I went through all of the resources again and continuously extended the model, adding anything I missed. With each iteration I discovered something new, something else to consider, another potential threat. I have a lot more ideas what to check for and I am far from finished refining the model. There's so much more to look into, like checking the tech stack we use for known vulnerabilities, cross-checking with the
OWASP Top 10 security risks, considering social engineering attacks, and more.
What are we going to do about it?
Identifying threats is not enough. We also need to decide what to do about them. For each individual threat, we have the following options at hand.
- Remove the threat (e.g. removing the respective functionality)
- Mitigate the threat (e.g. through standard practices like encryption)
- Accept the threat (be careful about “accepting” risk for your customers)
- Transfer the threat (e.g. via license agreements or terms of service)
For each threat I was aware of, I made a first assessment, or rather educated guess. I was happy to involve our product owner in this step, presenting him the whole picture. He was intrigued seeing our whole service landscape visualized and recognized that it grew a lot more over the years than he had realized. He asked further valuable questions, and also helped assess the threats from his point of view.
At this point I decided it was time to document the current state in our wiki and invite the team for a presentation. I wanted to get everyone introduced to threat modeling and our current model version, including all assumptions it was built on. We had a short session, and promptly I got further invaluable feedback! More pairs of eyes instantly caught what we missed before, and also detected a flaw in the visualized data flow. Perfect input to refine our model further.
The more I learn, the more I know what I don't know yet. There are so many more things to think about, yet having this model is a great discussion base. We're far from done - yet a big step further.
Did we do a good job?
Finally, we need to validate that the identified threats have actually been handled.
We haven't done this yet for all identified threats. There's still a lot more to do indeed. Yet again, the effort already proved worth it. I know a lot better what to look for, also with each new change we're implementing. My hope is, that my team does know that better now as well, and we all use this increased awareness to find good solutions together.
A Living Model
In the end, our threat model is supposed to be a living document. As our socio-technical system changes, this model will change as well. There are several triggers for a revision, like the following examples.
- We develop a new service or remove a previous one.
- The architecture of one of our services changes.
- We introduce a new technology.
- The infrastructure conditions change.
- The knowledge and skills of our development team grow.
- External actors and their interactions with our product change.
I'm curious how threat modeling will help us in the future, as it already helped us in the present. Our awareness increased, we can make more informed decisions together when it comes to security. That's a big step for us indeed. When it comes to anything else, only time will tell.
All this was based on an experiment. Could I prove my hypothesis and identify a previously unknown attack vector for our product through threat modeling? The answer is clear: yes, and more than one. We have work to do.
One more question remains. Having read this story, have you learned something new around information security that you weren't aware of before? If so, please leave a comment or
write me a direct message on Twitter. Have fun with threat modeling!