A Universal Experience
If there's one task that can send shivers down the spine of any and every technical individual, it's being told to "write documentation." It's tedious, it's boring, and it doesn't scratch the problem-solving itch that our other tasks do. Unfortunately, documentation is one of the most important and vital resources we have at our disposal as technical professionals. It can teach us how to use things, and sometimes more importantly, it can explain why things were built the way they are. It makes technology accessible, it makes teams more cohesive, and it makes processes resilient. To say it's a necessity would be an understatement; it's the lifeblood of any technical program. So why does it have to be so painful?
The Problem: Not All Docs Are Equal
Now that we've established how great documentation is, let's talk about some documentation that isn't so great. While documentation can apply to many things in the tech industry, we here at Rilevera want to focus on documentation as it relates to Detection Engineering. After all, the theme of this blog is Effective Detection Engineering, so from here on out, any mention of documentation will be confined to the space of Detection Engineering.
Let me share some experiences from a former role. When I worked for an MSSP, documentation lived in a few different places. This isn't inherently a bad thing. Documentation living in separate places for different teams can make sense, since Engineers wouldn't need to know the internal processes used by HR or Finance in most cases. At this MSSP, however, it was a bit of a problem. Let me elaborate.
We more or less ran SIEM platforms for our customers. We kept them running, we developed detection rules for them, and we responded to the alerts those rules created. That last part is key, because it was ultimately a partnership between the MSSP and the customer security teams. Any time an alert fired, we had to let the customer know, and we had to communicate any changes to the detection logic that resulted from our investigations. This is where the trouble started. All of the communication about our investigations and updates to detection logic happened through a ticketing system. That meant if someone wanted the full picture of why changes were made to a specific Detection Rule, they would have to sift through dozens, sometimes even hundreds, of tickets to understand the context behind those changes and the current state of the rule.
On its own, this was more of an annoyance than an immediate problem, since the customers we worked with had incredibly smart and talented technical resources who were able to retain that context. But what happens when those teams scale, or when the individuals holding that contextmove to another team or organization? That's when the real problem emerges. Now the scattered context forces the new technical resources to spend time hunting for the invaluable information they need to understand the nuances of an alert and how best to respond, costing the organization precious time and effectiveness in its responses.
The Beginning of a Solution
Centralizing the documentation is clearly part of the answer, but it's not the full solution. We always have to account for the human element of these technical problems. The issue only gets worse when we consider that not every change is documented the same way. Some individuals perform more thorough investigations and provide more detail in their escalations and changes to the detection logic than others, leaving us with further problems to solve.
This is where a real solution starts: applying a framework to all of the documentation. We use the Alerting and Detection Strategies (ADS) Framework, a structured approach that breaks a detection down into a consistent set of components so that every rule is described the same way. There are several schools of thought here, and everyone has their own opinion, but one thing we can all agree on is that the team needs to settle on a framework and follow it. That consistency ensures every update to a Detection Rule carries the same information, organized the same way, every time.
Piecing It All Together for an Effective Solution
Rilevera was built with these problems in mind. Adopting Detection-as-Code doesn't end with having a repo and a working pipeline for your Detection Rules. Truly operationalizing Detection-as-Code in an effective manner requires every change to be documented, centralized, and kept consistent across every Detection Rule.
Rilevera applies the ADS Framework to our Detection Rule documentation so that every rule carries an audit trail showing how each update affects:
- Threat Description
- Impact
- Sensitivity & Severity
- Blindspots & Assumptions
- Detection Goal
- Detection Functionality
- MITRE ATT&CK alignment
Every update to the documentation goes through a change approval process to ensure it complies with the ADS Framework and that everyone who needs to know about the change is informed. This documentation is centralized for every single rule, so you're never looking in more than one place for the context you need to respond to threats or update your detection logic.
That's the difference between sifting through a hundred tickets to reconstruct why a rule looks the way it does and opening a single, structured record that tells you in seconds. It's documentation that actually does what documentation is supposed to do: keep your team fast, aligned, and resilient, no matter who's holding the context today. The necessary evil, finally made a little less evil.
