Reference project · German defence sector
How an early-adopter customer took the IdoubleS platform from beta to operation on-premises, with a validation plan, a baseline built by both teams, independent raters and a number on every step. Including the first result, which missed the target.
The customer
A federal IT service provider in the German defence sector, commissioned in 2023 to have a Cyber Threat Modelling platform built and operated on-premises. Threat intelligence arrived daily, in volume and in natural language. Refining it by hand into bespoke models was essential and did not scale; before the project, no threat modelling was practised at all.
The objective
attack graphs with at least 80 % average accuracy against a baseline built jointly by IdoubleS and the customer's analysts
six automated runs per report, compared technique by technique and relation by relation on both layers
the customer's analysts, three independent experts for detection logic, SVA's SIEM specialists and penetration testers for the native rules
the team that generates a graph, a pattern or a rule never declares it valid
Tender: provision of a threat intelligence platform, built and operated on-premises.
Test scenario 1: relevant adversaries and threat scenarios prioritised from business parameters and events.
First automated runs against the baseline: about 61 % (Emotet delivery) and under 50 % (Ryuk ransomware). Below target, measured per technique, and said so.
New release: context building, image analysis, technique classification, extended entity recognition, threat-actor intelligence, confidence pipelines.
Re-validation with the same baseline and runs: 99 % and 84.17 % exact technique match; 96.14 % and 93.57 % relation quality on the observable layer; up to 51.85 % additional correct techniques the analysts had missed.
Test scenario 3: 14 STIX detection patterns scored by three independent experts on six criteria: 82.47 %.
Test scenario 4: 11 native SIEM rules scored on seven criteria: 90.47 %. Adversary emulation by SVA: 11 of 11 rules raised an offense, 9 complete, 2 partial. Three techniques needed a log source the lab did not have, reported as such.
Next: attack vectors, attack paths (the bridge between adversary graphs and critical functions), the customer's integration environment.
Results
1
prose, tables, code, images
99 %
of baseline techniques, plus ones the analysts missed
82.5 %
quality, behaviour-based, portable
90.5 %
quality in the customer's SIEM
11 of 11
under adversary emulation
From author to reviewer. Analysts validate instead of capturing reports by hand; confidence scores steer a sample check. One quality criterion of the plan had to be rewritten during the project, from “effort to rework AI-generated graphs” to “effort for quality assurance”, because the AI had found what the analysts had missed.
Detection logic and native rules from the validated model, coverage per technique, mapping documented. Every re-validation reuses the same baseline, rubrics and test scenarios, so coverage is proven again after every change.
The flood of CTI reports used to be impossible to handle by hand. Today the IdoubleS platform refines it into a model we trust: traceable to the source, scored with confidence, validated by analysts, and at a quality and speed that were unthinkable before.
Results of a defined-scope pilot, 2025–2026. Scenarios from CrowdStrike Intelligence Tippers CSIT-22052 and CSIT-21260, used with CrowdStrike's approval. Evidence for the method, not a benchmark.
Contact
Turn the reports and feeds you already receive into a governed Cyber Threat Model that your SOC and risk and management teams can act on. Tell us where you want to start.
