How Null Sec Counts

Every data report we publish counts public threat intel reporting the same way. This is that way, written down once: the units, the windows, the exclusions, the statistics, and the limits that come with them.

Marking
TLP:CLEAR
Type
Method
Author
Null Sec team
Published
· 8 min read
Applies to
Version 1, for every Null Sec data report from 13 Nov 2026
An engraving of a surveyor's theodolite on a tall tripod, standing alone on a flat, empty plain.

Null Sec's data reports measure the public CTI reporting itself: who reported an activity, when, under which names, and whether anyone independent reported it too. They all count the same way, and this report says how. Each data report links here and states any place where it departs from what follows.

Reporting, Not Activity

Everything we publish counts what was written, not what attackers did. A rise in reports about an actor means more was published about it. It can mean the actor did more, or that one incident drew a lot of coverage, or that a vendor launched a blog. Every chart we publish carries the line "Measures reporting, not activity", and every report says which of those readings its numbers can and can't separate.

The same goes for the words around the numbers. We write "reported" and "mentioned in", not "attacked" or "most active".

The Reports We Count

Our data comes from Asterism, the CTI platform Null Sec builds, which collects public research, advisories and indicator feeds from a few hundred sources. A report counts when all of these hold:

  • It is dated by its publisher. An undated item only has the day Asterism first saw it, and a backfilled feed stamped with today's date once looked like a surge. Undated items are left out.
  • It is intelligence. Product news and marketing posts are left out.
  • It is TLP:CLEAR. See "TLP:CLEAR Only" below.
  • Its publisher hasn't opted out of AI use. Asterism honors the opt-out signals publishers send, and our research reads Asterism through its MCP server, its interface for AI tools, so those publishers' reports are invisible to us. They are missing from our counts, not counted as zero.

Round-ups are weekly bulletins, digests, newsletters and industry reports that name dozens of actors in passing. Asterism finds them by their titles: a strong cue such as "weekly bulletin" or "monthly threat report" is enough alone, and a weak cue such as "quarterly" or "threat landscape" counts only when the report also names three or more actors, malware families, tools or campaigns. Round-ups never join a story, and they don't count toward rankings.

Stories: One Activity, Many Reports

Ten outlets rewriting one vendor's post are one activity, not ten. Asterism groups reports about the same specific activity (a campaign, the exploitation of one vulnerability, a malware operation, a patch release) into Stories. In our reports a story, lowercase, is one of those clusters.

Two reports can join a story when they are published within 10 days of each other and share enough distinctive entities: indicators, CVEs, malware, campaigns, victims, or a rare name in both titles. A shared actor name isn't enough by itself, because two reports about one actor's separate intrusions are two stories. Asterism's documentation reports how well this works on 409 report pairs labeled by hand: the clustering joined no pair it shouldn't have (precision 1.000) and found about 92% of the pairs it should (recall 0.924). It calls those numbers optimistic for reporting the rules weren't tuned on, and so do we.

A single report that nobody else wrote about is not a story. When a report's subject is "activities" (the Corroboration Monitor, for one), it says how much of the window's reporting stood alone.

Organizations Are the Witnesses

Independence is counted by organization, not by source. A vendor's blog and its indicator repository are one witness, and so are five feeds run by one project. Asterism records the organization behind each source and counts a story's organizations, not its feeds.

Asterism's MCP server names the organization only for a story's first source. Where we need the organization behind another source, we map it by name and publish the mapping with the data, so you can check it. Each report says whether its mapping agreed with Asterism's own organization count for every story.

First Reports, Corroboration and Lag

Table 1The timing terms every data report uses, and what each one is not.
termwhat we meanwhat it isn't
first reportthe earliest publication day (UTC) of any report in a story, among the sources Asterism collectswho discovered the activity, or when it began
corroborationa report in the same story from a different organizationconfirmation that the first report was right
lagthe day of the second organization's first report in the story, minus the day of the first report, in whole daysa measure of anyone's speed at finding the activity
same daya lag of 0 daysa tie, never a lead; and never "zero-day"

Publication dates are whole days. When a publisher's feed gives a time, it rarely says which time zone it meant, so we don't order reports within a day.

Some stories are still waiting for a second organization on the day we pull the data. Dropping them would make waits look shorter than they are, so we keep them as censored cases: we know they waited at least that long. Lags are estimated with a Kaplan-Meier survival curve, which counts each waiting story until the as-of date and no further.1

Primary and Passing Mentions

A report can be about an actor, cover it at length, or name it once. Asterism records which, for each actor, malware family, tool and campaign a report names:

  • subject: in the title, or named in the lead and again, or at least 35% of the report's mentions of named entities;
  • substantive: under a heading, across two or more paragraphs or table rows, or named three times;
  • passing: the rest.

Rankings and "most reported" lists count primary reports only: subject or substantive, and not a round-up. A report that names an actor once in passing is not a report about it.

Names and Who Uses Them

Vendors name the same intrusions differently. Asterism resolves every alias to one entity, so a query for one name finds the reports that use any of them. Where a report says how often each name appears, it searches for the name's exact text instead, so a count per name is a count of reports that wrote that name.

We credit a name to the vendor whose scheme it belongs to only from a public source:

  • MITRE ATT&CK's group records, which cite where each alias comes from;2
  • Microsoft's published map of its actor names to other vendors' names;3
  • a naming scheme a vendor publishes and no other vendor uses: Mandiant's UNC numbers, Microsoft's Storm- numbers, Unit 42's constellations, CrowdStrike's animals.

A name none of those credits is listed as "not credited". Clusters that share a name, or members, or tools, overlap. We write that two clusters are the same actor only when the vendors who own the names say so, and we cite them.

Coverage and the Panel

Asterism only knows what it collected. A source added today arrives with about the last 45 days of its history, so the corpus is complete only from a date Asterism computes: the day from which the sources carrying 80% of its reporting were all being collected. Every window in our reports starts on or after that date, which the report gives as "coverage since". Anything earlier would measure Asterism's own ramp-up.

Sources are added over time, so a rising count can be Asterism growing. When a report compares periods, it uses shares, or only the sources collected through the whole span (the panel), and it gives the panel's size at both ends of its window.

The panel leans toward English-language security vendors, because that is most of what is published openly and most of what Asterism collects. A region or sector covered mainly in other languages is under-counted.

Uncertainty and Small Samples

  • Shares come with a Wilson 95% interval, which behaves well for small samples and for shares near 0% or 100%.4 We write them as "38% (95% CI 31 to 46%, n = 162)".
  • Waits come from the Kaplan-Meier curve, with Greenwood's variance on a log-log scale;5 the median wait is where the curve crosses 50%, and its interval is where the curve's band crosses 50%.6
  • Typical values with no censoring are medians, with a percentile bootstrap interval from a fixed seed, so you get the same interval we did.7
  • Many tests at once are corrected for false discoveries with the Benjamini and Hochberg procedure, and a report says how many things it tested.8
  • Under a sample of 30, a number is labeled a small sample and the interval matters more than the point. Under 5, we don't compute it at all.
  • Each headline number is recomputed with one reasonable alternative (a shorter window, or the largest contributor left out). The report says whether it held. If it didn't, the headline is the range.

When a number depends on what Asterism extracted from text (an actor, a technique, a sector), we read a sample of the reports behind it and give the error rate in the report's method.

TLP:CLEAR Only

Everything in a Null Sec data report comes from TLP:CLEAR reporting or from public datasets. That includes the aggregates: a count drawn partly from TLP:GREEN reports would be TLP:GREEN-derived, so the key our research uses can't read anything above TLP:CLEAR, and every query also asks for TLP:CLEAR explicitly. The reports Asterism holds under other markings are not in our numbers.

Open Data

Each data report publishes, beside its text:

  • the CSV behind every figure, and data.json with every number the text quotes;
  • queries.aql, every query in Asterism's query language, with its window and the date it ran;
  • sources.csv, every report cited or counted by name: title, publisher, date, URL and marking;
  • public.csv, every public dataset used (CISA's KEV catalog, FIRST's EPSS, CVE records, ATT&CK) with the day we fetched it.

You can check the arithmetic from the CSVs alone, and anyone with Asterism can rerun the queries. Public datasets change, so a rerun on a later day can differ; the fetch dates say which day ours came from. We don't republish report text: a quote is a sentence at most, credited and linked.

Our Interest

Null Sec sells Asterism, the platform these numbers come from. Every data report says so in its "About the Data" block. That is one reason the queries and data are open: you shouldn't have to take a vendor's word for a number about the field it sells into. We don't publish rankings of named vendors, and we don't claim Asterism saw anything first.

Claims We Don't Make

  • A count of reports is not a count of attacks. See the first section.
  • "First in our panel" is not "first discovered". It is the earliest report among the sources Asterism collects, from publishers that allow AI use.
  • Reported together is not related. Two names in the same reports are co-mentioned. A relationship needs a report that asserts it, and we cite that report.
  • A sector or country mentioned is not a confirmed victim. Reports name sectors for many reasons; we write "mentioned".
  • Corroboration is not confirmation. A second organization can repeat a first report's mistake.

Versions and Corrections

This is version 1 of the method, for reports from 13 Nov 2026. When a definition changes, this report gets a dated entry saying what changed and why, and the version number goes up. A series edition names the version it follows, and the first edition under a new version says how the change moves its numbers, so editions stay comparable.

A correction to a published number is dated in that report's corrections, with the old number and the new one. We don't change a published number silently.

References

  1. Kaplan, E. L. and Meier, P. (1958). Nonparametric estimation from incomplete observations. Journal of the American Statistical Association 53(282), 457-481. doi:10.1080/01621459.1958.10501452 doi.org/10.1080/01621459.1958.10501452 ↑
  2. MITRE ATT&CK, Groups. attack.mitre.org/groups attack.mitre.org/groups/ ↑
  3. Microsoft Threat Intelligence, threat actor naming map (MicrosoftMapping.json), microsoft/mstic on GitHub. github.com/microsoft/mstic github.com/microsoft/mstic/blob/master/PublicFeeds/ThreatActorNaming/MicrosoftMapping.json ↑
  4. Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association 22(158), 209-212. doi:10.1080/01621459.1927.10502953 doi.org/10.1080/01621459.1927.10502953 ↑
  5. Greenwood, M. (1926). The natural duration of cancer. Reports on Public Health and Medical Subjects 33. London: His Majesty's Stationery Office. ↑
  6. Brookmeyer, R. and Crowley, J. (1982). A confidence interval for the median survival time. Biometrics 38(1), 29-41. doi:10.2307/2530286 doi.org/10.2307/2530286 ↑
  7. Efron, B. (1979). Bootstrap methods: another look at the jackknife. The Annals of Statistics 7(1), 1-26. doi:10.1214/aos/1176344552 doi.org/10.1214/aos/1176344552 ↑
  8. Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B 57(1), 289-300. doi:10.1111/j.2517-6161.1995.tb02031.x doi.org/10.1111/j.2517-6161.1995.tb02031.x ↑
TLP:CLEAR Share freely, subject to standard copyright rules.Corrections: none so far

Spot an error? Write to us and we'll fix it here, with a dated note.