Catalog design review

Why each type is shaped as it is. Catalog v3 reviewed every catalog v2 type against the same questions: what real thing it models, whether one thing can get two spellings or two things one key, whether its required properties are what tools actually emit, and whether a merge can clobber a better value with a worse one. The answers live in each type's description, which kb_types publishes and the catalog reference renders. This page records the verdicts and the defects the review found, each with the regression test that holds the fix. Later versions answer the same questions for the types they add, and mark those rows with the version: catalog v5 added advisory and package.

Defects

ID	Types	Finding	Resolution	Regression test
B1	txt_record, spf_record	A malformed SPF value had no home.	A value is diverted exactly when its dedicated type accepts it.	tests/test_catalog.py::test_every_version_tag_spelling_has_exactly_one_home
B2	docs	The scope enumerations omitted mta_sts_policy.	The prose is pinned to scope_relations().	tests/test_catalog_reference.py::test_the_parent_scope_prose_enumerates_exactly_the_scoped_types
D-01	endpoint	The query was removed before validation, so a malformed query passed.	The query is removed only from a URL that validates with it.	tests/test_catalog.py::test_golden_canonicalization_cases
D-02	ip_address, endpoint	IPv4-mapped IPv6 was accepted, differently per interpreter.	ip refuses every IPv4-mapped address.	tests/test_catalog.py::test_golden_format_cases
D-03	port	Port 0 was accepted.	port_number_assigned.1 refuses it; uint16 keeps 0 for SRV and MX.	tests/test_catalog.py::test_golden_check_cases
D-04	asn	AS0 was accepted.	asn_assigned.1 refuses it (RFC 7607).	tests/test_catalog.py::test_golden_check_cases
D-05	spf_record	SPF values had no length bound.	Bounded at 4096 characters.	tests/test_catalog.py::test_golden_format_cases
D-06	secret, exposes_secret	The plaintext-key check was top-level and case-sensitive, and edges were unchecked.	Casefolded, recursive, and applied to the edges; the error names a pointer.	tests/test_catalog.py::test_a_plaintext_key_is_named_by_its_pointer_and_never_by_its_value
D-07	endpoint	URL hosts refused underscore labels that subdomain accepts.	A URL host accepts every subdomain spelling.	tests/test_catalog.py::test_valid_node_fields_and_boundaries
D-08	catalog	Checks, canonicalizations and registries were outside the fingerprint.	Versioned ids and registry digests are fingerprinted.	tests/test_catalog.py::test_renaming_or_versioning_a_rule_id_changes_the_contract
D-09	docs	The tool page claimed kb_types published rules it did not.	kb_types publishes check ids and descriptions, and the page says so.	tests/unit/test_graph.py::test_types_publish_what_each_type_models_and_the_rules_it_runs
D-10	reference	The txt_record rule text was stale.	Rule prose lives beside the rule and is generated.	tests/test_catalog_reference.py::test_the_page_documents_every_published_rule_with_its_types
D-11	reference	The drift test compared 4 sections, and endpoint prose was hand-copied.	The page is compared byte for byte after formatting.	tests/test_catalog_reference.py::test_the_committed_page_is_what_the_generator_produces
D-12	catalog	Cross-field rules ran on nodes only.	Relations declare checks as well.	tests/test_catalog.py::test_relation_checks_run_inside_record_validation
D-13	finding	Title identity collided across matchers and forked on renames.	Identity is (rule, matcher) under the parent.	tests/test_identity.py::test_a_finding_is_its_rule_and_matcher_under_its_parent_whatever_its_title
D-14	formats	cpe23_or_empty was published but unused.	Replaced by cpe23, consumed by technology and runs_technology.	tests/test_catalog.py::test_cpe23_rule_rejects_empty_legacy_uri_and_malformed_bindings
D-15	several	Documented attribute conventions were not validated.	Declared as validated optional properties.	tests/test_catalog.py::test_declared_optional_properties_are_validated_when_present
D-16	technology	The slug has no naming authority.	Kept; a product-level CPE anchors it, and the slug convention is documented.	tests/test_catalog.py::test_golden_check_cases
D-17	ip_address, ip_cidr	version is required although derivable.	Kept; it is filterable and compared by the containment checks.	tests/test_catalog.py::test_golden_check_cases
D-18	storage	observed_at is last-writer-wins.	Resolved in v4: records keep monotonic first_seen and last_seen.	tests/storage/test_graph.py::test_an_older_observation_only_adds_the_properties_a_record_lacks
D-19	finding	nuclei's unknown severity had no home.	Added to the enum, documented as non-ordinal.	tests/test_catalog.py::test_enum_rule_rejection_keeps_the_published_list_spelling
D-20	has_contact	A CAA iodef had no home.	Role iodef with endpoint targets restricted to POST.	tests/storage/test_graph.py::test_a_contact_role_is_checked_on_the_merged_edge
D-21	service	The secure flag the TLS check requires was undeclared, found by the coverage harness.	Declared as an optional boolean.	tests/test_catalog.py::test_invalid_node_types_bounds_and_noncanonical_spellings
D-22	endpoint	The URL grammar refused - in paths and queries, found by the coverage fixtures.	- is accepted wherever RFC 3986 allows it.	tests/test_catalog.py::test_valid_node_fields_and_boundaries
D-23	domain	wildcard_answer was declared on registrable domains, where no source can feed it.	It stays on subdomain only.	tests/test_catalog.py::test_ac10_attribute_properties_are_declared

Node types

Node	Verdict	Why
advisory	new in v5	Replaces cve. One published vulnerability record keyed on its id: CVE, GHSA, or an id from an OSV database that publishes one record per vulnerability. Vendor and distribution bulletins, which bundle many CVEs, are refused. CVSS and the publication time are declared on every id; EPSS and the KEV date only on a CVE, the only id FIRST and CISA score or list. It is never a finding source.
asn	changed	Keyed on the number. AS0 is refused (D-04). The holder label, country and RIR are declared attributes, because RDAP publishes them for the number resource itself.
certificate	changed	Keyed on the DER SHA-256, the one value every scanner can compute. The validity window, subject CN, issuer DN, serial and self-signed flag are declared. Trust-store-dependent flags stay out, because they describe the observer.
cloud_account	new in v3	An AWS account, GCP project or Azure subscription, keyed on (provider, account id). Each provider has one spelling, enforced by a check, because the leading zeros of an AWS id are significant.
cloud_resource	new in v3	Keyed on the provider-assigned default hostname, which names one resource at a time. The 14 hostname patterns are structural and proved disjoint at import. Alias hosts are refused with a pointer to the canonical form.
cwe	changed	Shared vocabulary keyed on the MITRE id, and the join point for scanner-specific titles. name is declared, fed by the CWE catalog.
dkim_record	kept	Scoped to its domain and keyed on the selector, so the node is a current-state view of the key. A rotated key survives in evidence.
dmarc_record	kept	Shared by every name that publishes the same policy string. The TXT diversion guarantees each tagged spelling one home.
domain	changed	Registrable names only, as the bundled PSL classifies them. The wildcard flag is declared; the wildcard-answer flag is not, because a public suffix has no answer to compare (D-23). Registration facts moved to whois_registration.
email_address	kept	A lowercase mailbox, so one mailbox is one node although the local part is case-sensitive on the wire. Person types stay out of scope.
endpoint	changed	One method on one path. The query is validated, then removed (D-01), and - is a path character (D-22). The latest response's HTTP metadata is declared. A secret in a URL path stays in url, an accepted residual.
finding	changed	Keyed on (rule, matcher) under a per-source parent (D-13). title is a latest-wins attribute, so a template rename patches it in place. The severity enum gains a non-ordinal unknown (D-19).
host_key	kept	Keyed on the algorithm and the SHA-256 of the key blob. It is the strongest host-correlation pivot the catalog has.
http_fingerprint	kept	A clustering pivot, never an identifier. The declared kind fixes the value spelling.
identity_tenant	kept	Only providers with one canonical tenant spelling are in the enum. The spelling is enforced per provider.
ip_address	changed	Canonical compressed spelling; IPv4-mapped IPv6 is refused (D-02). Range-list classifications (CDN, WAF, cloud) are declared attributes of the address. version is kept although it is derivable (D-17).
ip_cidr	changed	A canonical network. The RDAP netname, country and RIR are declared.
mta_sts_policy	changed	Scoped because the TXT value is only a version pointer that tenants publish verbatim. The policy file's mode, max_age and sorted mx patterns are declared.
organization	changed	Keyed on (registry, handle), never on a free-text name. name is declared as a latest-wins attribute.
package	new in v5	A package or container image the target publishes, keyed on its versionless purl in one canonical spelling per purl type, so every release is one node. An oci image keeps its registry repository in repository_url, so same-named images in two registries are two nodes. It carries inventory state, but allowed_ownership refuses dependency: a component the target runs is a technology.
parameter	kept	A parameter name and location, scoped to its endpoint. Values stay in evidence.
phone	kept	An E.164 contact. Person types stay out of scope.
port	changed	An open port scoped to its address. Port 0 is refused on the node while the shared uint16 format keeps it (D-03).
registrar	kept	Keyed on the IANA id, which survives renames. Id 0 is refused. It is now reached from a registration, not from a domain.
repository	changed	Keyed on host, owner and name, lowercase. The default branch, visibility, archived and fork facts are declared.
secret	changed	Digest only. Plaintext-bearing keys are refused in any case and at any depth (D-06). kind is optional so that a later write lacking it cannot clobber it, and the harness requires scanners to send it.
service	changed	A registry service name scoped to its port. The version-probe product and version, and the secure flag the TLS check requires, are declared (D-21).
spf_record	changed	Shared by every name that publishes the string. It is bounded at 4096 characters like every TXT rule (D-05).
storage_bucket	kept	Only providers with a global namespace are in the enum. It maps to a cloud_account provider for in_account.
subdomain	changed	A name below a registrable domain. Underscore labels are accepted left of the registrable domain. Wildcard DNS flags are declared.
technology	changed	A shared product slug. Only a product-level CPE is accepted on the node. The per-host version and versioned CPE live on runs_technology (D-14, D-16).
tls_cipher_suite	kept	Shared vocabulary keyed on the protocol version and the IANA name. It is never a finding source.
tls_fingerprint	kept	A stack clustering pivot. The declared kind fixes the length.
txt_record	kept	Generic TXT. A value is diverted exactly when its dedicated type accepts it (B1, D-10).
whois_registration	new in v3	One registry-level registration, keyed on (zone, registry object id) under its domain, so a re-registration is a new node. Placeholder ids are refused. A missing handle writes no node.

Relation types

Relation	Verdict	Why
affected_by	kept	From a service, endpoint, finding or published package to an advisory.
aliases	new in v5	Two advisory ids for one vulnerability. One edge per pair, pointing toward the CVE, then the GHSA; two CVE ids are refused.
announced_by	kept	A network announced by an AS; distinct from operated_by.
authenticates	new in v3	From a secret to what it logs in to, keyed on (username, breach token). One pair in two breaches is two edges. The plaintext check applies.
backed_by_bucket	kept	A name or URL that serves from a bucket.
caa_issue	kept	CAA issue, with parameter names folded because the list is identity-bearing.
caa_issuewild	kept	CAA issuewild, as above.
cname_to	kept	A CNAME record; self edges are allowed.
contains_cidr	kept	Proper-subnet containment, checked on the merged edge.
contains_ip	kept	Address containment, checked on the merged edge.
covers_name	kept	Exact and wildcard coverage are two identities.
dname_to	kept	A DNAME record.
exposes_secret	changed	Keyed on location. The plaintext check now applies to the edge too (D-06, D-12). A cloud resource or package may expose a secret.
federates_with	changed	The realm's namespace_type is declared.
has_contact	changed	Registration roles attach to the registration. A CAA iodef report URL is an endpoint target, restricted to POST (D-20). A package has only maintainer and security emails.
has_dkim_selector	kept	The scope relation of dkim_record.
has_dmarc	kept	Attaches the policy to the name itself, not to a _dmarc node.
has_finding	changed	A registration, cloud account, cloud resource and package may carry findings. Shared vocabulary never does.
has_http_fingerprint	kept	From an endpoint to a response fingerprint.
has_mail_exchange	kept	Keyed on the preference.
has_mta_sts_policy	kept	The scope relation of mta_sts_policy.
has_nameserver	kept	An NS delegation.
has_open_port	kept	The scope relation of port.
has_parameter	kept	The scope relation of parameter.
has_registration	new in v3	The scope relation of whois_registration. It pins the registration's zone to its domain.
has_service	kept	The scope relation of service.
has_soa_primary	kept	The SOA primary server.
has_spf	kept	Attaches an SPF record.
has_srv_target	kept	Keyed on the SRV fields.
has_subdomain	kept	Naming structure, checked on the merged edge.
has_svcb_binding	kept	HTTPS and SVCB records, written only after the SvcParams are parsed.
has_tls_fingerprint	kept	From a service to its stack fingerprint.
has_txt_record	kept	Attaches a generic TXT value.
has_weakness	kept	Finding or advisory to CWE.
hosted_on	new in v3	A name or URL that serves from a cloud resource, typically through a CNAME to its default hostname.
in_account	new in v3	A resource, bucket or cloud-registry image that belongs to an account, with the provider matched on the edge.
issued_by	kept	A self edge records a self-signed certificate.
links_to	new in v3	Crawl links keyed on the referencing element, so an anchor and a script include are two edges.
loads_package	new in v5	From an endpoint to the target's own package it loads. A third-party component stays a technology.
operated_by	kept	Number-resource holding, keyed on the RIR handle.
owns_repository	kept	An attribution claim that needs evidence.
presents_certificate	changed	name_mismatch is declared, because it is a fact of one presentation.
presents_host_key	kept	From an SSH service to its host key.
protected_by	kept	A CDN or WAF in the request path. Range-list classification is an address attribute instead.
published_from	new in v5	From a package to the repository it is published from. Not a reliance relation, so it never blocks a rejection.
redirects_to	kept	Keyed on the redirect status.
registered_through	changed	From a registration, not a domain, so each registration keeps its own registrar.
resolves_to	kept	A or AAAA resolution.
reverse_resolves_to	kept	PTR resolution.
runs_technology	changed	Carries the per-host version and versioned CPE.
serves_endpoint	kept	From a service to an endpoint.
supports_tls_cipher	kept	From a service to an accepted suite.

What is not modeled

Every field a covered source emits has a home or a stated reason, as the ASM and OSINT coverage page shows per field. These are the recurring reasons:

Not modeled	Why
Person and company identities	Out of scope by decision; registrant names, leak-corpus names and addresses are non-storable.
A registration whose registry publishes no id	No whois_registration is written; the RDAP or WHOIS response stays evidence on the domain.
Cloud Run services and Cloud Functions	Neither has a one-resource default hostname; recorded as endpoints and a project cloud_account.
Trust-store verdicts on certificates	untrusted, revoked and expired describe the observer, not the certificate.
Volatile response metrics	Word, line and timing counts change on every request.
Advisory applicability data	NVD references, configurations and descriptions describe the advisory, not an asset.
Per-source first and last seen on records	Records keep one monotonic first_seen/last_seen; per-source times live on evidence.
Keyed secret digests	A per-workspace keyed digest needs a storage change; unsalted SHA-256 is kept for joins.
Cross-corpus breach equivalence	No naming authority maps one corpus's breach ids to another's.