Overview
This topic provides a detailed description of data formats found in Data Export files created by the AWS S3 Data Export and Data Export features.
Format
Data is exported and uploaded in JSON format and separated into paginated files for each type of top-level object.
Files with full in their name contain a complete upload of all data. Alternatively, files with delta in their name contain only data that has changed since the last scan (e.g., new findings, changed values).
Note: Delta uploads for external issues is not currently supported.
| Object | Description | File Names |
|---|---|---|
| Assets | Anything that can have a vulnerability correlated with it. This object contains data for all assets. | asset-data-FULL/DELTA-TIMESTAMP |
| Asset Findings | Anything that can have a vulnerability correlated with it. This object only contains data for assets with findings associated with it. | assets-finding-data-FULL/DELTA-TIMESTAMP |
| Asset Groups | Flexible collections of assets that allow you to organize your assets across your entire organization. | assetGroups-finding-data-FULL/DELTA-TIMESTAMP |
| Findings | Unique top-level vulnerabilities that can exist in assets. | 'Vuln' and 'Vuln-Compliance' type findings: findings-finding-data-FULL/DELTA-TIMESTAMP'Compliance' and 'Vuln-Compliance' type findings: compliance-findings-finding-data-FULL/DELTA-TIMESTAMP |
| Instances | When a finding is detected in an asset, that makes it an instance of that finding. If the same vulnerability is discovered in 5 assets, there will be 5 instances of that vulnerability. | 'Vuln' and 'Vuln-Compliance' type findings: instances-finding-data-FULL/DELTA-TIMESTAMP'Compliance' and 'Vuln-Compliance' type findings: compliance-instances-finding-data-FULL/DELTA-TIMESTAMP |
| Resolved Instances | Resolved finding instances. | /instances/instances-resolved-finding-data-FULL/DELTA-TIMESTAMP |
| Justifications | Justifications track the changes (e.g., a status change) to instances over time and subsequent scans. Any time there is a change to a finding (manually or via scan) there will be a justification record for that change. | /findings/justifications-finding-data-FULL/DELTA-TIMESTAMP |
| Teams | Groups of users that collectively own assets and vulnerabilities in Nucleus. These Teams can then be linked to assets to define asset ownership, and then be further linked to vulnerabilities to define vulnerability ownership. | /teams-finding-data-FULL/DELTA-TIMESTAMP |
| External Issues | External issues are created by Nucleus through integrations with 3rd party Issue Trackers like Jira and ServiceNow. This object can be linked to finding instances. | /externalIssues/externalissues-finding-data-FULL/DELTA-TIMESTAMP |
About asset files
The /assets/asset-data-full-TIMESTAMP files contain data for all assets, including those without any findings associated with them. The /findings/assets-finding-data-FULL/DELTA-TIMESTAMP files contain data only for assets with findings associated with them.
These files are connected to each other via the following joining keys:
- Findings and Instances are connected via
finding_key. - Findings and Justifications are connected via
finding_number. - Instances and Justifications are connected via
instance_key. - Instances and External Issues are connected via
instance_key.
Schemas
Assets
| Primary Key | Type |
|---|---|
| host_id | INT |
| Foreign Keys | Type |
|---|---|
| ip_address | VARCHAR(64) |
| host_name | VARCHAR(2048 |
| host_type | VARCHAR(128) |
| host_criticality | VARCHAR(16) |
| operating_system_name | VARCHAR(256) |
Asset Groups
| Foreign Keys | Type |
|---|---|
| host_id | INT |
| asset_group_sources | VARCHAR(16) |
| asset_group_name | VARCHAR(256) |
Findings
| Foreign Keys | Type |
|---|---|
| finding_number | VARCHAR(251) |
| scan_type | VARCHAR(24) |
| finding_severity | VARCHAR(32) |
| finding_severity_adjusted | VARCHAR(32) NULL |
| finding_exploitable | TINYTIN(1) |
| finding_name | VARCHAR(128) |
| finding_cve | VARCHAR(8192) |
| finding_iava | VARCHAR(170) |
| finding_type | VARCHAR(32) |
| finding_description | TEXT |
| finding_recommendation | TEXT |
| finding_references | JSON |
| finding_key | CONCAT(scan_type, '-', finding_number) |
| epss_score | FLOAT NULL |
| cisa_vulnerability_name | VARCHAR |
| nucleus_threat_rating | VARCHAR |
| nucleus_exploited | ENUM('Yes', 'No', '') |
| nucleus_exploited_by_malware | ENUM('Yes', 'No', '') |
| nucleus_exploited_by_ransomware | ENUM('Yes', 'No', '') |
| nucleus_likely_to_be_exploited | ENUM('Yes', 'No', '') |
| nucleus_public_exploit_available | ENUM('Yes', 'No', '') |
| nucleus_private_exploit_available | ENUM('Yes', 'No', '') |
| nucleus_exploit_weaponized | ENUM('Yes', 'No', '') |
| nucleus_impacts_ot | ENUM('Yes', 'No', '') |
| nucleus_fix_available | ENUM('Yes', 'No', '') |
| nucleus_patch_available | ENUM('Yes', 'No', '') |
| nucleus_ease_of_exploitation | VARCHAR |
| nucleus_exploitation_consequence | VARCHAR |
| nucleus_remote_exploitation | ENUM('Yes', 'No', '') |
| nucleus_zero_day | ENUM('Yes', 'No', '') |
| nucleus_zero_day_previously | ENUM('Yes', 'No', '') |
| nucleus_mitigation_available | ENUM('Yes', 'No', '') |
| nucleus_widely_exploited | ENUM('Yes', 'No', '') |
| nucleus_media_mentions_alltime | INT NULL |
| nucleus_media_mentions_30day | INT NULL |
| nucleus_media_mentions_90day | INT NULL |
| nucleus_media_mentions_180day | INT NULL |
| threat_group_insights | JSON |
| malware_insights | JSON |
| mitigations_insights | JSON |
| patch_insights | JSON |
| exploits_insights | JSON |
| nucleus_actors | JSON |
| nucleus_malware | JSON |
Instances
| Foreign Keys | Type |
|---|---|
| project_id | INT |
| assessment_id | INT |
| finding_number | VARCHAR(251) |
| scan_type | VARCHAR(24) |
| finding_type | VARCHAR (24) |
| finding_justification_key | VARCHAR(256) |
| finding_discovered | DATETIME |
| finding_output | TEXT |
| finding_port | VARCHAR(12) |
| finding_path | VARCHAR(4096) |
| finding_result | VARCHAR(16) |
| host_id | INT |
| scan_date | DATETIME |
| instance_key | CONCAT(scan_type, '-', finding_number, '-', host_id, '-', finding_justification_key) |
| finding_key | CONCAT(scan_type, '-', finding_number) |
| justification_assignee | VARCHAR(512) |
| justification_text | TEXT |
| justification_status_name | VARCHAR(64) |
| justification_verification_date | DATETIME |
| justification_datetime | DATETIME |
| due_date | DATE |
| justification_status_mitigating | TINYINT(1) |
| finding_score | INT |
NOTE:
The finding_score field (Nucleus Risk Score) is only included in full exports, not delta exports. If a risk score cannot be calculated, the value will be 0.
Justification fields are not included in instances files by default. If you would like this option enabled for your organization, please contact Nucleus support.
Resolved Instances
| Foreign Keys | Type |
|---|---|
| project_id | INT |
| assessment_id | INT |
| finding_number | VARCHAR(251) |
| scan_type | VARCHAR(24) |
| finding_type | VARCHAR(24) |
| finding_justification_key | VARCHAR(256) |
| finding_discovered | DATETIME |
| finding_output | TEXT |
| finding_port | VARCHAR(12) |
| finding_path | VARCHAR(4096) |
| finding_result | VARCHAR(16) |
| host_id | INT |
| scan_date | DATETIME |
| mitigated_date | DATETIME |
| instance_key | CONCAT(scan_type, '-', finding_number, '-', host_id, '-', finding_justification_key) |
| finding_key | CONCAT(scan_type, '-', finding_number) |
| justification_assignee | VARCHAR(512) |
| justification_text | TEXT |
| justification_status_name | VARCHAR(64) |
| justification_verification_date | DATETIME |
| justification_datetime | DATETIME |
| due_date | DATE |
| justification_status_mitigating | TINYINT(1) |
NOTE: Justification fields are not included in instances files by default. If you would like this option enabled for your organization, please contact Nucleus support.
Justifications
| Foreign Keys | Type |
|---|---|
| host_id | INT |
| finding_number | VARCHAR(251) |
| scan_type | VARCHAR(24) |
| finding_justification_key | VARCHAR(256) |
| instance_key | CONCAT(scan_type, '-', finding_number, '-', host_id, '-', finding_justification_key) |
| justification_assignee | VARCHAR(512) |
| justification_text | TEXT |
| justification_status_id | INT |
| justification_verification_date | DATETIME |
| has_file | TINYINT(1) |
| security_comment_text | VARCHAR(8224) |
| justification_datetime | DATETIME |
| due_date | DATE |
Justification Status
| Foreign Keys | Type |
|---|---|
| justification_status_id | INT |
| justification_status_name | VARCHAR(64) |
| justification_status_mitigating | BOOL |
Teams
| Primary Key | Type |
|---|---|
| team_id | INT |
| Foreign Keys | Type |
|---|---|
| host_id | INT |
| finding_number | VARCHAR(251) |
| scan_type | VARCHAR(24) |
| finding_justification_key | VARCHAR(256) |
| instance_key | CONCAT(scan_type, '-', finding_number, '-', host_id, '-', finding_justification_key) |
External Issues
| Foreign Keys | Type |
|---|---|
| host_id | INT |
| finding_number | VARCHAR(251) |
| scan_type | VARCHAR(24) |
| finding_justification_key | VARCHAR(256) |
| instance_key | CONCAT(scan_type, '-', finding_number, '-', host_id, '-', finding_justification_key) |
Examples
Below are example files for each object type:
Assets
{
"page":1,
"pageTotal":1,
"assets":[
{
"asset_id":"1234567",
"asset_name":"abcdefg",
"asset_name_secondary":[
"ec2-233-252-0-211.us-east-2.compute.amazonaws.com",
"ip-198-51-100-120.us-east-2.compute.internal"
],
"ip_address":"233.252.0.211",
"ip_address_secondary":[
"198.51.100.120"
],
"asset_groups":[
"\/Hosts\/ec2",
"\/Hosts"
],
"asset_type":"Host",
"scan_date":"2022-06-14 21:32:59",
"asset_info":{
"aws.account-id":"112233445566",
"aws.region":"us-east-2",
"aws.service":"ec2",
"aws.partition":"aws",
"aws.arn":"arn:aws:ec2:us-east-2:111111111:instance\/i-111111111111",
"aws.ec2.instance-id":"i-111111111111",
"aws.ec2.image-id":"ami-123abc456def",
"aws.ec2.instance-state.code":"16",
"aws.ec2.instance-state.name":"running",
"aws.ec2.private-dns-name":"ip-198-51-100-120.us-east-2.compute.internal",
"aws.ec2.dns-name":"ec2-233-252-0-211.us-east-2.compute.amazonaws.com",
"aws.ec2.key-name":"key",
"aws.ec2.ami-launch-index":"0",
"aws.ec2.instance-type":"t2.micro",
"aws.ec2.launch-time":"2021-02-10T23:55:55.000Z",
"aws.ec2.placement.availability-zone":"us-east-2a",
"aws.ec2.placement.tenancy":"default",
"aws.ec2.monitoring.state":"disabled",
"aws.ec2.subnet-id":"subnet-123456abc",
"aws.ec2.vpc-id":"vpc-12345678",
"aws.ec2.private-ip-address":"198.51.100.120",
"aws.ec2.ip-address":"233.252.0.211",
"aws.ec2.source-dest-check":"true",
"aws.ec2.group-set-names":[
"launch-wizard-1"
],
"aws.ec2.group-set-ids":[
"sg-987654321"
],
"aws.ec2.architecture":"x86_64",
"aws.ec2.root-device-type":"ebs",
"aws.ec2.root-device-name":"name",
"aws.ec2.virtualization-type":"hvm",
"aws.tags.keyname":"value",
"aws.tags.owner":"Owner",
"aws.tags.name":"Name",
"aws.ec2.hypervisor":"xen",
"aws.ec2.iam-instance-profile.arn":"arn:aws:iam::123456789:instance-profile\/ec2role",
"aws.ec2.iam-instance-profile.id":"ABCDEFGHIJKLMNOP",
"aws.ec2.ebs-optimized":"false",
"aws.ec2.ena-support":"true"
},
"scan_date_timestamp":1655242379,
"operating_system_name":"",
"mac_address":"10:00:00:00:00:10",
"finding_count_critical":"23",
"finding_count_high":"321",
"finding_count_medium":"285",
"finding_count_low":"65",
"finding_count_informational":"4",
"finding_count_pass":"0",
"finding_count_fail":"0",
"finding_vulnerability_score":"900",
"asset_public":"1",
"asset_criticality":"Critical",
"asset_data_sensitivity_score":"10",
"asset_complianced_score":"10",
"asset_criticality_score":"10",
"asset_inactive_date":"",
"image_config":"",
"image_config_digest":"",
"image_manifest":"",
"image_manifest_digest":"",
"image_secondary_registries":[
],
"image_tags":[
],
"image_registry":"",
"image_repo":"",
"image_distro":"",
"operating_system_features":"",
"operating_system_version":"",
"image_platform_os":"",
"image_platform_os_features":[
],
"image_platform_os_version":"",
"image_platform_arch":"",
"image_platform_arch_features":[
],
"image_platform_arch_variant":"",
"active":true,
"business_owners"" [
"John Doe"
],
"owner_team": {
"team_id": "4000040",
"team_name": {
team_id": "400040",
"team_name": "Incident Response"
}
},
"support_team": {
"team_id": "4000032",
"team_name": {
team_id": "400032",
"team_name": "IT Support",
}
},
"branch":"",
"repo_url":""
},
...
]
}
Asset Findings
{
"page":1,
"pageTotal":1,
"assets":[
{
"host_id":"123456789",
"ip_address":"192.0.2.134",
"host_name":"192.0.2.134",
"host_type":"Host",
"host_criticality":"Critical",
"operating_system_name":""
},
...
]
}
Asset Groups
{
"page":1,
"pageTotal":1,
"assetGroups":[
{
"host_id":"123456789",
"asset_group_name":"Example Asset Group",
"asset_group_source":"manual"
},
...
]
}
Findings and Compliance Findings
{
"page":1,
"pageTotal":1,
"findings":[
{
"finding_number":"12345",
"scan_type":"QUALYS",
"finding_type":"Vuln",
"finding_severity":"Low",
"finding_severity_adjusted":null,
"finding_exploitable":"0",
"finding_name":"Web Server HTTP Trace\/Track Method Support Cross-Site Tracing Vulnerability",
"finding_cve":"CVE-2004-2320,CVE-2007-3008",
"finding_iava":"",
"finding_type":"Vuln",
"finding_description":"A Web server was detected that supports the HTTP TRACE method. A vulnerability related to this method was discovered. A malicious, active component in a Web page can send Trace requests to a Web server that supports this Trace method. Although unlikely and difficult to achieve, it's possible, in the presence of other browser vulnerabilities, for the active HTML content to make external requests to arbitrary Web servers beyond the hosting Web server.",
"finding_recommendation":"Solutions for some of the common Web servers are supplied below. For other Web servers, please check your vendor's documentation...",
"finding_references":{
"ID":"86473",
"CVE":"CVE-2004-2320,CVE-2007-3008",
"Category":"Web server",
"PCI Flagged":"1",
"Bugtraq IDs":"<a href=\"http:\/\/www.securityfocus.com\/bid\/24456\" target=\"_blank\">24456<\/a>,<a href=\"http:\/\/www.securityfocus.com\/bid\/9506\" target=\"_blank\">9506<\/a>"
},
"finding_key":"QUALYS-12345",
"finding_description_adjusted":null,
"finding_recommendation_adjusted":null,
"epss_score":null,
"cisa_vulnerability_name":"",
"nucleus_threat_rating":"",
"nucleus_exploited":"",
"nucleus_exploited_by_malware":"",
"nucleus_exploited_by_ransomware":"",
"nucleus_likely_to_be_exploited":"",
"nucleus_public_exploit_available":"",
"nucleus_private_exploit_available":"",
"nucleus_exploit_weaponized":"",
"nucleus_impacts_ot":"",
"nucleus_fix_available":"",
"nucleus_patch_available":"",
"nucleus_ease_of_exploitation":"",
"nucleus_exploitation_consequence":"",
"nucleus_remote_exploitation":"",
"nucleus_zero_day":"",
"nucleus_zero_day_previously":"",
"nucleus_mitigation_available":"",
"nucleus_widely_exploited":"",
"nucleus_media_mentions_alltime":null,
"nucleus_media_mentions_30day":null,
"nucleus_media_mentions_90day":null,
"nucleus_media_mentions_180day":null,
"threat_group_insights":[],
"malware_insights":[],
"mitigations_insights":[],
"patch_insights":[],
"exploits_insights":[],
"nucleus_actors":[],
"nucleus_malware":[]
},
...
]
}
Instances and Compliance Instances
{
"page":1,
"pageTotal":6,
"instances":[
{
"project_id":"123456789",
"assessment_id":"111111111",
"finding_number":"112345-abcdefg-67890",
"scan_type":"NESSUS",
"finding_type":"Vuln"
"finding_justification_key":"abcdefghijklmnopqrstuvwxyz",
"finding_discovered":"2022-05-31 21:00:39",
"finding_output":"The remote host (203.0.113.2) is considered as dead - not scanning\nThe remote host (203.0.113.2) did not respond to the following ping methods :\n- TCP ping\n- UDP ping\n- ICMP ping\n",
"finding_port":"0\/tcp",
"finding_path":"",
"finding_result":"FAILED",
"finding_severity": "Medium",
"finding_severity_adjusted": null,
"host_id":"222222222",
"scan_date":"2022-05-31 21:00:39",
"finding_score": 700,
"instance_key":"NESSUS-112345-abcdefg-67890-abcdefghijklmnopqrstuvwxyz",
"finding_key":"NESSUS-112345-abcdefg-67890"
},
...
]
}
** Resolved Instances**
{
"page":1,
"pageTotal":6,
"instances":[
{
"project_id":"123456789",
"assessment_id":"111111111",
"finding_number":"112345-abcdefg-67890",
"scan_type":"NESSUS",
"finding_type": "Vuln",
"finding_justification_key":"abcdefghijklmnopqrstuvwxyz",
"finding_discovered":"2022-05-31 21:00:39",
"finding_output":"The remote host (203.0.113.2) is considered as dead - not scanning\nThe remote host (203.0.113.2) did not respond to the following ping methods :\n- TCP ping\n- UDP ping\n- ICMP ping\n",
"finding_port":"0\/tcp",
"finding_path":"",
"finding_result":"FAILED",
"finding_severity": "Medium",
"finding_severity_adjusted": null,
"host_id":"222222222",
"scan_date":"2022-05-31 21:00:39",
"mitigated_date": "2024-12-16",
"instance_key":"NESSUS-112345-abcdefg-67890-abcdefghijklmnopqrstuvwxyz",
"finding_key":"NESSUS-112345-abcdefg-67890"
},
...
]
}
Justifications
{
"page":1,
"pageTotal":1,
"justifications":[
{
"host_id":"111111111",
"finding_number":"abcdefg",
"scan_type":"ACUNETIX",
"finding_justification_key":"abcdefg-static\/111213\/js\/-",
"instance_key":"ACUNETIX-abcdefg-111111111-abcdefg-static\/111213\/js\/-",
"justification_assignee":null,
"justification_text":"",
"justification_status_id":"10",
"justification_verification_date":null,
"has_file":"0",
"security_comment_text":"Not yet provided.",
"justification_datetime":"2022-06-09 20:29:04",
"due_date":null,
"justification_status_name":"Potential",
"justification_status_mitigating":0
},
...
]
}
Teams
{
"page":1,
"pageTotal":1,
"teams":[
{
"team_id":"1111111",
"host_id":"2222222",
"finding_number":"FINDING-5",
"scan_type":"SUPERGREP",
"finding_justification_key":"FINDING-5-SUPERGREP-1111111-path\/to\/vulnerable\/file5.txt-",
"instance_key":"SUPERGREP-FINDING-5-1111111-FINDING-5-SUPERGREP-1111111-path\/to\/vulnerable\/file5.txt-",
"team_name":"RemediationTeam"
},
...
]
}
External Issues
{
"page": 1,
"pageTotal": 1,
"externalIssues": [
{
"host_id": "31000018",
"finding_number": "FINDING-1",
"scan_type": "CUSTOMSCANNER",
"finding_justification_key": "NETWORK-1-CUSTOMSCANNER-31000018--0/tcp",
"instance_key": "CUSTOMSCANNER-NETWORK-1-31000018-NETWORK-1-CUSTOMSCANNER-31000018--0/tcp",
"issue_type": "SERVICENOWAPP",
"issue_key": "INC0047849",
"issue_title": "Nucleus - [High] - FINDING-1",
"issue_url": "https://exampleco.service-now.com/task.do?sys_id=32474ab787db4e1007a6a60d3fbb35a8",
"issue_parent": "INC0047844",
"issue_priority": "",
"issue_status": "New",
"issue_assignee": "",
"issue_created": "2024-07-16 19:20:03",
"issue_updated": "0000-00-00 00:00:00",
"issue_synced": "0000-00-00 00:00:00",
"issue_closed": "0",
"finding_mitigated": "0"
},
...
]
}