--- title: "S3 Data Upload" slug: "s3-data-upload" updated: 2026-05-31T14:48:28Z published: 2026-05-31T14:48:28Z canonical: "help.nucleussec.com/s3-data-upload" --- > ## Documentation Index > Fetch the complete documentation index at: https://help.nucleussec.com/llms.txt > Use this file to discover all available pages before exploring further. # AWS S3 Bucket Data Export You can configure the AWS connector to upload all active asset and finding data from your Nucleus project to your own S3 buckets for your use outside of the Nucleus platform. Only active, manually mitigated, and custom findings are uploaded. Deployment Restriction At this time, this feature is only available for customers using Nucleus on AWS. GCP, Azure, and Self-Hosted customers may not use this feature. Before you get started, please review the documentations for [Creating a cross-account role and setting permissions](/v1/docs/aws-setting-permissions) and follow the steps to setup permissions for that role to allow S3 exports ## Setting up S3 data uploads 1. Log in to your Nucleus project. 2. From the navigation bar on the left, under **Project Administration**, select **Connector Setup**. 3. Select **Amazon Web Services**. ![aws-connector-icon.png](https://cdn.document360.io/3888970a-6501-459e-acc9-c47b71c6d64c/Images/Documentation/aws-connector-icon.png) 4. In the **General** tab, click the **green plus button** to add a new Role. Alternatively, you can bulk import credentials using a CSV file as detailed below. 5. In the **Label** field, enter a label for the AWS account. 6. In the **Role ARN** field, enter the Amazon Resource Name (ARN) of the role used for the connection. 7. Click **Verify Credentials**. If the credentials were entered correctly, a message confirming a successful connection will appear. 8. Select the **S3 Data** tab at the top. 9. Click the **green plus button** to add a new S3 bucket. Alternatively, you can bulk import credentials using a CSV file as detailed below. 10. In the **Label** field, enter a label for the AWS account. 11. In the **S3 Bucket** field, enter the address of the S3 bucket. 12. From the **Region** menu, select the region the S3 bucket is located in. 13. In the **Role ARN** field, enter the Amazon Resource Name (ARN) of the role used for the connection. 14. Click **Verify Credentials**. If the credentials were entered correctly, a message confirming a successful connection will appear. 15. Set the frequency to upload data. - To upload data on a set schedule: 1. Select the **Upload asset data (daily)** checkbox. 2. From the **Start time** menu, select the time asset data will upload every day. The time selected is in Coordinated Universal Time (UTC). 3. From the **Buckets** menu, select which buckets the asset data will upload to. 4. Select the **Upload finding data (weekly)** checkbox. 5. From the **Day** menu, select which day of the week finding data will upload. 6. From the **Start time** menu, select the time finding data will upload every day. The time selected is in Coordinated Universal Time (UTC). 7. From the **Buckets** menu, select which buckets the finding data will upload to. - To upload data in realtime: 1. Select the **Upload finding deltas as new scans are ingested** checkbox. When a new custom finding is created, or a custom finding is edited (any changed parameter, either at the unique level or the justification level), the finding will be uploaded to S3. 2. From the **Buckets** menu, select which buckets data will upload to. - To upload all asset and finding data immediately: 1. Select the **Upload entire asset and finding data from Nucleus to S3 on save** checkbox. 2. From the **Buckets** menu, select which buckets data will upload to. 16. Optionally select **Zip files prior to uploading them to S3** to compress the file size of the data uploaded to the S3 bucket. 17. Click **Save & Finish**. ### Bulk import credentials template If your organization has many S3 buckets, you can bulk import role ARNs by clicking **Bulk Import Buckets** and uploading a CSV structured in the following way: ``` label,s3bucket,region,crossaccountrole ``` ## Example template The following is an example CloudFormation template that can be deployed to AWS accounts to provide access to S3 buckets. Warning When using the template below, replace `ACCOUNT_ID_FROM_NUCLEUS`, `EXTERNAL_ID_FROM_NUCLEUS_PROJECT`, and `BUCKET_NAME_#` with the relevant **AWS Account**, **AWS External ID**, and bucket name values. ``` AWSTemplateFormatVersion: "2010-09-09" Resources: NucleusAWSIntegrationRole: Type: "AWS::IAM::Role" Properties: RoleName: "NucleusAWSS3BucketConnectorRole" Description: "Role used by a Nucleus Project AWS connector to upload asset and finding data to one or more S3 buckets." AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Principal: AWS: - "arn:aws:iam::ACCOUNT_ID_FROM_NUCLEUS:root" Action: - "sts:AssumeRole" Condition: StringEquals: "sts:ExternalId": "EXTERNAL_ID_FROM_NUCLEUS_PROJECT" Policies: - PolicyName: "NucleusAWSConnectorPolicy" PolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Action: "iam:ListAccountAliases" Resource: "*" - Effect: Allow Action: "s3:PutObject" Resource: - "arn:aws:s3:::BUCKET_NAME_1" - "arn:aws:s3:::BUCKET_NAME_1/*" - "arn:aws:s3:::BUCKET_NAME_2" - "arn:aws:s3:::BUCKET_NAME_2/*" ``` ## About uploaded data ### Format The data is uploaded in JSON format and separated into paginated files for each type of top-level object. These files are uploaded into three directories; `/assets`, `/findings` and `/externalIssues`. The `/assets` directory contains data about all assets, including those without any findings associated with them, and includes metadata regarding the assets themselves (e.g., host information, Nucleus statistics, etc). The `/findings` directory contains data for all findings, including associated assets, groups, instances, justifications, and teams. The `/externalIssues` directory contains data about issues created by Nucleus in 3rd party Issue Trackers like Jira and ServiceNow. Files with `full` in their name contain a complete upload of all data. Alternatively, files with `delta` in their name contain only data that has changed since the last scan (e.g., new findings, changed values). **Note:** Delta uploads for external issues is not currently supported. | Object | Description | File Name | | --- | --- | --- | | [Assets](/v1/docs/assets) | Anything that can have a vulnerability correlated with it. This object contains data for all assets. | `/assets/asset-data-FULL/DELTA-TIMESTAMP` | | [Asset Findings](/v1/docs/findings-and-their-properties) | Anything that can have a vulnerability correlated with it. This object only contains data for assets with findings associated with it. | `/findings/assets-finding-data-FULL/DELTA-TIMESTAMP` | | [Asset Groups](/v1/docs/asset-groups) | Flexible collections of assets that allow you to organize your assets across your entire organization. | `/findings/assetGroups-finding-data-FULL/DELTA-TIMESTAMP` | | [Findings](/v1/docs/findings-and-their-properties) | Unique (top-level) and instance-level vulnerabilities that exist on assets. When a finding is detected in an asset, that makes it an instance of that finding. If the same vulnerability is discovered in 5 assets, there will be 5 instances of that vulnerability. | 'Vuln' and 'Vuln-Compliance' type unique findings: `/findings/findings-finding-data-FULL/DELTA-TIMESTAMP` 'Compliance' and 'Vuln-Compliance' type unique findings: `/findings/compliance-findings-finding-data-FULL/DELTA-TIMESTAMP` 'Vuln' and 'Vuln-Compliance' type findings: `/instances/instances-finding-data-FULL/DELTA-TIMESTAMP` 'Compliance' and 'Vuln-Compliance' type findings: `/instances/compliance-instances-finding-data-FULL/DELTA-TIMESTAMP` | | Justifications | Justifications track the changes (e.g., a status change) to instances over time and subsequent scans. Any time there is a change to a finding (manually or via scan) there will be a justification record for that change. | `/findings/justifications-finding-data-FULL/DELTA-TIMESTAMP` | | [Teams](/v1/docs/teams) | Groups of users that collectively own assets and vulnerabilities in Nucleus. These Teams can then be linked to assets to define asset ownership, and then be further linked to vulnerabilities to define vulnerability ownership. | `/teams-finding-data-FULL/DELTA-TIMESTAMP` | | [External Issues](/v1/docs/external-tickets) | External issues are created by Nucleus through integrations with 3rd party Issue Trackers like Jira and ServiceNow. This object can be linked to finding instances. | `/externalIssues/externalissues-instances-finding-data-FULL/DELTA-TIMESTAMP` | About asset files The `/assets/asset-data-full-TIMESTAMP` files contain data for all assets, including those without any findings associated with them. The `/findings/assets-finding-data-FULL/DELTA-TIMESTAMP` files contain data only for assets with findings associated with them. Additionally, a "connectivity-verification" directory is created when when [first verifying the S3 bucket connection](/v1/docs/s3-data-upload#setting-up-s3-data-uploads). This directory contains empty, timestamped JSON files to confirm the AWS connector can successfully upload files. These files are connected to each other via the following joining keys: - **Findings** and **Instances** are connected via `finding_key`. - **Findings** and **Justifications** are connected via `finding_number`. - **Instances** and **Justifications** are connected via `instance_key`. - **Instances** and **External Issues** are connected via`instance_key`. - **Assets** and **Instances** are connected via `host_id` (where `assets.asset_id` = `instances.host_id`). ### Schemas **Assets** | Primary Key | Type | | --- | --- | | host_id | INT | | Foreign Keys | Type | | --- | --- | | ip_address | VARCHAR(64) | | host_name | VARCHAR(2048 | | host_type | VARCHAR(128) | | host_criticality | VARCHAR(16) | | operating_system_name | VARCHAR(256) | **Asset Groups** | Foreign Keys | Type | | --- | --- | | host_id | INT | | asset_group_sources | VARCHAR(16) | | asset_group_name | VARCHAR(256) | **Findings** | Foreign Keys | Type | | --- | --- | | finding_number | VARCHAR(251) | | scan_type | VARCHAR(24) | | finding_type | VARCHAR(24) | | finding_severity | VARCHAR(32) | | finding_severity_adjusted | VARCHAR(32) NULL | | finding_exploitable | TINYTIN(1) | | finding_name | VARCHAR(128) | | finding_cve | VARCHAR(8192) | | finding_iava | VARCHAR(170) | | finding_type | VARCHAR(32) | | finding_description | TEXT | | finding_recommendation | TEXT | | finding_references | JSON | | finding_key | CONCAT(scan_type, '-', finding_number) | | epss_score | FLOAT NULL | | cisa_vulnerability_name | VARCHAR | **Instances** | Foreign Keys | Type | | --- | --- | | project_id | INT | | assessment_id | INT | | finding_number | VARCHAR(251) | | scan_type | VARCHAR(24) | | finding_type | VARCHAR(24) | | finding_justification_key | VARCHAR(256) | | finding_discovered | DATETIME | | finding_output | TEXT | | finding_port | VARCHAR(12) | | finding_path | VARCHAR(4096) | | finding_result | VARCHAR(16) | | host_id | INT | | scan_date | DATETIME | | instance_key | CONCAT(scan_type, '-', finding_number, '-', host_id, '-', finding_justification_key) | | finding_key | CONCAT(scan_type, '-', finding_number) | | *justification_assignee* | *VARCHAR(512)* | | *justification_text* | *TEXT* | | *justification_status_name* | *VARCHAR(64)* | | *justification_verification_date* | *DATETIME* | | *justification_datetime* | *DATETIME* | | *due_date* | *DATE* | | *justification_status_mitigating* | *TINYINT(1)* | | *finding_score* | INT | | nucleus_threat_rating | VARCHAR | | nucleus_exploited | ENUM('Yes', 'No', '') | | nucleus_exploited_by_malware | ENUM('Yes', 'No', '') | | nucleus_exploited_by_ransomware | ENUM('Yes', 'No', '') | | nucleus_likely_to_be_exploited | ENUM('Yes', 'No', '') | | nucleus_public_exploit_available | ENUM('Yes', 'No', '') | | nucleus_private_exploit_available | ENUM('Yes', 'No', '') | | nucleus_exploit_weaponized | ENUM('Yes', 'No', '') | | nucleus_impacts_ot | ENUM('Yes', 'No', '') | | nucleus_fix_available | ENUM('Yes', 'No', '') | | nucleus_patch_available | ENUM('Yes', 'No', '') | | nucleus_ease_of_exploitation | VARCHAR | | nucleus_exploitation_consequence | VARCHAR | | nucleus_remote_exploitation | ENUM('Yes', 'No', '') | | nucleus_zero_day | ENUM('Yes', 'No', '') | | nucleus_zero_day_previously | ENUM('Yes', 'No', '') | | nucleus_mitigation_available | ENUM('Yes', 'No', '') | | nucleus_widely_exploited | ENUM('Yes', 'No', '') | | nucleus_media_mentions_alltime | INT NULL | | nucleus_media_mentions_30day | INT NULL | | nucleus_media_mentions_90day | INT NULL | | nucleus_media_mentions_180day | INT NULL | | threat_group_insights | JSON | | malware_insights | JSON | | mitigations_insights | JSON | | patch_insights | JSON | | exploits_insights | JSON | | nucleus_actors | JSON | | nucleus_malware | JSON | NOTE:
The finding_score field (Nucleus Risk Score) is only included in full exports, not delta exports. If a risk score cannot be calculated, the value will be 0 *Justification fields are not included in instances files by default. If you would like this option enabled for your organization, please contact Nucleus support. **Justifications** | Foreign Keys | Type | | --- | --- | | host_id | INT | | finding_number | VARCHAR(251) | | scan_type | VARCHAR(24) | | finding_justification_key | VARCHAR(256) | | instance_key | CONCAT(scan_type, '-', finding_number, '-', host_id, '-', finding_justification_key) | | justification_assignee | VARCHAR(512) | | justification_text | TEXT | | justification_status_id | INT | | justification_verification_date | DATETIME | | has_file | TINYINT(1) | | security_comment_text | VARCHAR(8224) | | justification_datetime | DATETIME | | due_date | DATE | **Justification Status** | Foreign Keys | Type | | --- | --- | | justification_status_id | INT | | justification_status_name | VARCHAR(64) | | justification_status_mitigating | BOOL | **Teams** | Primary Key | Type | | --- | --- | | team_id | INT | | Foreign Keys | Type | | --- | --- | | host_id | INT | | finding_number | VARCHAR(251) | | scan_type | VARCHAR(24) | | finding_justification_key | VARCHAR(256) | | instance_key | CONCAT(scan_type, '-', finding_number, '-', host_id, '-', finding_justification_key) | **External Issues** | Foreign Keys | Type | | --- | --- | | host_id | INT | | finding_number | VARCHAR(251) | | scan_type | VARCHAR(24) | | finding_justification_key | VARCHAR(256) | | instance_key | CONCAT(scan_type, '-', finding_number, '-', host_id, '-', finding_justification_key) | ### Examples Below are example files for each object type: **Assets** ``` { "page":1, "pageTotal":1, "assets":[ { "asset_id":"1234567", "asset_name":"abcdefg", "asset_name_secondary":[ "ec2-233-252-0-211.us-east-2.compute.amazonaws.com", "ip-198-51-100-120.us-east-2.compute.internal" ], "ip_address":"233.252.0.211", "ip_address_secondary":[ "198.51.100.120" ], "asset_groups":[ "\/Hosts\/ec2", "\/Hosts" ], "asset_type":"Host", "scan_date":"2022-06-14 21:32:59", "asset_info":{ "aws.account-id":"112233445566", "aws.region":"us-east-2", "aws.service":"ec2", "aws.partition":"aws", "aws.arn":"arn:aws:ec2:us-east-2:111111111:instance\/i-111111111111", "aws.ec2.instance-id":"i-111111111111", "aws.ec2.image-id":"ami-123abc456def", "aws.ec2.instance-state.code":"16", "aws.ec2.instance-state.name":"running", "aws.ec2.private-dns-name":"ip-198-51-100-120.us-east-2.compute.internal", "aws.ec2.dns-name":"ec2-233-252-0-211.us-east-2.compute.amazonaws.com", "aws.ec2.key-name":"key", "aws.ec2.ami-launch-index":"0", "aws.ec2.instance-type":"t2.micro", "aws.ec2.launch-time":"2021-02-10T23:55:55.000Z", "aws.ec2.placement.availability-zone":"us-east-2a", "aws.ec2.placement.tenancy":"default", "aws.ec2.monitoring.state":"disabled", "aws.ec2.subnet-id":"subnet-123456abc", "aws.ec2.vpc-id":"vpc-12345678", "aws.ec2.private-ip-address":"198.51.100.120", "aws.ec2.ip-address":"233.252.0.211", "aws.ec2.source-dest-check":"true", "aws.ec2.group-set-names":[ "launch-wizard-1" ], "aws.ec2.group-set-ids":[ "sg-987654321" ], "aws.ec2.architecture":"x86_64", "aws.ec2.root-device-type":"ebs", "aws.ec2.root-device-name":"name", "aws.ec2.virtualization-type":"hvm", "aws.tags.keyname":"value", "aws.tags.owner":"Owner", "aws.tags.name":"Name", "aws.ec2.hypervisor":"xen", "aws.ec2.iam-instance-profile.arn":"arn:aws:iam::123456789:instance-profile\/ec2role", "aws.ec2.iam-instance-profile.id":"ABCDEFGHIJKLMNOP", "aws.ec2.ebs-optimized":"false", "aws.ec2.ena-support":"true" }, "scan_date_timestamp":1655242379, "operating_system_name":"", "mac_address":"10:00:00:00:00:10", "finding_count_critical":"23", "finding_count_high":"321", "finding_count_medium":"285", "finding_count_low":"65", "finding_count_informational":"4", "finding_count_pass":"0", "finding_count_fail":"0", "finding_vulnerability_score":"900", "asset_public":"1", "asset_criticality":"Critical", "asset_data_sensitivity_score":"10", "asset_complianced_score":"10", "asset_criticality_score":"10", "asset_inactive_date":"", "image_config":"", "image_config_digest":"", "image_manifest":"", "image_manifest_digest":"", "image_secondary_registries":[ ], "image_tags":[ ], "image_registry":"", "image_repo":"", "image_distro":"", "operating_system_features":"", "operating_system_version":"", "image_platform_os":"", "image_platform_os_features":[ ], "image_platform_os_version":"", "image_platform_arch":"", "image_platform_arch_features":[ ], "image_platform_arch_variant":"", "active":true, "business_owners"" [ "John Doe" ], "owner_team": { "team_id": "4000040", "team_name": { team_id": "400040", "team_name": "Incident Response" } }, "support_team": { "team_id": "4000032", "team_name": { team_id": "400032", "team_name": "IT Support", } }, "branch":"", "repo_url":"" }, ... ] } ``` **Asset Findings** ``` { "page":1, "pageTotal":1, "assets":[ { "host_id":"123456789", "ip_address":"192.0.2.134", "host_name":"192.0.2.134", "host_type":"Host", "host_criticality":"Critical", "operating_system_name":"" }, ... ] } ``` **Asset Groups** ``` { "page":1, "pageTotal":1, "assetGroups":[ { "host_id":"123456789", "asset_group_name":"Example Asset Group", "asset_group_source":"manual" }, ... ] } ``` **Findings and Compliance Findings** ``` { "page":1, "pageTotal":1, "findings":[ { "finding_number":"12345", "scan_type":"QUALYS", "finding_type":"Vuln", "finding_severity":"Low", "finding_severity_adjusted":null, "finding_exploitable":"0", "finding_name":"Web Server HTTP Trace\/Track Method Support Cross-Site Tracing Vulnerability", "finding_cve":"CVE-2004-2320,CVE-2007-3008", "finding_iava":"", "finding_type":"Vuln", "finding_description":"A Web server was detected that supports the HTTP TRACE method. A vulnerability related to this method was discovered. A malicious, active component in a Web page can send Trace requests to a Web server that supports this Trace method. Although unlikely and difficult to achieve, it's possible, in the presence of other browser vulnerabilities, for the active HTML content to make external requests to arbitrary Web servers beyond the hosting Web server.", "finding_recommendation":"Solutions for some of the common Web servers are supplied below. For other Web servers, please check your vendor's documentation...", "finding_references":{ "ID":"86473", "CVE":"CVE-2004-2320,CVE-2007-3008", "Category":"Web server", "PCI Flagged":"1", "Bugtraq IDs":"24456<\/a>,9506<\/a>" }, "finding_key":"QUALYS-12345", "finding_description_adjusted":null, "finding_recommendation_adjusted":null, "epss_score":null, "cisa_vulnerability_name":"", "nucleus_threat_rating":"", "nucleus_exploited":"", "nucleus_exploited_by_malware":"", "nucleus_exploited_by_ransomware":"", "nucleus_likely_to_be_exploited":"", "nucleus_public_exploit_available":"", "nucleus_private_exploit_available":"", "nucleus_exploit_weaponized":"", "nucleus_impacts_ot":"", "nucleus_fix_available":"", "nucleus_patch_available":"", "nucleus_ease_of_exploitation":"", "nucleus_exploitation_consequence":"", "nucleus_remote_exploitation":"", "nucleus_zero_day":"", "nucleus_zero_day_previously":"", "nucleus_mitigation_available":"", "nucleus_widely_exploited":"", "nucleus_media_mentions_alltime":null, "nucleus_media_mentions_30day":null, "nucleus_media_mentions_90day":null, "nucleus_media_mentions_180day":null, "threat_group_insights":[], "malware_insights":[], "mitigations_insights":[], "patch_insights":[], "exploits_insights":[], "nucleus_actors":[], "nucleus_malware":[] }, ... ] } ``` **Instances and Compliance Instances** ``` { "page":1, "pageTotal":6, "instances":[ { "project_id":"123456789", "assessment_id":"111111111", "finding_number":"112345-abcdefg-67890", "scan_type":"NESSUS", "finding_type":"Vuln" "finding_justification_key":"abcdefghijklmnopqrstuvwxyz", "finding_discovered":"2022-05-31 21:00:39", "finding_output":"The remote host (203.0.113.2) is considered as dead - not scanning\nThe remote host (203.0.113.2) did not respond to the following ping methods :\n- TCP ping\n- UDP ping\n- ICMP ping\n", "finding_port":"0\/tcp", "finding_path":"", "finding_result":"FAILED", "finding_severity": "Medium", "finding_severity_adjusted": null, "host_id":"222222222", "scan_date":"2022-05-31 21:00:39", "finding_score": 700, "instance_key":"NESSUS-112345-abcdefg-67890-abcdefghijklmnopqrstuvwxyz", "finding_key":"NESSUS-112345-abcdefg-67890" }, ... ] } ``` **Justifications** ``` { "page":1, "pageTotal":1, "justifications":[ { "host_id":"111111111", "finding_number":"abcdefg", "scan_type":"ACUNETIX", "finding_justification_key":"abcdefg-static\/111213\/js\/-", "instance_key":"ACUNETIX-abcdefg-111111111-abcdefg-static\/111213\/js\/-", "justification_assignee":null, "justification_text":"", "justification_status_id":"10", "justification_verification_date":null, "has_file":"0", "security_comment_text":"Not yet provided.", "justification_datetime":"2022-06-09 20:29:04", "due_date":null, "justification_status_name":"Potential", "justification_status_mitigating":0 }, ... ] } ``` **Teams** ``` { "page":1, "pageTotal":1, "teams":[ { "team_id":"1111111", "host_id":"2222222", "finding_number":"FINDING-5", "scan_type":"SUPERGREP", "finding_justification_key":"FINDING-5-SUPERGREP-1111111-path\/to\/vulnerable\/file5.txt-", "instance_key":"SUPERGREP-FINDING-5-1111111-FINDING-5-SUPERGREP-1111111-path\/to\/vulnerable\/file5.txt-", "team_name":"RemediationTeam" }, ... ] } ``` **External Issues** ``` { "page": 1, "pageTotal": 1, "externalIssues": [ { "host_id": "31000018", "finding_number": "FINDING-1", "scan_type": "CUSTOMSCANNER", "finding_justification_key": "NETWORK-1-CUSTOMSCANNER-31000018--0/tcp", "instance_key": "CUSTOMSCANNER-NETWORK-1-31000018-NETWORK-1-CUSTOMSCANNER-31000018--0/tcp", "issue_type": "SERVICENOWAPP", "issue_key": "INC0047849", "issue_title": "Nucleus - [High] - FINDING-1", "issue_url": "https://exampleco.service-now.com/task.do?sys_id=32474ab787db4e1007a6a60d3fbb35a8", "issue_parent": "INC0047844", "issue_priority": "", "issue_status": "New", "issue_assignee": "", "issue_created": "2024-07-16 19:20:03", "issue_updated": "0000-00-00 00:00:00", "issue_synced": "0000-00-00 00:00:00", "issue_closed": "0", "finding_mitigated": "0" }, ... ] } ```