Overview
The Data Export feature provides the ability to download active asset, finding, and ticketing data in bulk from your Nucleus project using the Nucleus API. This provides the same export capabilities as S3 Data Export without having to configure your own S3 buckets. Data Export files are generated on a scheduled basis, and expire after 7 days. Note that larger files will automatically be broken up into 16Mb chunks. A detailed description of the format and contents of the files exported can be found here.
Deployment Restriction
At this time, this feature is only available for customers using Nucleus deployed on AWS. GCP, Azure, and Self-Hosted customers may not use this feature.
Data Export Setup
Warning
Enabling Data Export on a project will automatically remove the S3 Data Export feature of the Amazon Web Services connector. If you are currently using this, support will need to disable S3 Data Exports prior to enabling this feature.
Before downloading bulk data using the API, you must enable the type(s) of data you want available for download and schedule when and how often it should be refreshed.
Use the following steps to enable Data Export on your Nucleus project:
Click on the Data Export item under Reports in the left menu.
To enable downloading bulk asset data, check the Upload asset data (daily) option and select a Start time to process bulk asset data for download:
.png)
To enable downloading bulk finding and ticketing data, check the Upload finding data (weekly) option and select the Day and Start Time to process bulk finding data for download:
.png)
To also download resolved finding data, check the Upload resolved finding data (weekly).
Select the Upload finding deltas as new scans are ingested option to automatically upload new findings when new scans are ingested as delta files. This option will also upload any new custom findings created, or when a custom finding is edited (any changed parameter, either at the unique level or the justification level).
Note: Uploading deltas for external issues is not currently supported.
If you want all enabled data to begin processing immediately upon saving, check the option to Upload entire asset and finding data from Nucleus on save.
Click Save to save your changes.
Downloading Bulk Data via the API
After enabling Data Export and allowing time for processing and upload, the Nucleus API is used to download the generated bulk data files.
To get a list of available files for download, call the following API endpoint using your project's project_id:
Syntax:
GET /projects/{project_id}/dataexport
Example Response:
[
{
"file_id": "4024876",
"file_path": "/4000106/6530404d4bdd1/asset-data-full-2023-10-18T20-30-05-1.zip",
"data_type": "assets",
"mime_type": "application/zip",
"file_upload_date": "2023-10-18 20:30:05"
}
]
To initiate the download of a bulk export file, call the following API endpoint using your project's project_id and the file_id to download:
Syntax:
POST nucleus/api/projects/{project_id}/dataexport/{file_id}
If you’d like to cURL this, please replace the variables as needed:
curl -L -v -o report.zip -d "" "https://test.nucleussec.com/nucleus/api/projects/123123/dataexport/123123" -H "accept: application/zip" -H "x-apikey: xxxxxx"
If the call is successful, you will automatically be redirected and the file will begin to download.
Additional Resources
API Documentation - additional information about these endpoints, including a detailed list of response codes.
Data Export Formats - detailed descriptions and examples of file formats found in data export files.