Archive Incremental Export

    Incremental export refers to the process of exporting only the data that has changed or been added since the last export, rather than exporting the entire dataset every time.


    In Archive, the incremental export job will run each day and export as much as possible, in a 23-hour period.
    The next day, a new incremental export job begins, starting after the last recorded exported record of the previous day. It can take some time for the incremental export to catch up on all the data in Archive, depending on how much data has accumulated before the incremental export was initiated.


    After it catches up, only newly archived data is exported - so there may be days when the incremental export job does not run.
     

    When you export to your S3 bucket or your Azure Blob Container, you can choose to select specific objects.

    NOTE: You can only export to storage that matches your account platform (i.e. S3 Bucket if running on AWS, and Azure Blob if running on Azure)

    We recommend Export all objects.
    Selecting Export only the selected objects means that once the incremental export has begun, if objects selected have been changed/added, the current incremental export stops, and a new incremental export starts from scratch.


    The new incremental export will review all records in the archive, exporting those that match the selection criteria. Again, this will be done in daily runs of up to 23 hours, until all the data in the archive is covered, and any new data will be exported as it is detected.
     

    For more information on how to, see here (S3 bucket) and here (Azure).

    « Previous ArticleNext Article »