LogoLogo
Go to Syntho.AI
English
English
  • Welcome to Syntho
  • Overview
    • Get started
      • Syntho bootcamp
        • 1. What is Syntho?
        • 2. Introduction data anonymization
        • 3. Connectors & workspace creation
        • 4. PII scan
        • 5. Generators
          • Mockers
          • Maskers
          • AI synthesize
          • Calculated columns
          • Free text de-identification
        • 6. Referential integrity & foreign keys
        • 7. Workspace synchronization & validation
        • 8. Workspace & user management
        • 9. Large workloads​
        • 10. Data pre-processing
        • 11. Continuous Success
      • Prerequisites
      • Sample datasets
      • Introduction to data generators
    • Frequently asked questions
  • Setup Workspaces
    • View workspaces
    • Create a workspace
      • Connect to a database
        • PostgreSQL
        • MySQL / MariaDB
        • Oracle
        • Microsoft SQL Server
        • DB2
        • Databricks
          • Importing Data into Databricks
        • Hive
        • SAP Sybase
        • Azure Data Lake Storage (ADLS)
        • Amazon Simple Storage Service (S3)
      • Workspace modes
    • Edit a workspace
    • Duplicate a workspace
    • Transfer workspace ownership
    • Share a workspace
    • Delete a workspace
    • Workspace default settings
  • Configure a Data Generation Job
    • Configure table settings
    • Configure column settings
      • AI synthesize
        • Sequence model
          • Prepare your sequence data
        • QA report
        • Additional privacy controls
        • Cross-table relationships limitations
      • Mockers
        • Text
          • Supported languages
        • Numeric (integer)
        • Numeric (decimal)
        • Datetime
        • Other
      • Mask
        • Text
        • Numeric (integer)
        • Numeric (decimal)
        • Datetime
        • UUID
      • Duplicate
      • Exclude
      • Consistent mapping
      • Calculated columns
      • Key generators
        • Differences between key generators
      • JSON de-identification
    • Manage personally identifiable information (PII)
      • Privacy dashboard
      • Discover and de-identify PII columns
        • Identify PII columns manually
        • Automatic PII discovery with PII scanner
      • Remove columns from PII list
      • Automatic PII discovery and de-identification in free text columns
      • Supported PII & PHI entities
    • Manage foreign keys
      • Foreign key inheritance
      • Add virtual foreign keys
        • Add virtual foreign keys
        • Use foreign key scanner
        • Import foreign keys via JSON
        • Export foreign keys via JSON
      • Delete foreign keys
    • Validate and Synchronize workspace
    • View and adjust generation settings
  • Deploy Syntho
    • Introduction
      • Syntho architecture
      • Requirements
        • Requirements for Docker deployments
        • Requirements for Kubernetes deployments
      • Access Docker images
        • Online
        • Offline
    • Deploy Syntho using Docker
      • Preparations
      • Deploy using Docker Compose
      • Run the application
      • Manually saving logs
      • Updating the application
    • Deploy Syntho using Kubernetes
      • Preparations
      • Deploy Ray using Helm
        • Upgrading Ray CRDs
        • Troubleshooting
      • Deploy Syntho using Helm
      • Validate the deployment
      • Troubleshooting
      • Saving logs
      • Upgrading the applications
    • Manage users and access
      • Single Sign-On (SSO) in Azure
      • Manage admin users
      • Manage non-admin users
    • Logs and monitoring
      • Does Syntho collect any data?
      • Temporary data storage by application
  • Syntho API
    • Syntho REST API
Powered by GitBook
On this page

Was this helpful?

  1. Deploy Syntho
  2. Logs and monitoring

Temporary data storage by application

The Syntho platform processes data securely, within the secure infrastructure of the customer. Below is an overview of how any temporary files are handled.

Temporary Files in Use

  1. Parquet Files:

    • Purpose: Used as staging data prior to writing the processed data to the destination.

    • Contents: Contains generated data during intermediate stages of processing.

    • Lifecycle:

      • These files are created temporarily and are designed to be deleted upon successful completion, cancelling, or failure of the processing job.

      • In case of an unexpected application failure, these files may remain in the internal storage until the application is restarted. After the restart they will be removed automatically.

    • Security Controls:

      • Access Controls: The internal storage location is secured with restricted access controls to protect the parquet files during their temporary existence.

      • Cleanup Mechanisms: Application-level watchdogs and cleanup routines exist to mitigate the risk of files persisting unnecessarily.

  2. Engine JSON Files:

    • Purpose: Generated to encapsulate job configuration metadata before submission to the Ray cluster for distributed processing.

    • Contents: Contains only non-sensitive metadata (e.g., configuration details and application runtime information).

    • Lifecycle:

      • These files are typically ephemeral and removed once the job is submitted.

      • If the application crashes at a specific processing point, the engine JSON file with public application data may persist.

    • Security Consideration:

      • As the file contains only public and non-sensitive data, it poses no security risk if retained within the secure infra of the customer.

PreviousDoes Syntho collect any data?NextSyntho REST API

Last updated 5 months ago

Was this helpful?