LogoLogo
Go to Syntho.AI
English
English
  • Welcome to Syntho
  • Overview
    • Get started
      • Syntho bootcamp
        • 1. What is Syntho?
        • 2. Introduction data anonymization
        • 3. Connectors & workspace creation
        • 4. PII scan
        • 5. Generators
          • Mockers
          • Maskers
          • AI synthesize
          • Calculated columns
          • Free text de-identification
        • 6. Referential integrity & foreign keys
        • 7. Workspace synchronization & validation
        • 8. Workspace & user management
        • 9. Large workloads​
        • 10. Data pre-processing
        • 11. Continuous Success
      • Prerequisites
      • Sample datasets
      • Introduction to data generators
    • Frequently asked questions
  • Setup Workspaces
    • View workspaces
    • Create a workspace
      • Connect to a database
        • PostgreSQL
        • MySQL / MariaDB
        • Oracle
        • Microsoft SQL Server
        • DB2
        • Databricks
          • Importing Data into Databricks
        • Hive
        • SAP Sybase
        • Azure Data Lake Storage (ADLS)
        • Amazon Simple Storage Service (S3)
      • Workspace modes
    • Edit a workspace
    • Duplicate a workspace
    • Transfer workspace ownership
    • Share a workspace
    • Delete a workspace
    • Workspace default settings
  • Configure a Data Generation Job
    • Configure table settings
    • Configure column settings
      • AI synthesize
        • Sequence model
          • Prepare your sequence data
        • QA report
        • Additional privacy controls
        • Cross-table relationships limitations
      • Mockers
        • Text
          • Supported languages
        • Numeric (integer)
        • Numeric (decimal)
        • Datetime
        • Other
      • Mask
        • Text
        • Numeric (integer)
        • Numeric (decimal)
        • Datetime
        • UUID
      • Duplicate
      • Exclude
      • Consistent mapping
      • Calculated columns
      • Key generators
        • Differences between key generators
      • JSON de-identification
    • Manage personally identifiable information (PII)
      • Privacy dashboard
      • Discover and de-identify PII columns
        • Identify PII columns manually
        • Automatic PII discovery with PII scanner
      • Remove columns from PII list
      • Automatic PII discovery and de-identification in free text columns
      • Supported PII & PHI entities
    • Manage foreign keys
      • Foreign key inheritance
      • Add virtual foreign keys
        • Add virtual foreign keys
        • Use foreign key scanner
        • Import foreign keys via JSON
        • Export foreign keys via JSON
      • Delete foreign keys
    • Validate and Synchronize workspace
    • View and adjust generation settings
  • Deploy Syntho
    • Introduction
      • Syntho architecture
      • Requirements
        • Requirements for Docker deployments
        • Requirements for Kubernetes deployments
      • Access Docker images
        • Online
        • Offline
    • Deploy Syntho using Docker
      • Preparations
      • Deploy using Docker Compose
      • Run the application
      • Manually saving logs
      • Updating the application
    • Deploy Syntho using Kubernetes
      • Preparations
      • Deploy Ray using Helm
        • Upgrading Ray CRDs
        • Troubleshooting
      • Deploy Syntho using Helm
      • Validate the deployment
      • Troubleshooting
      • Saving logs
      • Upgrading the applications
    • Manage users and access
      • Single Sign-On (SSO) in Azure
      • Manage admin users
      • Manage non-admin users
    • Logs and monitoring
      • Does Syntho collect any data?
      • Temporary data storage by application
  • Syntho API
    • Syntho REST API
Powered by GitBook
On this page
  • Before you begin
  • Usage
  • Connect and Set Up the Workspace
  • File Formats
  • Limitations & Considerations

Was this helpful?

  1. Setup Workspaces
  2. Create a workspace
  3. Connect to a database

Local Filesystem

Last updated 6 months ago

Was this helpful?

Syntho beta feature

The Local Filesystem connector enables data writing to a specified path in the host's local filesystem. To utilize this connector, specific configuration steps are required to ensure proper functionality.

Destination Only

This connector can only be used as a destination for writing your generated data.

Supported File Types: Parquet and ORC

Before you begin

Environment Variable Setup:

Before running the Syntho application, declare the environment variable SYNTHO_LOCALFS. This variable should be set to the desired path in the host local filesystem where the data will be written. Example:

export SYNTHO_LOCALFS="/path/on/host"

Alternatively, you can declare the environment variable when running Docker:

SYNTHO_LOCALFS="/path/on/host" docker-compose up

Note: It is recommended to declare the environment variable using export.

Usage

  • Destination Only: The Local Filesystem connector can only be used as a target destination.

  • Parameter Requirement: The only required parameter is path. The value of this parameter will be concatenated with the value of SYNTHO_LOCALFS, and the resulting path will be where the data is written.

Example

If SYNTHO_LOCALFS is set to /mnt/data and the path parameter is output, the data will be written to /mnt/data/output.

This configuration allows the Syntho application to write data to the local filesystem, ensuring compatibility with various deployment environments without affecting other users who do not require this functionality.

Connect and Set Up the Workspace

Launch Syntho and select Connect to a database, or under Create workspace, select Local Filesystem. Then do the following:

  1. Enter the local target path where the data will be written.

  2. Choose a file format: Parquet or ORC.

  3. Select Create Workspace.

If Syntho can't make the connection, verify that the target path is correct. If you still can't connect, contact your database administrator.

File Formats

Supported file type formats include:

  • Parquet

  • ORC

Limitations & Considerations

  • For ORC files, columns full of None values which are of type Char, String, or Varchar will be written as "None" (i.e., a string value) to the destination database instead of None.

  • Local Filesystem connector is only supported for Docker Compose.

Contact your Syntho representative to discuss possible limitations regarding this connector.

Source and Destination Databases