LogoLogo
Go to Syntho.AI
English
English
  • Welcome to Syntho
  • Overview
    • Get started
      • Syntho bootcamp
        • 1. What is Syntho?
        • 2. Introduction data anonymization
        • 3. Connectors & workspace creation
        • 4. PII scan
        • 5. Generators
          • Mockers
          • Maskers
          • AI synthesize
          • Calculated columns
          • Free text de-identification
        • 6. Referential integrity & foreign keys
        • 7. Workspace synchronization & validation
        • 8. Workspace & user management
        • 9. Large workloads​
        • 10. Data pre-processing
        • 11. Continuous Success
      • Prerequisites
      • Sample datasets
      • Introduction to data generators
    • Frequently asked questions
  • Setup Workspaces
    • View workspaces
    • Create a workspace
      • Connect to a database
        • PostgreSQL
        • MySQL / MariaDB
        • Oracle
        • Microsoft SQL Server
        • DB2
        • Databricks
          • Importing Data into Databricks
        • Hive
        • SAP Sybase
        • Azure Data Lake Storage (ADLS)
        • Amazon Simple Storage Service (S3)
      • Workspace modes
    • Edit a workspace
    • Duplicate a workspace
    • Transfer workspace ownership
    • Share a workspace
    • Delete a workspace
    • Workspace default settings
  • Configure a Data Generation Job
    • Configure table settings
    • Configure column settings
      • AI synthesize
        • Sequence model
          • Prepare your sequence data
        • QA report
        • Additional privacy controls
        • Cross-table relationships limitations
      • Mockers
        • Text
          • Supported languages
        • Numeric (integer)
        • Numeric (decimal)
        • Datetime
        • Other
      • Mask
        • Text
        • Numeric (integer)
        • Numeric (decimal)
        • Datetime
        • UUID
      • Duplicate
      • Exclude
      • Consistent mapping
      • Calculated columns
      • Key generators
        • Differences between key generators
      • JSON de-identification
    • Manage personally identifiable information (PII)
      • Privacy dashboard
      • Discover and de-identify PII columns
        • Identify PII columns manually
        • Automatic PII discovery with PII scanner
      • Remove columns from PII list
      • Automatic PII discovery and de-identification in free text columns
      • Supported PII & PHI entities
    • Manage foreign keys
      • Foreign key inheritance
      • Add virtual foreign keys
        • Add virtual foreign keys
        • Use foreign key scanner
        • Import foreign keys via JSON
        • Export foreign keys via JSON
      • Delete foreign keys
    • Validate and Synchronize workspace
    • View and adjust generation settings
  • Deploy Syntho
    • Introduction
      • Syntho architecture
      • Requirements
        • Requirements for Docker deployments
        • Requirements for Kubernetes deployments
      • Access Docker images
        • Online
        • Offline
    • Deploy Syntho using Docker
      • Preparations
      • Deploy using Docker Compose
      • Run the application
      • Manually saving logs
      • Updating the application
    • Deploy Syntho using Kubernetes
      • Preparations
      • Deploy Ray using Helm
        • Upgrading Ray CRDs
        • Troubleshooting
      • Deploy Syntho using Helm
      • Validate the deployment
      • Troubleshooting
      • Saving logs
      • Upgrading the applications
    • Manage users and access
      • Single Sign-On (SSO) in Azure
      • Manage admin users
      • Manage non-admin users
    • Logs and monitoring
      • Does Syntho collect any data?
      • Temporary data storage by application
  • Syntho API
    • Syntho REST API
Powered by GitBook
On this page
  • Source schema validation
  • Source schema synchronization
  • Destination schema validation
  • Table issues:
  • Table warnings:
  • Column issues:
  • Column warnings:

Was this helpful?

  1. Configure a Data Generation Job

Validate and Synchronize workspace

PreviousDelete foreign keysNextView and adjust generation settings

Last updated 14 days ago

Was this helpful?

When you select Generate from the Job configuration panel, Syntho performs several validation steps against the schema of the source database and the destination database.

Source schema validation

The schema of your source database can change over time. For instance, a table might be added, a column removed, or a column's data type altered.

It's crucial that you keep track of these modifications and adjust your job settings accordingly. Failure to do so could result in the data generation process to fail. For instance, if a column's encoding type changes from categorical to discrete, and it was previously assigned a Custom String mocker, the Mocker will now be removed when it is not compatible with the new data type anymore.

Syntho is designed to track changes in the source database and inform you of any modifications to the data schema.

When selecting Generate, the Source Database Validation tab will alert you of any schema changes and give the option to Sync those changes with your workspace.

Source schema synchronization

Select the Sync button on the Job configuration panel to automatically synchronize any inconsistencies between your workspace and the source database schema.

If schema issues are left unresolved, they can result in the failure of the data generation process.

Syntho is equipped to automatically identify and synchronize the following types of schema changes:

Source table changes:

  1. The primary key of the table has changed.

  2. The primary key of the table has been removed.

  3. A primary key has been added to the table.

  4. A foreign of the table has been removed.

  5. A foreign key has been added to the table.

  6. The unique constraints for the table have been updated.

  7. The check constraints for the table have been updated.

  8. The table was removed.

Source column changes:

  1. The data type of the column has been updated.

  2. The column is now auto incrementing.

  3. The column is not auto incrementing anymore.

  4. The maximum value of a column has changed.

  5. The maximum length of a column has changed.

  6. The default of the column has been updated.

  7. The column is not nullable anymore.

  8. The column is now nullable.

  9. The column is not unique anymore.

  10. The column is now unique.

  11. The column is not part of the primary key anymore.

  12. The column is now part of the primary key.

  13. The column is not part of a foreign key anymore.

  14. The column is now part of a foreign key.

  15. The column is not part of a composite unique constraint anymore.

  16. The column is now part of a composite unique constraint.

  17. The generator of the column has been removed.

Destination schema validation

Syntho automatically validates the destination database's schema against your workspace for any included tables. This may result in several issues or warnings, which can be resolved by altering the destination database.

Table issues:

Table issues will cause your synthetic data job to fail. The following issues in your destination database may occur:

  • Table does not exist. To resolve this issue:

    1. Make sure that the schema names in the source and destination database match, and

    2. Add a table with the exact same name to your destination database.

  • Table has PK columns that are not in source.

    To resolve this issue:

    • Remove any primary key constraints in the destination schema that do not exist in the source schema. Ensure that primary keys align across both schemas to avoid conflicts.

  • Table has FK columns that are not in source.

    To resolve this issue:

    • Remove any foreign key constraints in the destination schema that do not exist in the source schema. Confirm that foreign key relationships are consistent with the source schema to maintain referential integrity.

Table warnings:

Table warnings do not automatically cause your synthetic data job to fail, but could still cause issues. The following warnings in your destination database may occur:

  • Table already contains data. To resolve this warning:

    • Truncate the table in the destination database.

  • Table is part of a circular reference. To resolve this warning:

    • Disable (or remove) any foreign key constraint(s) to break the circular reference.

Column issues:

Column issues will cause your synthetic data job to fail. The following issues in your destination database may occur:

  • Column does not exist. To resolve this issue:

    • Add a column with the exact same name in the table in your destination database.

  • Extra column is non-nullable and without a default value.

    To resolve this issue:

    • Either make the extra column nullable or set a default value in the destination schema. Alternatively, remove the column if it is unnecessary to ensure successful data generation.

  • Table has not-nullable columns that are not in source.

    To resolve this issue:

    • Remove or adjust any not-null constraints in the destination schema that do not exist in the source schema. Ensure that not-null constraints are aligned between the source and destination schemas to prevent conflicts.

Column warnings:

Column warnings do not automatically cause your synthetic data job to fail, but could still cause issues. The following warnings in your destination database may occur:

  • Column has mismatching data types. To resolve this warning:

    • Change the data type of the destination column to match the data type of the source column.

  • Column has a uniqueness constraint that is not in the source. To resolve this warning:

    • Remove the unique constraints from the destination.

  • Column has a check constraint that is not in the source. To resolve this warning:

    • Remove the check constraints from the destination.

  • Column has missing default value. To resolve this warning:

    • Ensure that the default value set on the column in the source database is also set in the destination database.

  • Destination has composite unique constraints that are not in source.

    To resolve this warning:

    • Remove any composite unique constraints in the destination schema that are not defined in the source schema. This will help prevent potential unique constraint violations.

  • Column maximum value is lower than source.

    To resolve this warning:

    • Increase the maximum value limit in the destination column to match or exceed the source column's maximum value. This will ensure that all data values fit within the destination schema without truncation.

  • Column maximum length is lower than source.

    To resolve this warning:

    • Increase the maximum length of the destination column to match or exceed the length in the source column. This will prevent potential data truncation and ensure consistency between source and destination schemas.

Validation warning
Successful validation
Start generating after a successful validation