# Welcome to Syntho

Welcome to the Syntho product documentation! You can find more information about Syntho's synthetic data generation platform here. Syntho is an all-in-one platform for generating privacy-safe synthetic data. It enables organizations to create high-quality, statistically accurate datasets that mirror the original data, without compromising sensitive information.

## Video overview

Got a couple of minutes? Check out a demo video of our product:

{% embed url="<https://www.youtube.com/watch?v=N5cb_SFqOSU>" %}


# Release notes

Use this page to track changes per Syntho release. New releases are listed first.

{% hint style="info" %}
Upgrading a self-hosted deployment? Use the matching guide:

* [Upgrade (Docker Compose)](/deploy-syntho/deploy-syntho-using-docker/upgrade)
* [Upgrade (Kubernetes / Helm)](/deploy-syntho/deploy-syntho-using-kubernetes/upgrade)
  {% endhint %}

### What we include

* **Added**: new features.
* **Improved**: UX, performance, and maintainability changes.
* **Fixed**: bug fixes.
* **Security**: security fixes, CVEs, and dependency upgrades.
* **Breaking changes**: anything that requires action after upgrading.

### Releases

### 2026.1.15

**Release date: 6 June 2026**

####

<details>

<summary>Changes</summary>

#### Improved

* Updated Random Character Swap 2.0 to be case-insensitive w/ consistent mapping ON.
* Improved validation for generator schema parameters.
* Improved table validation by skipping unnecessary `n_rows` checks.
* Improved Parquet output handling for `INT64` precision.
* Renamed Redis-related artifacts to queue for clearer terminology.

#### Fixed

* Fixed date formulas in Computed mockers on `DATE` columns.
* Fixed handling of implied unique generators.
* Databricks: Updated token handling.

</details>

### 2026.1.14

**Release date: 4 June 2026**

<details>

<summary>Changes</summary>

#### Added

* Added support for generating rows based on the source row count.

#### Improved

* Improved the job summary to make generation checks clearer.
* Improved column syncing performance by removing an unnecessary generator preview step.
* Simplified generation settings by removing the read random subset option.

</details>

### 2026.1.13

**Release date: 29 May, 2026**

<details>

<summary>Changes</summary>

#### Improved

* Improved migration `bf5a008a9e78` to only consider `CURRENT` databases and safely remove stale rows.

#### Fixed

* Fixed an issue where the build artifact step did not use the specified version.

</details>

### 2026.1.12

**Release date: 22 May, 2026**

<details>

<summary>Changes</summary>

#### Improved

* Avoided loading the full workspace when possible to improve the speed of workspace operations.

</details>

### 2026.1.11

**Release date: 15 May, 2026**

<details>

<summary>Changes</summary>

#### Improved

* Enabled identity writing by default.
* Combined table warnings into a single warning tooltip.
* Improved application messaging for clearer and more consistent communication.
* Improved alignment of permission-limited action buttons across the UI.
* Improved tooltip visibility for excluded tables.
* Improved engine import performance by adding an index to the column configuration table.
* Read and write batch sizes are now unified into one parameter.

#### Fixed

* Fixed various small UI issues.
* Fixed duplicate owner handling.
* Fixed issues in custom string generation.
* Fixed job timestamp serialization in the jobs endpoint.

#### Security

* Fixed CVE-2026-39892.
* Fixed CVE-2026-40192.
* Fixed CVE-2026-3902.
* Fixed CVE-2026-28684.
* Fixed CVE-2026-33672.
* Fixed CVE-2025-69277.
* Fixed AIKIDO-2026-10385.
* Fixed AIKIDO-2026-10369.

</details>

### 2026.1.10

**Release date: 13 April, 2026**

<details>

<summary>Changes</summary>

#### Added

* Datasource caching to improve performance and reduce repeated data fetching.

</details>

### 2026.1.9

**Release date: 13 April, 2026**

<details>

<summary>Changes</summary>

#### Added

* Workspace export and import, making it easier to move workspace configurations between environments.
* Support for generating empty tables when needed.

#### Improved

* Main hub and Hash key generator are no longer marked as beta.
* Calculated columns behavior for `DATEDIFF` and `DATEADD`, including better handling of nullable columns, RAND-based intervals, pasted expressions, and int-like values generated by mockers.
* Generator selection in the table view, including better ordering and hiding generator types that are not compatible with the column data type.
* Workspace synchronization flow, including a reset of the active workspace after sync actions and fresher table data after updates.
* UI refinements across the platform, including improved hover tooltips, bulk include/exclude toggle alignment, table scrolling in sync mode, and icon sizing in job confirmation summaries.
* API notebook improvements.

#### Fixed

* An issue that could prevent a generator from being serialized.
* Datetime-like data clamping behavior.
* Decimal clamping validation.
* Computed formulas on nullable columns.
* Hardcoded PII modification for key columns.

#### Security

* Implemented CSP and Permissions-Policy headers.
* Fixed CVE-2026-27459.
* Fixed CVE-2026-30922.
* Fixed CVE-2026-4800.
* Fixed CVE-2026-32597.
* Fixed CVE-2026-22815.

</details>

### 2026.1.8

**Release date: 27 March, 2026**

<details>

<summary>Changes</summary>

#### Added

* Workspace export and import functionality to move configurations between environments.

#### Improved

* Decimal clamping validation to ensure more accurate handling of numeric limits.

</details>

### 2026.1.7

**Release date: 24 March, 2026**

<details>

<summary>Changes</summary>

#### Added

* Column Privacy Ranking in the privacy audit flow, to help prioritize columns based on privacy status.
* A warning for Databricks cases where a case statement result is truncated.
* Support for configuring the maximum parallel jobs through an environment variable.

#### Improved

* Target form validation in the generation flow.
* Capabilities handling for better overall performance.

#### Fixed

* Fixed update engine conditions for update engine jobs.

#### Security

* None.

</details>

### 2026.1.6

Release date: **19 March 2026**

<details>

<summary><strong>Changes</strong></summary>

**Added**

* Support for configuring core application settings through environment variables.
* Initial supported variables: `DATABRICKS_READ_STMT_MAX_ATTEMPTS`, `DATABRICKS_READ_STMT_INITIAL_WAIT`, and `LOG_LEVEL`.

**Security**

* Fixed security issues.
* Addressed dependency advisories for: `CVE-2025-69223`, `CVE-2026-21441`, `CVE-2026-21226`, `CVE-2026-23490`, `CVE-2025-69277`, `CVE-2025-66034`, `CVE-2026-0994`, `CVE-2025-14550`, `GHSA-h25m-26qc-wcjf`, `CVE-2026-22036`, `CVE-2025-13465`, and `CVE-2026-24486`.

</details>

### 2026.1.5

Release date: **19 March 2026**

<details>

<summary>Changes</summary>

**Added**

* **Databricks**: Added a retry mechanism with exponential backoff for warehouse reads to handle the 1-hour limit on available data chunks.

</details>

### 2026.1.4

Release date: **16 March 2026**

<details>

<summary>Changes</summary>

**Added**

* Added support for the Generators domain model.
* Added a new sync condition to trigger source sync after target sync.

**Improved**

* Databricks: Added a timeout when retrieving statement results to improve connector reliability.
* Improved handling of target schema updates when connection details change.

**Fixed**

* Fixed an issue where the Noise generator could fail on decimal values.

</details>

### 2026.1.3

Release date: **9 March 2026**

<details>

<summary>Changes</summary>

**Added**

* Added include and exclude states to table ordering for easier table selection management. This aligns with Syntho’s include/exclude table workflow in Table view.

**Improved**

* Removed subsetting-related code from Backend and CoreAPI to simplify internal maintenance. Syntho documents subsetting as a dedicated workflow, so this change appears to streamline separation of concerns.
* Databricks: Updated Warehouse ID handling. Syntho supports an optional Warehouse ID for Databricks workspaces, where it is recommended for faster and more stable querying.
* Added the missing `light_version` parameter on the engine repository call.
* Introduced the [Privacy audit report](/configure-a-data-generation-job/privacy-audit-report).

**Fixed**

* Fixed an issue affecting custom string and Custom sampler behavior. Custom sampler is a documented generator in Syntho.
* Oracle: Removed an issue related to `DROP INDEX` compilation.
* Fixed a regression introduced in an earlier release.
* Fixed internal connector/backend behavior related to custom string and sampling configuration.

**Security**

* No CVE IDs were listed for this release.

</details>

### 2026.1.2

Release date: **24 February 2026**

<details>

<summary>Changes</summary>

**Added**

* Added a substring method to version and build information to truncate long strings.

**Improved**

* Refactored API endpoints to use services instead of repositories, improving maintainability and separation of concerns.
* Databricks: Made the Warehouse ID parameter optional when configuring the connection.

**Fixed**

* MySQL: Fixed the query used to check `DROP INDEX` operations.

**Security**

* No CVE IDs were listed for this release.

</details>

### 2026.1.1

Release date: **17 February 2026**

<details>

<summary>Changes</summary>

**Improved**

Implementation of data reading by utilizing Databricks API for better memory performance and faster data read.

</details>

### 2026.1.0

Release date: **16 February 2026**

<details>

<summary>Changes</summary>

**Added**

* Job attachment support.
* Source sync when editing a job.
* Bulk selection toggle for all subcategories.
* GitBook AI assistant integration.
* Feature functionality management enabled by default.

**Improved**

* Faster engine settings update.
* Refactored engine and connection forms.
* Long table names now display with a middle ellipsis.
* Optimized CI/CD pipeline.
* Upgraded core Ray base image to `2.53.0`.
* Updated `oracledb` to the latest version.
* Removed unused packages in the core module.

**Fixed**

* Schema issues in status updates.
* Unstable schema views rendering.
* Workspace creation status updates.
* Capability selection reset after validation.
* Protected/Unprotected Key PII columns classification.
* Workspace sync issues when using a filesystem-based target connector.
* Default settings update.
* Job status issues.
* Inheritance issue in CoreAPI services.
* Generate page layout issue.

**Security**

* Improved security by addressing a template injection vulnerability in GitHub Workflows Action.

**Dependency advisories fixed**

* CVE-2025-69223
* CVE-2026-21441
* CVE-2026-21226
* CVE-2026-23490
* CVE-2025-69277
* CVE-2025-66034
* CVE-2026-0994
* CVE-2025-14550
* GHSA-h25m-26qc-wcjf
* CVE-2026-22036
* CVE-2025-13465
* CVE-2026-24486

**Breaking changes**

None.

</details>


# Get started


# Syntho bootcamp

Welcome to the Syntho bootcamp! You can find more information about Syntho's bootcamp here.

The Syntho Bootcamp is an interactive learning track designed to help you quickly become confident in using the Syntho platform. Whether you're a data engineer, analyst, or privacy specialist, this bootcamp provides hands-on, structured guidance to help you master privacy-preserving data generation with Syntho.

## What you'll learn

***

This bootcamp is organized into clear, focused sections that cover the full data generation workflow, from connecting to your data sources to generating synthetic data and managing privacy compliance.

By the end, you’ll understand:

* How Syntho anonymizes and synthesizes data using generators
* How to configure workspaces and connectors
* How to manage PII scanning, masking, and advanced data generation strategies
* How to maintain referential integrity and validate generated datasets

## Who it's for

***

* New users looking for a guided onboarding experience
* Technical teams setting up test data pipelines
* Privacy teams exploring anonymization techniques
* Product teams testing applications without using production data

## How to use this bootcamp

***

* Each module is self-contained and can be completed independently.
* Follow along using your own data or test environments.
* Use the embedded demos to apply what you've learned.
* Explore linked documentation when you want to go deeper.

If you're ready to get started, proceed to the next section: **What is Syntho?** Alternatively, check the contents below and jump to the section that you wish to explore.

## Contents

***

1. [What is Syntho?](/overview/get-started/syntho-bootcamp/1.-what-is-syntho)
2. [Introduction data anonymization](/overview/get-started/syntho-bootcamp/2.-introduction-to-data-generators)
3. [Connectors & workspace creation](/overview/get-started/syntho-bootcamp/3.-connectors-and-workspace-creation)
4. [PII scan](/overview/get-started/syntho-bootcamp/4.-pii-scan)
5. [Generators](https://github.com/syntho-ai/syntho/blob/main/docs/public/overview/get-started/syntho-bootcamp/5.-generators)
6. [Referential integrity and foreign keys](/overview/get-started/syntho-bootcamp/6.-referential-integrity-and-foreign-keys)
7. [Workspace synchronization and validation](/overview/get-started/syntho-bootcamp/7.-workspace-synchronization-and-validation)
8. [Workspace & user management](/overview/get-started/syntho-bootcamp/8.-workspace-and-user-management)
9. [Large workloads](/overview/get-started/syntho-bootcamp/9.-large-workloads)
10. [Date pre-processing](/overview/get-started/syntho-bootcamp/10.-ai-synthesis-data-pre-processing-when-using)
11. [Continuous success](/overview/get-started/syntho-bootcamp/11.-continuous-success)


# 1. What is Syntho?

Syntho is an all-in-one platform for generating privacy-safe synthetic data. It enables organizations to create high-quality datasets that mirror the original data, without compromising sensitive information.

Check out a brief walkthrough of Syntho and some of the functionalities we will demonstrate:

{% embed url="<https://www.guidejar.com/guides/e25ac7e6-e635-4259-87d5-96d4a1d76e5e>" %}

***

### One platform, multiple data generation approaches

Syntho unifies all synthetic data generation approaches into a single solution. It supports:

* **Data masking**: directly modify sensitive fields to anonymize them while preserving data formats.
* **Rule-based synthetic data:** define custom rules and logic to generate data from scratch or simulate scenarios specific to your organization.
* **AI-generated synthetic data**: uses advanced generative models to create entirely new, statistically representative data that has no direct link to original records.

Whether you need to develop software, test integrations, or analyze data, Syntho allows you to generate synthetic data tailored for development, testing, or acceptance environments, commonly referred to as a synthetic data twin of your real environment.

### Why choose Syntho?

Syntho is designed to be:

#### **Most versatile**

Use and combine all available data generation methods in one place—AI synthesis, masking, mock generators, calculated columns, and more.

#### **Fastest**

Accelerate your workflows with minimal manual configurations, optimized batch processing, and automation for test data creation.

#### **Easy to use**

Designed for both technical and non-technical users, Syntho offers a collaborative interface with features like role-based access, connector templates, and a privacy dashboard to support every stage of the data lifecycle.

In summary, Syntho empowers teams to safely and quickly generate data that protects privacy while preserving utility, enabling faster innovation and broader data access without the risks.


# 2. Introduction to data generators

Syntho offers a flexible set of data generators that help anonymize sensitive data based on the nature of the dataset, privacy requirements, and use case. Below is a summary of the main generator types and when to use each.

### [**AI-generated synthetic data**](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation)

Trains a generative model to create synthetic rows that mimic the original dataset, without any one-to-one relation.\
\&#xNAN;*Use when:* you need statistical fidelity and privacy, e.g. for machine learning or testing large datasets.\
\&#xNAN;*Avoid when:* you need to preserve correlations and data consistency across related tables.

### [**Mockers**](/configure-a-data-generation-job/configure-column-settings/mockers)

Generate fully random, user-defined values.\
\&#xNAN;*Use when:* format matters, but relationship to original values is not important.\
\&#xNAN;*Avoid when:* consistency or referential integrity is needed.

### [**Mockers with consistent mapping**](/configure-a-data-generation-job/configure-column-settings/consistent-mapping)

Maps original values to consistent mock values.\
\&#xNAN;*Use when:* consistent replacement of values is needed across datasets or environments.\
\&#xNAN;*Avoid when:* randomness is more important than consistency.

### [**Mask**](/configure-a-data-generation-job/configure-column-settings/mask)

Directly modifies original values while preserving format.\
\&#xNAN;*Use when:* the output must remain in a recognizable or valid format.\
\&#xNAN;*Avoid when:* preserving exact values or reversibility is required.

### [**Calculated columns**](/configure-a-data-generation-job/configure-column-settings/calculated-columns)

Uses business logic to generate values.\
\&#xNAN;*Use when:* you need calculated outputs based on specific conditions.\
\&#xNAN;*Avoid when:* data generation is simple and preserving logic is not required.

### [**Key generators**](/configure-a-data-generation-job/configure-column-settings/key-generators)

Create or transform keys while maintaining or removing relational links.\
\&#xNAN;*Use when:* managing primary and foreign keys across tables.\
\&#xNAN;*Avoid when:* relationships are not needed.


# 3. Connectors & workspace creation

To start working in Syntho, the first step is to connect your data sources and initialize your workspace.

***

### Supported databases

<table data-view="cards"><thead><tr><th></th><th align="center"></th><th></th><th data-hidden data-card-target data-type="content-ref"></th><th data-hidden data-card-cover data-type="files"></th></tr></thead><tbody><tr><td></td><td align="center"><img src="/files/CNuH9ZhC4cXEE7O4SGZI" alt="" data-size="original"></td><td></td><td><a href="/pages/1AgoQHNXkEOFZHDZYoAM">/pages/1AgoQHNXkEOFZHDZYoAM</a></td><td></td></tr><tr><td></td><td align="center"><img src="/files/9InHAycy9uKNoCJmQUqt" alt="" data-size="original"></td><td></td><td><a href="/pages/pznUwXwGX0laz6Z8Vm28">/pages/pznUwXwGX0laz6Z8Vm28</a></td><td></td></tr><tr><td></td><td align="center"><img src="/files/X4UT7sCbuhaR9DWAn5Ow" alt=""></td><td></td><td><a href="https://github.com/shahin-hu/Translations/blob/main/English/setup-workspaces/create-a-workspace/connect-to-a-database/broken-reference/README.md">https://github.com/shahin-hu/Translations/blob/main/English/setup-workspaces/create-a-workspace/connect-to-a-database/broken-reference/README.md</a></td><td></td></tr><tr><td></td><td align="center"><img src="/files/dgFl0Em2sgyAYdTNHE6B" alt=""></td><td></td><td><a href="/pages/RynPf8Bl1w8VZOUWc3La">/pages/RynPf8Bl1w8VZOUWc3La</a></td><td></td></tr><tr><td></td><td align="center"><img src="/files/zHAzcsKmA8Lc8N9PV4Vt" alt=""></td><td></td><td><a href="/pages/xkIy1PQOBX7okiF6Qit0">/pages/xkIy1PQOBX7okiF6Qit0</a></td><td></td></tr><tr><td></td><td align="center"><img src="/files/6cP5UOewt66pJ90SddtY" alt=""></td><td></td><td><a href="/pages/Vxaa98oEtqQ7bbb2F8Ut">/pages/Vxaa98oEtqQ7bbb2F8Ut</a></td><td></td></tr><tr><td></td><td align="center"><img src="/files/ZAoG2FClzZ54bTViSWcz" alt=""></td><td></td><td><a href="/pages/tVqZES0tnUCTNmri20wx">/pages/tVqZES0tnUCTNmri20wx</a></td><td></td></tr></tbody></table>

### Supported filesystems (destination only)

<table data-view="cards"><thead><tr><th align="center"></th><th align="center"></th><th align="center"></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td align="center"></td><td align="center"><img src="/files/DjcjcG0N9R423RsCYsuz" alt=""></td><td align="center"></td><td><a href="/pages/tdYBoBDngVGXePMYIAu0">/pages/tdYBoBDngVGXePMYIAu0</a></td></tr><tr><td align="center"></td><td align="center"><img src="/files/Tgmql4IRatHvmMZOhtPs" alt="" data-size="original"></td><td align="center"></td><td><a href="/pages/ke8Vqm2KENKJdJ6HvE97">/pages/ke8Vqm2KENKJdJ6HvE97</a></td></tr></tbody></table>

{% hint style="info" %}
Missing a database connector? Let us know. Custom connectors can be built on request.
{% endhint %}

### Initialization modes overview

| Mode             | When to use                                                                                                                                                                 | What happens                                                                                                     |
| ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| Synthesize all   | For one or more two related tables. To preserve maximum statistical validity with high privacy.                                                                             | PII columns receive mock generator suggestions, and non-PII columns use AI synthesis.                            |
| Mock all         | Theoretically the most secure, since all data is generated independently of the original data. This is also the best choice if there is no or limited input data available. | All PII and non-PII columns get one or more suggested mock generators.                                           |
| Mock or mask all | Like “Mock all”, except masking generators may be applied in certain scenarios to generate more realistic data.                                                             | All PII and non-PII columns get one or more suggested mock and/or mask generators.                               |
| De-identify      | When only replacement of PII is required, while non-PII columns should remain in-tact, making it suitable for internal test cases.                                          | Suggested generators are applied only to PII columns. Non-PII columns do not receive any generator suggestions.. |
| From scratch     | For custom configurations or when only a few tables need to be configured.                                                                                                  | All tables start excluded by default, and the application doesn't suggest generators.                            |

### Interactive guide: How to create a workspace

Use the demo below to walk through the process of creating a connector and setting up a workspace.

{% embed url="<https://www.guidejar.com/guides/4a0b2d1e-b093-4910-b550-81c863921bba>" %}


# 4. PII scan

Syntho provides a built-in PII scanner to help you identify columns that may contain personally identifiable information (PII) in your datasets. This step is essential to ensure data privacy compliance and to prevent accidental exposure of sensitive fields.

You can run a [shallow scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner#shallow-scan-uses-metadata) (faster, based on metadata) or a [deep scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner#deep-scan-uses-metadata--data) (more accurate, inspects data contents using NLP).

{% hint style="warning" %}
**Important:** PII Scanner results should always be reviewed manually. Some columns may be falsely flagged or missed.
{% endhint %}

***

### Interactive guide: How to start the PII scanner

Follow the interactive guide below to run your first PII scan

{% embed url="<https://www.guidejar.com/guides/14115eb8-62b8-41a1-b382-43a0bd19bd05>" %}

***

### Scan modes explained

| Scan Type                                                                                                                              | Description                                                             | Speed  | Accuracy                                   |
| -------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | ------ | ------------------------------------------ |
| [Shallow Scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner#shallow-scan-uses-metadata) | Uses column names and regex rules to infer PII                          | Fast   | Medium                                     |
| [Deep Scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner#deep-scan-uses-metadata--data) | Analyzes actual data content using NLP models (for string/text columns) | Slower | Higher (but possibly more false positives) |

#### What to watch for

* Red PII column headers: These columns are flagged but not yet handled (e.g., no [Mocker](/configure-a-data-generation-job/configure-column-settings/mockers) or [Mask](/configure-a-data-generation-job/configure-column-settings/mask)).
* Exclamation mark (!) next to table name: Indicates at-risk PII columns are still in [Duplicate](/configure-a-data-generation-job/configure-column-settings/duplicate) mode.

#### To resolve

* Apply a [Mocker](/configure-a-data-generation-job/configure-column-settings/mockers)
* Or [exclude](/configure-a-data-generation-job/configure-column-settings/exclude) the column from data generation


# 5. Generators

Syntho provides a variety of generators to support different data anonymization and test data generation needs. These generators allow you to define how each column in your dataset is handled, depending on privacy requirements and data utility goals.

This section introduces the main generator types and shows how to apply and configure them in your workspace. Check the contents below and jump to the section that you wish to explore.

## Contents

***

1. [Mockers](/overview/get-started/syntho-bootcamp/5.-generators/mock)
2. [Maskers](/overview/get-started/syntho-bootcamp/5.-generators/mask)
3. [AI synthesize](/overview/get-started/syntho-bootcamp/5.-generators/ai-synthesize)
4. [Calculated Columns](/overview/get-started/syntho-bootcamp/5.-generators/calculated-columns)
5. [Free text PII obfuscation](/overview/get-started/syntho-bootcamp/5.-generators/free-text-de-identification)


# Mock

[Mock generators](/configure-a-data-generation-job/configure-column-settings/mockers) are used to generate entirely new, random values for a column. They are ideal when privacy is the top priority, and there’s no need to maintain a link to the original data values.

#### When to use

* To fill columns that contain directly identifiable information (PII)
* To populate empty columns
* When format matters, but data relationships do not

#### When not to use

* When relationships with original data must be preserved
* For key columns (e.g., primary or foreign keys)
* When data correlations or dependencies matter
* When maintaining statistical properties is important

#### Available mock generators

Mock are grouped by data type:

* [Text](https://app.gitbook.com/s/SUjgQeXGZtfM3xg5Rj73/configure-a-data-generation-job/configure-column-settings/mockers/text)
* Numeric ([integer ](https://app.gitbook.com/s/SUjgQeXGZtfM3xg5Rj73/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer)and [decimal](https://app.gitbook.com/s/SUjgQeXGZtfM3xg5Rj73/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal))
* [Datetime](https://app.gitbook.com/s/SUjgQeXGZtfM3xg5Rj73/configure-a-data-generation-job/configure-column-settings/mockers/datetime)
* [Other](https://app.gitbook.com/s/SUjgQeXGZtfM3xg5Rj73/configure-a-data-generation-job/configure-column-settings/mockers/other)

#### Interactive guide: How to apply a mock generator

Follow the interactive guide below to apply a mock generator.

{% embed url="<https://www.guidejar.com/guides/c726b4cc-0e10-4c4d-a6e4-3ffcb3a70560>" %}

* **Constraints on Key Columns:** Mock generators cannot be applied to primary key or foreign key columns.
* **Column-by-Column Operation:** Mock generatorsfunction on individual columns. At this point, they can't be used to preserve logical relationships across multiple columns.
* **Automatically Cutoff Values:** The Syntho platform automatically cuts off generated mock text values based on the data type's supported length. For example, a mock generator applied on an NVARCHAR(5) column, will cutoff all values beyond the first 5 characters of the text.
* **Automatically Clip Values:** The Syntho platform automatically clips numerical values that exceed the maximum or minimum size, to the largest or smallest value supported by the data type, respectively.
* **No Link with Original Records:** Mock generators do not link back to the original data records, enhancing privacy but potentially reducing the usefulness of the data. If you want to retain the link with the original values, you can enable the [**Consistent Mapping**](#mocker-parameters) feature.
* **Database Type Compatibility:** The return type of a mock generator indicates its compatibility with specific database data types. For example, a mock generator with a text return type is compatible with database types like **(N)VARCHAR** or **TEXT**, but not with a database type **NUMERIC (INTEGER)**. It is important to this into account when applying mock generators on your columns to prevent your data generation job to fail.

Being aware of these limitations and considerations will help you effectively use mock generators while understanding their constraints.


# Mask

[Mask generators](/configure-a-data-generation-job/configure-column-settings/mask) directly modify existing values to anonymize them, while preserving the original format. This makes masked data more suitable for scenarios where format realism is required.

#### When to use

* To anonymize both directly and indirectly identifiable information
* When data must remain valid in format (e.g. date, email, UUID)
* For PII in non-production or staging environments

#### When not to use

* When exact values or relationships must be preserved
* When synthetic data is required instead of transformed real data
* In production environments where full de-identification is needed

#### Available mask generators

Mask generators are categorized by data type:

* [Text](/configure-a-data-generation-job/configure-column-settings/mask/text)
* Numeric ([integer](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) and [decimal](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal))
* [Datetime](/configure-a-data-generation-job/configure-column-settings/mask/datetime)
* [UUID](/configure-a-data-generation-job/configure-column-settings/mask/uuid)

#### Interactive guide: How to apply a mask generator

Follow the interactive guide below to apply a [mask generator](/configure-a-data-generation-job/configure-column-settings/mask).

{% embed url="<https://www.guidejar.com/guides/ea6bafd8-da84-4bdb-83cf-23a123da9a86>" %}


# AI synthesize

[AI synthesize](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation) allows you to synthesize realistic data using machine learning models trained on your original dataset. This method maintains statistical fidelity while ensuring privacy and unlinkability to the source records.

#### When to use

* To create synthetic datasets for machine learning or analytics
* When high statistical accuracy and maximum privacy are required
* To expand datasets while preserving original distributions

#### When not to use

* When working with multiple related tables
* When data consistency across systems is required
* When you need to be able to revert to original records
* If entirely new, unseen text values must be generated
* If the data needs to follow specific rules with 100% certainty​

The Syntho platform supports a wide variety of data types. Under the hood, Syntho uses an encoding scheme where each data type is mapped to one of the following encoding types.

| Data type                                                                                                   | Description                                  |
| ----------------------------------------------------------------------------------------------------------- | -------------------------------------------- |
| [Discrete](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)       | Numerical counts (e.g. number of visits)     |
| [Continuous](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)   | Continuous values (e.g. weight, temperature) |
| [Categorical](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | Predefined values (e.g. blood type, country) |
| [Datetime](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)       | Timestamps and dates (e.g. created at)       |

#### Interactive guide: How to apply AI synthesize

Follow the interactive guide below to apply AI synthesize.

{% embed url="<https://www.guidejar.com/guides/4ebdd966-24fe-4c67-b1a4-2ec1690d8c41>" %}

#### [Rare category protection](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#rare-category-protection)

To protect privacy, Syntho can automatically replace infrequent values in categorical columns:

* Threshold: minimum frequency before a value is considered rare (default = 10)
* Replacement: value used to replace rare categories (default = `*`)

#### [Advanced settings](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#advanced-settings)

[Generator-level](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#advanced-generator-settings)

* Max rows used for training: limit data for faster performance
* Take random sample: randomly sample rows for training

[Column-level](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#advanced-column-settings)

* Clipping thresholds: restrict extreme values in numeric/date columns
* Locale: set language model context for text/PII


# Calculated columns

[Calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns) allow you to define custom formulas to generate or transform data using logical, mathematical, statistical, or text-based operations. These formulas are applied after other generators like mockers or AI synthesis, ensuring that all base data is available before the calculation is performed.

#### When to use

* To clean or reformat data (e.g. trimming, date formatting)
* To apply conditional logic (e.g. gender-based name generation)
* To compute derived values from other columns (e.g. sales + tax)
* To categorize or flag data based on specific criteria

#### When not to use

* For straightforward mock data generation without dependencies
* When no conditional logic is needed
* If the column is already correctly populated or handled by simpler generators

***

#### Interactive guide: How to apply a calculated column formula

Follow the interactive guide below to apply a calculated column formula.

{% embed url="<https://www.guidejar.com/guides/f304989a-0496-4a02-8659-2762e0a157f8>" %}

#### [Syntax and structure](/configure-a-data-generation-job/configure-column-settings/calculated-columns#syntax)

Calculated columns follow a structured expression syntax combining functions, column references, constants, and mockers.

#### Example formulas

**1) `ROUND` — round a number to&#x20;*****N*****&#x20;decimals**

```dax
ROUND(
  [AMOUNT],
  2
)
```

* `number`: the value to round. Example: `[AMOUNT]`.
* `count`: decimal places. Example: `2` rounds to cents.

**2) `IF` — conditional logic**

```dax
IF(
  [COUNTRY] = "US",
  "domestic",
  "international"
)
```

* `test`: a condition that returns `TRUE` or `FALSE`. Example: `[COUNTRY] = "US"`.
* `then value`: returned when `test` is `TRUE`. Example: `"domestic"`.
* `otherwisevalue`: returned when `test` is `FALSE`. Example: `"international"`.

**3) `DATEADD` — shift a date by a fixed interval**

```dax
DATEADD(
  [ORDER_DATE],
  14,
  "day"
)
```

* `dates`: a date or datetime column. Example: `[ORDER_DATE]`.
* `number_of_intervals`: how many units to move. Example: `14`.
* `interval`: `"year"`, `"quarter"`, `"month"`, or `"day"`.

**4) Invoice total (combined) — arithmetic + null handling + rounding**

```dax
ROUND(
  ([AMOUNT] * (1 + IFNULL([TAX_RATE], 0))) - IFNULL([DISCOUNT], 0),
  2
)
```

Uses `IFNULL(value, replacement_value)` to guard against missing inputs.

**5) Normalized email (combined) — text functions + mocker**

```dax
SUBSTITUTE(
  CONCATENATE(
    LOWER(TRIM([FIRSTNAME])),
    ".",
    LOWER(TRIM([LASTNAME])),
    "@",
    MOCK_FREE_EMAIL_DOMAIN
  ),
  " ",
  ""
)
```

Builds an email-like string, normalizes casing, and removes spaces.

***

#### Key syntax rules

* **Column reference:** `[ColumnName]` for same-table columns
* **Functions:** `IF()`, `AND()`, `DATE()`, etc.
* **Mockers:** Use `MOCK_FIRST_NAME`, or `MOCK_CONSISTENT_FIRST_NAME` for consistent mapping
* **Operators:** `+`, `-`, `*`, `/`, `=`, `<>`, `<`, `>`
* **Constants:** Use strings `"text"`, numbers `100`, or dates `DATE(2020, 12, 31)`
* **Avoid:** Column names starting with `_`, which will cause formula errors

***

#### Using mockers in formulas

To insert mock data dynamically within formulas:

* Type `MOCK_` and choose from the autosuggest list
* Use names like `MOCK_FIRST_NAME`, `MOCK_COMPANY_EMAIL`
* For consistent mapping, use `MOCK_CONSISTENT_FIRST_NAME`, etc.

***

| Generator          | Supported data types                    |
| ------------------ | --------------------------------------- |
| Calculated Columns | Categorical, Discrete, Continuous, Date |

***

Calculated columns give you full control over how synthetic values are created or transformed. They’re ideal for applying custom logic while retaining flexibility in the data generation process.


# Free text PII obfuscation

[Free text PII obfuscation](/configure-a-data-generation-job/configure-column-settings/duplicate/automatic-pii-discovery-and-de-identification-in-free-text-columns) enables Syntho to automatically detect and anonymize personally identifiable information (PII) hidden in unstructured text columns. This is particularly useful for columns containing names, notes, comments, or descriptions that may include sensitive information.

{% hint style="warning" %}
Caution: Using this feature significantly increases processing time. Consider limiting the number of input rows or enabling GPU acceleration.
{% endhint %}

#### When to use

* To identify and anonymize PII within text fields like "notes", "comments", or "descriptions"
* When working with unstructured data that may contain embedded identifiers
* To prepare free-text data for AI-powered generation or duplication without privacy risk

#### When not to use

* When the text only contains a single identifiable value (e.g., just a name or number)
* When the text exceeds 1,000 characters or contains complex, domain-specific language
* When performance and speed are critical and anonymization of text is not essential

***

## Apply free text PII obfuscation

Apply free text PII obfuscation with these steps:

1. Select **Mask** as the generator.
2. Select **Free text PII obfuscation** as the mask generator.
3. Select the correct **Locale**.
4. Optionally, enable **Replace PII with mock data**.

When you apply the PII text scanner to specific columns, Syntho automatically scans for PII elements in those columns. Identified PII elements can then be replaced with mock data. Syntho employs a variety of algorithms and methods to improve the scanning process.

Here's an overview of the steps taken in the detection process, in chronological order:

1. **Regex**: for pattern recognition.
2. **Named Entity Recognition** **(NER)**: to recognize natural language PII entities.
3. **Checksums**: to validate detected patterns.
4. **Context words**: to increase detection certainty.
5. **Label**: to label detected PII entity with a descriptor of the entity.
6. **(Optional) Obfuscate**: to replace detected PII descriptors with mock data.

Under **Encoding type > Locale**, you can define the locale used by the text processing models for text columns containing PII.

Syntho supports detection and de-identification of PII fields for the languages **English** and **Dutch** in columns containing free text data.

Syntho allows adding **NLP (natural language processing)** models with limited support for different languages (see next section).

* **PII Detection and Confidence Score:** The PII text scanner may identify multiple potential **Personally Identifiable Information (PII)** entities within a text column. When this occurs, the entity with the highest confidence score is presented to the user. However, it's important to understand that a high confidence score doesn't guarantee accuracy. This could result in mislabeling the type of PII detected.
* **Internet Requirement for Non-Default NLP Models:** If you opt to use specialized **Natural Language Processing (NLP)** models to accommodate different languages or regions, an active internet connection is necessary to download these models.
* **Detection Methods:** The scanner employs a multi-method approach for PII detection, including the use of **Regex** patterns, **Named Entity Recognition (NER)** models, checksum validation, and examination of context words. Note that the effectiveness of the NER models can vary in different context it's being used. For instance, a NER model trained on Wikipedia text may not perform well when applied to medical data.

***

Free text PII obfuscation helps keep unstructured data privacy-safe. Use it for comments, descriptions, or any text column that may contain sensitive terms.


# 6. Referential integrity & foreign keys

Referential integrity ensures consistency between related tables in a relational database. In Syntho, preserving referential integrity is essential when generating test data or anonymizing production data across linked tables.

Foreign keys are used to link two tables together—for example, a patient table (with primary keys) and a medication table (with foreign keys referencing the patient IDs).

***

## What is referential integrity?

Referential integrity ensures that relationships between tables remain valid. For example, if `Patient ID 3456` exists in the `Patients` table, any reference to this patient in the `Medications` table must point to that exact ID.

In test or synthetic environments, maintaining referential integrity ensures:

* Consistency across linked datasets
* Valid references between primary and foreign keys
* Reliability of test results, especially in integration testing and staging environments

<figure><img src="/files/KJlKVcIhNcZCeDvviocd" alt=""><figcaption></figcaption></figure>

Syntho supports three types of key generators to handle referential integrity:

| Method    | Description                                                                                                                                                               | When to use                                                                                                                                                                                    | When not to use                                                                                                                                                                                                                                                       |
| --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Duplicate | Copies the original key values exactly as they appear in the source data, preserving both the correlations and referential integrity between primary and foreign keys.    | When it’s essential to maintain the original key values and relationships, particularly in de-identification scenarios where the data structure must be preserved without generating new keys. | Upsampling is not supported when using Duplicate, as the original keys are simply copied, not expanded. Additionally, it is not recommended when the keys are sensitive and need to be protected, as this method retains the original key values without obfuscation. |
| Generate  | Creates new, synthetic key values that do not correspond to the original keys. It preserves only the referential integrity, but not the correlations between key columns. | Use Generate for upsampling or creating synthetic datasets where there is no need to maintain relationships with the original data. It can also be used when creating data from scratch.       | The Generate function creates new keys independently of the original key order, which disrupts correlations. As a result, it is unsuitable for scenarios where maintaining the correlations and order is essential.                                                   |
| Hash      | Converts original key values into hashed representations. Both correlations between tables and relational integrity are maintained.                                       | Use Hash when you need to obscure the original key values, while ensuring correlations and referential integrity are preserved.                                                                | Upsampling, or situations where the original key values must be maintained for direct referencing, such as cases where exact key values are essential for business logic (e.g. country codes) or traceability in audit scenarios.                                     |

***

## [Foreign key management in Syntho](/configure-a-data-generation-job/manage-foreign-keys)

### Interactive guide: How to manage foreign keys

Follow the interactive guide below to manage foreign keys

{% embed url="<https://www.guidejar.com/guides/ed309b4d-e131-4f50-8ac1-6d3396d88c23>" %}

#### Best practices

* Define foreign keys in your source database where possible
* Use **Hash** to anonymize keys while keeping relationships intact
* Use **Generate** to create entirely new key structures
* Avoid **Duplicate** when privacy or transformation of key values is required


# 7. Workspace synchronization & validation

Before generating synthetic data, Syntho performs [automated checks](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace) to ensure your workspace aligns with the schemas of your **source** and **destination** databases. This ensures compatibility, preserves referential integrity, and prevents generation failures.

***

### Interactive guide: How to validate a workspace

Follow the interactive guide below to validate a workspace

{% embed url="<https://www.guidejar.com/guides/8547be69-4ff3-49ca-b60a-4f1718463fe2>" %}

## [Source schema validation](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace#source-schema-validation)

Your source database schema may evolve over time—tables may be added, columns removed, or data types changed. Syntho automatically detects such changes during generation and warns you if your workspace is out of sync.

#### [**Automatically detected changes**](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace#source-schema-synchronization)

Syntho will detect and synchronize:

* **Table-level changes**: primary key updates, constraint modifications, table removal
* **Column-level changes**: type changes, nullable status, constraints, or key assignments

Unresolved schema mismatches may result in job failure, so it’s best practice to sync regularly.

***

## [Destination schema validation](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace#destination-schema-validation)

Syntho also validates your destination schema to ensure it is ready to receive the generated synthetic data. Issues here can block the generation process or result in incorrect output.

### [**Table-level issues (blocking)**](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace#table-issues)

* Table does not exist
* Primary or foreign key mismatch with source

#### **Resolution**

Ensure table structure matches the source schema. Remove or align keys as needed.

### [**Table-level warnings (non-blocking)**](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace#table-warnings)

* Table already contains data\
  → Truncate the table in the destination
* Table is part of a circular reference\
  → Disable foreign key constraints temporarily

### [**Column-level issues (blocking)**](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace#column-issues)

* Missing columns
* Extra non-nullable columns without defaults
* Not-nullable columns not present in the source

#### **Resolution**

Adjust column definitions to match source schema exactly.

### [**Column-level warnings (non-blocking)**](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace#column-warnings)

* Data type mismatches
* Unexpected uniqueness or check constraints
* Missing default values
* Column length or max value is lower than source

#### **Resolution**

Align data types, constraints, and value limits between source and destination columns.

***

## Final step: generate

After resolving all validation issues and confirming schema synchronization, you can safely proceed to generate synthetic data.


# 8. Workspace & user management

Syntho provides robust tools for [managing your workspaces](/setup-workspaces/view-workspaces) and collaborating with other users. This section explains how to create, edit, share, and manage workspace permissions, as well as how to assign roles and transfer ownership.

***

## [Viewing and managing workspaces](/setup-workspaces/view-workspaces)

{% embed url="<https://www.guidejar.com/guides/f9e85458-2642-49f4-8446-86594152c4b0>" %}

***

## [Creating a workspace](/setup-workspaces/create-a-workspace)

{% embed url="<https://www.guidejar.com/guides/d27734b5-703a-4aa7-9db8-f040717bb28a>" %}

{% hint style="info" %}
The source and destination databases must be of the same type.
{% endhint %}

***

## [Editing a workspace](/setup-workspaces/edit-a-workspace)

{% embed url="<https://www.guidejar.com/guides/4f692004-285d-46e0-818b-3d5fa434b7de>" %}

***

## [Duplicating a workspace](/setup-workspaces/duplicate-a-workspace)

{% embed url="<https://www.guidejar.com/guides/2936e5b7-d3f8-4bd6-9b8e-c6e0e1fcf57b>" %}

***

## [Deleting a workspace](/setup-workspaces/delete-a-workspace)

{% embed url="<https://www.guidejar.com/guides/e139b5b3-6adb-4838-bbef-8b29c752ce3b>" %}

{% hint style="info" %}
This is permanent and cannot be undone.
{% endhint %}

***

## [Sharing a workspace](/setup-workspaces/share-a-workspace)

{% embed url="<https://www.guidejar.com/guides/5bf1d1f4-c366-4457-95d9-04e72d3c2f2d>" %}

***

## [Transferring workspace ownership](/setup-workspaces/transfer-workspace-ownership)

{% embed url="<https://www.guidejar.com/guides/599b28cb-fe17-420d-adcb-7a52bbc8a512>" %}

***

## [Role permissions](/setup-workspaces/share-a-workspace#available-workspace-roles)

Each user in a workspace is assigned one of three roles:

| Action                                | Reader   | Editor   | Owner |
| ------------------------------------- | -------- | -------- | ----- |
| View settings & history               | ✓        | ✓        | ✓     |
| View data                             | Optional | Optional | ✓     |
| Edit settings                         |          | ✓        | ✓     |
| Edit/delete/rename workspace          |          |          | ✓     |
| Run scans, generate data, sync schema | ✓        | ✓        | ✓     |
| Share, duplicate, transfer ownership  |          |          | ✓     |

{% hint style="info" %}
Owners have full permissions. Editors can modify configurations. Readers have view-only access unless specified otherwise.
{% endhint %}

***

This concludes the Syntho Bootcamp’s foundational modules. You’re now ready to configure, collaborate, and generate safe, privacy-preserving synthetic data using Syntho.


# 9. Large workloads​

## Large workloads

### Speeding up data generation jobs and reducing memory footprint

Working with large databases can significantly impact the performance and success of your synthetic data generation jobs. These tips will help you configure your workspace for large workloads by minimizing memory consumption and optimizing execution speed.

***

### Lowering memory footprint

To reduce memory usage and avoid potential timeouts or job failures, consider these strategies:

* [**Decrease parallel connections**](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings#advanced-generation-settings): Lower the number of concurrent connections to reduce memory usage.
* [**Decrease batch size**](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings#advanced-generation-settings): Smaller batches consume less memory per operation.
* [**Limit free text PII detection**](/configure-a-data-generation-job/configure-column-settings/duplicate/automatic-pii-discovery-and-de-identification-in-free-text-columns#use-syntho-pii-text-scanner): This is a resource-intensive process. Only enable it when absolutely necessary.
* [**Reduce the number of training rows**](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#advanced-generator-settings) *(AI synthesis only)*: Limiting the training data size speeds up processing and conserves resources.

***

### Speeding up data generation jobs

To accelerate data generation for large-scale datasets, apply the following optimizations:

* [**Increase parallel connections**](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings#advanced-generation-settings): More connections can speed up data reading and writing through parallel execution.
* **Enable schema-independent scheduling**: By removing constraints in the destination schema, Syntho can parallelize processing based on the number of records instead of schema dependencies.
* **Write to Parquet instead of a database**: Writing directly to a database is often slower. When dealing with very large datasets, consider exporting to efficient columnar file formats like Parquet.

### Interactive guide: How to handle large workloads

Follow the interactive guide below to handle large workloads

{% embed url="<https://www.guidejar.com/guides/b84216d2-d33f-4f42-9aed-fbfad9c4a10c>" %}

***

### Best practice

Always aim to use the minimal viable dataset to validate your configurations before executing large jobs. Scaling up becomes much easier and more stable when you're confident in your setup.


# 10. AI synthesis: Data pre-processing when using

## Preparing your data

When using AI synthesis, to ensure the highest quality of synthetic data output, proper data preparation is essential. Below are guidelines on how to best prepare your dataset before initiating a generation job.

***

### Preparing your data – entity table

When working with standalone or flat tables, consider the following practices:

1. [**Maintain a column-to-row ratio of at least 1:500**](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#entity-tables)\
   This minimizes privacy risks and improves generalization. For example, a table with 6 columns should ideally have a minimum of 3,000 rows.
2. [**Each entity should be described in one row**<br>](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#entity-tables)One row per unique entity avoids data fragmentation.
3. [**Ensure each row is independent**](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#entity-tables)\
   The order of rows should not affect the dataset. Each row must be self-contained and analyzable on its own.
4. [**Avoid privacy-sensitive column names**](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#entity-tables)\
   For instance, do not use names like `patient_a_medications`. Instead, consolidate sensitive names under generic columns like `patient`.
5. [**Remove derived or redundant columns**](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#entity-tables)\
   If one column is a direct function of another (e.g., `duration = end_time - start_time`), remove the derived column. This also includes categorical redundancies, such as having both `treatment` and `treatment_category`.

***

By adhering to these data preparation guidelines, you ensure that your AI model learns from meaningful patterns, avoids overfitting on redundant information, and respects privacy constraints. This leads to stronger and more reliable synthetic data generation.


# 11. Continuous Success

Syntho is not just a tool—it’s a platform for continuous improvement, scalable adoption, and long-term value generation across your organization. This section will guide you from initial usage to full-scale deployment and show how to engage with Syntho’s ongoing support and feedback ecosystem.

***

## From Initial Usage to Broad Adoption

### **Phase 1: Data Distribution – Initial Usage**

Start small and demonstrate Syntho's value quickly:

* **Deploy in a controlled setting**: Begin with a small-scale or Syntho-hosted deployment.
* **Train your first users**: Identify initial users and provide hands-on training.
* **Functionally evaluate the platform**: Test and validate key use cases.
* **Iterative refinement**: Configure workspaces based on early findings.
* **Expand training**: Gradually train more early adopters and users.

{% hint style="success" %}
This phase ensures confidence and builds internal momentum.
{% endhint %}

***

### **Phase 2: Application Access Distribution – Broad Adoption**

Scale across teams and environments:

* **Go production-ready** (if needed): Deploy in your preferred environment with governance controls in place.
* **Run internal enablement sessions**: Showcase real-world use cases and success stories.
* **Train additional teams and users**: Expand platform usage across the organization.

{% hint style="success" %}
This phase focuses on operationalizing Syntho as a day-to-day solution.
{% endhint %}

***

## Syntho Support & Enablement

#### **Syntho’s Support Channels**

Need help? You’re never alone:

* 📧 Email: <support@syntho.ai>
* 📲 Or use your dedicated support communication channel.

**User Documentation**

* 📚 Access comprehensive guides and FAQs at:\
  [docs.syntho.ai](https://docs.syntho.ai/)\
  and our [Getting Started Guides](https://docs.syntho.ai/getting-started)

**Feedback & Improvement**

* Participate in quarterly check-ins with Syntho's team.
* Share feedback to shape the roadmap and improve the platform.

***

#### ✅ Your Ongoing Journey with Syntho

By following this framework, you can:

* Ensure sustainable platform adoption
* Continuously improve synthetic data workflows
* Empower more teams with privacy-safe and statistically accurate data
* Establish long-term data governance and innovation

Unlock the full potential of Syntho—your journey to privacy-first, AI-powered data begins here.


# Example data generation scenarios

The examples below show how you can combine different Syntho generators to get the data that you need for your use case. Note that:

{% hint style="info" %}
For multi-table setups:

**AI synthesis** works best on **single tables**.

If you need to preserve **cross-table relationships**, **replace the AI synthesis examples below with duplicate,** or **mask/mock/rule-based generators using consistent mapping.**
{% endhint %}

### 1. Deterministic relations (e.g. male → male name)

#### Scenario

Hospital EHR table `patients` with:

* `patient_id`
* `gender` (M/F/X)
* `first_name`, `last_name`
* `date_of_birth`
* `diagnosis_code`
* `visit_count`

#### How you combine Syntho methods

* **AI synthesis** for:
  * `date_of_birth`, `diagnosis_code`, `visit_count`, and other clinical / behavioral fields, so you preserve realistic age distributions, ICD code combinations, visit patterns, etc., without 1-to-1 linkage to real people.
* **Mockers + calculated columns** for deterministic gender–name relation:
  * Use **Calculated Column** with mockers for first\_name and last\_name:\
    `IF([gender] = 'M', MOCK_FIRST_NAME_MALE, IF([gender] = 'F', MOCK_FIRST_NAME_FEMALE, MOCK_FIRST_NAME_UNISEX))`
* **Masking** for IDs:
  * Hash `patient_id` so you can still join across tables but never see the real identifier.

#### Added value

* **Business rule testing**: devs can test logic like “if gender = female, show pregnancy-related questions” on fully synthetic data that *always* respects the gender–name relation.
* **Realistic UX demos**: clinicians and product owners see plausible names matching gender, not nonsense combinations that break trust.

### 2. Absolute calculations (e.g. revenue – costs = profit)

#### Scenario

Banking `accounts` table:

* `account_id`
* `revenue`
* `costs`
* `profit`
* `segment`, `region`, etc.

In real systems, `profit = revenue – costs` and many downstream rules rely on that.

#### How you combine Syntho methods

* **AI synthesis**:
  * Use AI synthesis to generate realistic **joint distributions** of `revenue`, `costs`, `segment`, `region`, churn risk, etc. (statistically similar to real portfolio but no 1:1 linkage).
* **Rule-based calculated columns**:
  * Overwrite `profit` with a **Calculated Column**:\
    `[revenue] - [costs]`
  * Optionally add extra business logic:\
    `IF([revenue] - [costs] < 0, 0, [revenue] - [costs])` for scenarios where profit is stored as 0 instead of negative.
* **Masking / hashing**:
  * Hash `account_id` and mask PII (e.g. `IBAN`) while preserving format.

#### Added value

* **Guaranteed accounting consistency**: AI handles realistic ranges and correlations; rule-based logic guarantees accounting identities exactly match business rules so regression tests never fail on “impossible” numbers.
* **Stress testing analytics**: you can safely share this data with external consultants / vendors; they can calculate margins, perform profitability analysis and validate models, knowing the math holds.
* **Explaining model results**: analysts can cross-check their profitability dashboards on synthetic data that behaves exactly like production in terms of formulas.

### 3. Hierarchical relationship (City > Province > Country)

#### Scenario

Government benefits system:

* `citizen_id`
* `city`
* `province`
* `country`
* `benefit_type`
* `benefit_amount`

City, province and country must **always** form valid combinations.

#### How you combine Syntho methods

* **AI synthesis**:
  * Use AI synthesis on the *entity table* to generate realistic patterns for `benefit_type`, `benefit_amount`, `city` , demographics, etc.
* **Rule-based + mockers for geography hierarchy**:
  * Keep a real or mock **reference table** `geo_dim`:
    * `city`, `province`, `country` with valid combinations.
  * Use **Calculated Columns** or lookups to set `province` and `country` based on `city`:
    * e.g.

      ```
      SWITCH(UPPER(TRIM([CITY])), "TORONTO", "ONTARIO", "MONTREAL", "QUEBEC", ...)
      ```
* **Masking**:
  * For privacy, mask or aggregate small cities into “Other city (Province)” to avoid re-identification, while still keeping hierarchical consistency.

#### Added value

* **Location-based logic behaves correctly** (regional eligibility rules, tax brackets, language settings), because city–province–country combos are always valid.
* **Geospatial analytics** (heatmaps, per-province dashboards) remain meaningful on synthetic data, enabling external sharing without privacy issues.
* **Complex test scenarios**: QA can create rule-based test cases (e.g. cross-border regions, provinces with special rules) without messing up global statistics that AI synthesis preserves.

### 4. New data creation

#### Scenario

You’re launching a **new SaaS product** and don’t have real customers yet, but you want:

* A realistic multi-tenant database for demos and end-to-end integration testing.
* Later, once you have data, to augment it with AI-synthesized rows.

#### How you combine Syntho methods

1. **Phase 1 – no data yet (pure rule-based / mock)**
   * Use **Mock generators** to generate:
     * `company_name`, `user_email`, `first_name`, `last_name`, etc.
   * **Rule-based calculated columns**:
     * Column: tenant\_id
       * Set generator in column settings to Key generators → Generate. (Creates unique synthetic keys; no formula needed.)
     * Column: user\_email

       ```
       // Company email per user
       LOWER(CONCATENATE([FIRST_NAME], ".", [LAST_NAME], "@", MOCK_FREE_EMAIL_DOMAIN_0))
       ```
     * Column: role (weighted distribution)

       ```
       // ~10% OWNER, 20% ADMIN, 70% MEMBER
       IFS(
         RANDBETWEEN(1,100) <= 10,  "OWNER",
         RANDBETWEEN(1,100) <= 30,  "ADMIN",
         TRUE,                      "MEMBER"
       )
       ```
     * Column: signup\_date (recent signups)

       ```
       DATEADD(TODAY(), -RANDBETWEEN(0, 90), "day")
       ```
     * Column: trial\_end\_date = signup\_date + 14 days

       ```
       DATEADD([signup_date], 14, "day")
       ```
     * Column: plan\_tier (map by company size proxy)

       ```
       // Size proxy via random users per tenant influences tier
       SWITCH(TRUE,
         RANDBETWEEN(1,100) <= 10, "ENTERPRISE",
         RANDBETWEEN(1,100) <= 40, "PRO",
         "FREE"
       )
       ```
2. **Phase 2 – once you have initial data**:
   * Optionally, train on **AI synthesis** on the production entity table (per tenant or global) to create more users, sessions, transactions, etc., preserving correlations (usage patterns, feature adoption).
   * Combine with **calculated columns** to enforce specific business rules (e.g. multi-tenant isolation, SLA tiers).
     * Column: sla\_tier (carry forward or rebalance)

       ```
       // Keep existing if present, else assign
       IF(ISNULL([sla_tier]),
          IFS(RANDBETWEEN(1,100)<=10,"GOLD", RANDBETWEEN(1,100)<=40,"SILVER", TRUE,"BRONZE"),
          [sla_tier]
       )
       ```
     * Column: response\_sla\_hours (business rule from SLA tier)

       ```
       SWITCH(UPPER(TRIM([sla_tier])),
         "GOLD",   4,
         "SILVER", 8,
         "BRONZE", 24,
         24
       )
       ```
     * Column: trial\_end\_date (ensure consistent with signup\_date from mixed sources)

       ```
       IF(ISNULL([trial_end_date]),
          DATEADD([signup_date], 14, "day"),
          [trial_end_date]
       )
       ```
     * Column: environment\_flag (separate demo/test tenants)

       ```
       // Oversample demo tenants for testing flows
       IFS(
         RANDBETWEEN(1,100) <= 20, "DEMO",
         TRUE,                    "PROD"
       )
       ```
     * Column: tenant\_isolation\_guard (hard guard rails)

       ```
       // Example: block cross-tenant sharing flag in demo data
       IF([environment_flag]="DEMO", FALSE, [sharing_enabled])
       ```
3. **Add masking when seeding from production**:
   * If you seed from a production snapshot, use calculated columns/mask/mock for PII and AI/duplicate/mask/mock/calculated columns for behavioral attributes.

#### Added value

* **Instantly available realistic test DB** even before going live.
* As you grow, **AI synthesis scales up data volume** with realistic patterns, while rule-based generators maintain product-specific rules and identities (tenants, subscription tiers).

### 5. Edge case and rare scenario creation

#### Scenario

Health insurance claims system where:

* 99.9% of cases are “normal”
* But test teams need **lots of edge cases**:
  * extremely high claim amounts
  * rare combinations of diagnosis codes
  * weird date patterns (backdated claims, overlapping coverage)
  * special product types with exception logic

#### How you combine Syntho methods

* **AI synthesis**:
  * Generate the **normal background population** of claims based on your source data: realistic volumes, distributions, seasonal patterns, co-occurrence of diagnoses & procedures.
* **Rule-based edge case injection**:
  * Use **Calculated Columns** and **Mockers** to override or add specific rare scenarios on top of AI synthetic data:
    * Selector for 0.1% edge rows.

      ```
      // New column: EDGE_FLAG (TRUE for ~0.1% of rows)
      RAND() < 0.001
      ```
    * Extremely high claim amounts

      ```
      // Column: claim_amount
      IF([EDGE_FLAG],
         RANDBETWEEN(100000, 500000),
         [claim_amount]
      )
      ```
    * Rare diagnosis codes (weighted)

      ```
      // Column: diagnosis_code
      IFS(
        AND([EDGE_FLAG], RAND() < 0.33),      "E75.5",           // lysosomal storage d/o (example)
        AND([EDGE_FLAG], RAND() < 0.66),      "G12.2",           // motor neuron d/o (example)
        AND([EDGE_FLAG], TRUE),               "D42.0",           // rare tumor (example)
        TRUE,                                 [diagnosis_code]   // default keep
      )
      ```
    * Force product/diagnosis combos that trigger rules

      ```
      // Column: product_type
      IF([EDGE_FLAG], "SPECIAL_X", [product_type])
      ```

      ```
      // Column: diagnosis_code (pair with SPECIAL_X)
      IF([EDGE_FLAG], "Q87.1", [diagnosis_code])
      ```
    * Weird date patterns: backdated claims

      ```
      // Column: claim_date (move back 30–365 days)
      IF([EDGE_FLAG],
         DATEADD([claim_date], -RANDBETWEEN(30, 365), "day"),
         [claim_date]
      )
      ```
    * Weird date patterns: coverage overlaps or ends before claim

      ```
      // Column: coverage_end_date (ensure < claim_date to trigger exception)
      IF([EDGE_FLAG],
         DATEADD([claim_date], -RANDBETWEEN(1, 30), "day"),
         [coverage_end_date]
      )
      ```
    * Flip eligibility flags or set error codes

      ```
      // Column: eligible
      IF([EDGE_FLAG], FALSE, [eligible])
      ```

      ```
      // Column: error_code
      IF([EDGE_FLAG], "E_RULE_123", [error_code])
      ```
* **Masking**:
  * If you started from a de-identified copy of real claims, mock/mask remaining identifiers while keeping relationships so that edge cases are still context-rich.

#### Added value

* **Massive coverage of “what-if” conditions** without having to wait for those rare cases in production; key for regression testing and rules engines.
* **Regulatory / audit confidence**: you can prove that all critical business rules, alerts and exception flows have been tested against realistic but privacy-safe data.
* **Balanced datasets for QA and ML**: AI synthesis covers privacy and keeps realistic distributions, but rule-based injects edge-case oversampling where you need it (e.g. fraud, high-cost outliers).


# Go-live requirements

Use this page as the go-live requirements checklist for Syntho. Items labelled with a ⏰ icon usually take most time, start them as early as possible. Appoint a single accountable implementation lead to oversee and coordinate all implementation activities end-to-end, including onboarding, architecture, infrastructure, security, DevOps, database access, compliance, approval processes, and use-case prioritization.

{% hint style="info" %}
The Implementation Statement of Work Status Tracker contains the full list and latest status.
{% endhint %}

***

### A - Scoping & success

* [ ] Top 3–5 prioritized use cases identified (including database(s)/schema(s)). Usually more time consuming for non TDM use-cases ⏰
* [ ] Priority use case(s) selected
* [ ] Use-case execution order defined
* [ ] 1–3 super users assigned
* [ ] Super users are available 2–3 days per week during onboarding
* [ ] Super users are comfortable with database concepts (tables, permissions, referential integrity)
* [ ] If you use AI-powered generation a super user can prepare an [entity table](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#entity-tables) if needed

***

### B - Architecture & data flow

* [ ] Onboarding procedure approvals completed (as applicable) ⏰
  * [ ] InfoSec review
  * [ ] Vendor risk / procurement intake
  * [ ] Privacy officer / board approval
  * [ ] AI review (if applicable)
* [ ] Architecture defined, documented and approved ⏰
  * [ ] Network zone placement decided
  * [ ] Intended data flow documented (production → source → Syntho → destination)
  * [ ] Security requirements agreed
  * [ ] Architecture diagram available
* [ ] Data governance plan defined (if needed)
  * [ ] Data access model and approvals documented
  * [ ] Audit/compliance requirements documented
  * [ ] Data retention policy documented

***

### C - Deployment, databases & users

#### C-1 - Deployment

* [ ] Deployment method selected: [Docker Compose](/deploy-syntho/deploy-syntho-using-docker) or [Kubernetes (Helm)](/deploy-syntho/deploy-syntho-using-kubernetes)
* [ ] Deployment prerequisites met
  * Docker Compose: [Prerequisites](/deploy-syntho/deploy-syntho-using-docker/prerequisites)
  * Kubernetes (Helm): [Prerequisites](/deploy-syntho/deploy-syntho-using-kubernetes/prerequisites)
* [ ] Hardware sizing confirmed: [Deployment overview](/deploy-syntho/introduction)
* [ ] Registry access works (client can authenticate and pull images)
* [ ] If required, outbound access to `syntho.azurecr.io` is whitelisted (or images are mirrored to a local registry)
* [ ] License key received and available for deployment
* [ ] Deployment instructions received (and shared with the deployment team) ([Docker Compose](/deploy-syntho/deploy-syntho-using-docker), [Kubernetes (Helm)](/deploy-syntho/deploy-syntho-using-kubernetes))
* [ ] Infrastructure provisioned (compute, storage, networking, DNS as needed) ⏰
* [ ] Infrastructure readiness confirmed (required compute/storage available, DNS prepared if used)
* [ ] Syntho deployed and reachable by intended users
* [ ] UI reachable over HTTP(S). DNS/TLS and ingress/proxy posture defined (if applicable) ⏰
* [ ] If using DNS/TLS, hostname resolves and HTTPS works without browser warnings
* [ ] Deployment validated (UI loads, admin login works, Ray dashboard visible, logs clean) (see [Logs and monitoring](/deploy-syntho/logs-and-monitoring) and troubleshooting: [Docker Compose](/deploy-syntho/deploy-syntho-using-docker/troubleshooting), [Kubernetes](/deploy-syntho/deploy-syntho-using-kubernetes/troubleshooting))
* [ ] Backups confirmed functional (policy defined + restore tested):
  * Docker Compose: [Back up PostgreSQL](/deploy-syntho/deploy-syntho-using-docker/back-up-postgresql)
  * Kubernetes: [Back up PostgreSQL](/deploy-syntho/deploy-syntho-using-kubernetes/back-up-postgresql)

#### C-2 - Databases

* [ ] Data owner(s) identified for the priority use-case dataset/database
* [ ] Source and destination access request process started (firewall/proxy rules, permissions) ⏰
* [ ] Source database prepared for read-only access (database needs to be static during data generation, use a consistent snapshot) ⏰
* [ ] Destination database prepared for writing synthetic data (schema aligned) ⏰
* [ ] Source DB connectivity + read-only permissions validated ([Step 1. Validate source db](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace), [Connect to a database](/setup-workspaces/create-a-workspace/connect-to-a-database))
* [ ] Destination DB connectivity + write/truncate permissions validated ([Step 2. Validate destination db](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace-1), [Connect to a database](/setup-workspaces/create-a-workspace/connect-to-a-database))
* [ ] First end-to-end data generation job completed successfully

#### C-3 - Users

* [ ] Admin user exists: [Manage admin users](/deploy-syntho/manage-users-and-access/manage-admin-users)
* [ ] Non-admin users provisioned (Owner/Editor/Reader): [Manage non-admin users](/deploy-syntho/manage-users-and-access/manage-non-admin-users)
* [ ] If using SSO, configured and tested: [Single Sign-On (SSO) in Azure](/deploy-syntho/manage-users-and-access/single-sign-on-sso-in-azure) ⏰
* [ ] Database users created for Super Users (source read-only, destination write, alter & truncate) ⏰
* [ ] Super User credentials shared securely (platform + database)
* [ ] Super Users can log in and complete the full flow without errors
  * [ ] Connect source and destination ([Connect to a database](/setup-workspaces/create-a-workspace/connect-to-a-database))
  * [ ] Create a workspace ([Create a workspace](/setup-workspaces/create-a-workspace))
  * [ ] Validate source and destination ([Step 1. Validate source db](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace), [Step 2. Validate destination db](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace-1))
  * [ ] Run a generation job end-to-end ([Step 3. Generate](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings))


# Use Cases & Configuration

Start with your goal. Then pick the first use case to implement. Each use case optimizes for a different outcome. For example,. testing, realism, privacy, consistency, or speed. Each has different configuration requirements.

### Getting started

Before you dive into a use case, make sure the basics are covered:

* [Prerequisites](/overview/get-started/prerequisites)
* [Deploy Syntho](/deploy-syntho/introduction)

#### Generation approaches

* **AI-generated synthesis**: best when you need statistical utility with strong privacy, or extra rows.
* **Rule-based generation**: best when values must follow explicit business logic.
* **Masking / de-identification**: best when you need format-preserving replacements and stable keys/relationships.
* **Hybrid**: best when one approach alone does not meet your requirements.

#### Key configuration decisions

These decisions drive most success (and most rework).

**1) Pick the workspace mode that matches your starting point**

* **De-identify**: you already have a production-like dataset and mainly need to replace identifiers.
* **Mock or mask all**: you need “production-like” formats but you don’t want to keep original values.
* **Mock all**: you have little/no source data and want to generate everything from scratch.
* **Synthesize all**: you have enough rows and want maximum statistical utility with strong privacy.

**2) Decide if you should reshape to a single entity table**

AI synthesis works best on a single table. It is often worth creating a SQL view first (especially for ML, analytics and data sharing).

* [Use SQL views as input tables](/setup-workspaces/create-a-workspace/use-sql-views-as-input-tables)
* [Cross-table relationships limitations](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation/table-relationships)

**3) Choose masking, rule-based, and AI synthesis**

* Use **masking** when downstream systems validate formats (emails, IBANs, UUIDs).
* Use **rule-based / calculated columns** when the business logic must always hold (profit = revenue - costs).
* Use **AI synthesis** when you need privacy + statistical utility for indirect identifiers (age, gender, weight).

#### Governance, compliance, and automation (reference)

* Use workspace roles and sharing to control who can view data and change generators. See [Workspace & user management](/overview/get-started/syntho-bootcamp/8.-workspace-and-user-management).
* Automate workspace setup, scans, and runs via the [Syntho REST API](/syntho-api/syntho-rest-api).
* For deployment options, see [Deploy Syntho](/deploy-syntho/introduction).

### Use cases

<table data-card-size="large" data-view="cards"><thead><tr><th></th><th data-hidden data-card-cover data-type="image">Cover image</th><th data-hidden></th><th data-hidden></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Use Case 1: Application &#x26; API Testing</strong><br>Privacy-safe, production-like test data for application and API tests in non-production.</td><td><a href="/files/I0kbZfdGenRDB21GS5vO">/files/I0kbZfdGenRDB21GS5vO</a></td><td></td><td></td><td><a href="/pages/034fkmKsQX3DK0iB7UVm">/pages/034fkmKsQX3DK0iB7UVm</a></td></tr><tr><td><strong>Use Case 2: Load &#x26; Stress</strong><br>Generate large volumes and edge cases for performance testing without sensitive production data.</td><td><a href="/files/GzMpv3u3uevUiBGFKFqD">/files/GzMpv3u3uevUiBGFKFqD</a></td><td></td><td></td><td><a href="/pages/NkTf8L2XkXSNqgHYP8MA">/pages/NkTf8L2XkXSNqgHYP8MA</a></td></tr><tr><td><strong>Use Case 3: Demo Data</strong><br>Realistic demo data that contains no real identifiers and can be shared safely.</td><td><a href="/files/ujHbESJyWHjW78cL7Qu6">/files/ujHbESJyWHjW78cL7Qu6</a></td><td></td><td></td><td><a href="/pages/gtzgHUwDJVk6MFeP72L9">/pages/gtzgHUwDJVk6MFeP72L9</a></td></tr><tr><td><strong>Use Case 4: ETL &#x26; Data Pipeline Testing</strong><br>End-to-end pipeline testing without using production data.</td><td><a href="/files/oaK560iE4ICpL1AhHdLO">/files/oaK560iE4ICpL1AhHdLO</a></td><td></td><td></td><td><a href="/pages/m0lRco6Dnu1jsuaHJaEA">/pages/m0lRco6Dnu1jsuaHJaEA</a></td></tr><tr><td><strong>Use Case 5: Feature Development</strong><br>Shift-left testing with realistic synthetic data when production data is unavailable or restricted.</td><td><a href="/files/GzMpv3u3uevUiBGFKFqD">/files/GzMpv3u3uevUiBGFKFqD</a></td><td></td><td></td><td><a href="/pages/OABjH6uhizuKtzt5cuCG">/pages/OABjH6uhizuKtzt5cuCG</a></td></tr><tr><td><strong>Use Case 6: ML Model Development</strong><br>Generate feature datasets when real data is scarce or sensitive.</td><td><a href="/files/ujHbESJyWHjW78cL7Qu6">/files/ujHbESJyWHjW78cL7Qu6</a></td><td></td><td></td><td><a href="/pages/c4s5IDwQiiYHo4SD8Lb8">/pages/c4s5IDwQiiYHo4SD8Lb8</a></td></tr><tr><td><strong>Use Case 7: Analytics Sandboxes</strong><br>Secure sandboxes for exploratory analytics and data science.</td><td><a href="/files/ujHbESJyWHjW78cL7Qu6">/files/ujHbESJyWHjW78cL7Qu6</a></td><td></td><td></td><td><a href="/pages/n2MVcgykJHrCjz460ag5">/pages/n2MVcgykJHrCjz460ag5</a></td></tr><tr><td><strong>Use Case 8: Cloud &#x26; Data Migration</strong><br>Validate data workflows during migrations.</td><td><a href="/files/q1kuF8aCbmqiHC1gUDLz">/files/q1kuF8aCbmqiHC1gUDLz</a></td><td></td><td></td><td><a href="/pages/P8PBQJs0PbTAZSlhQmlX">/pages/P8PBQJs0PbTAZSlhQmlX</a></td></tr><tr><td><strong>Use Case 9: Data Sharing &#x26; Monetization</strong><br>Share data with strong privacy protection.</td><td><a href="/files/q1kuF8aCbmqiHC1gUDLz">/files/q1kuF8aCbmqiHC1gUDLz</a></td><td></td><td></td><td><a href="/pages/qjGJAhVnPr3eAhxOw7do">/pages/qjGJAhVnPr3eAhxOw7do</a></td></tr><tr><td><strong>Use Case 10: Data Subsetting</strong><br>Reduce data footprint while preserving integrity.</td><td><a href="/files/pRyl3u4klwSFcFnF2Lrk">/files/pRyl3u4klwSFcFnF2Lrk</a></td><td></td><td></td><td><a href="/pages/xq3wmL0rf8lmm7PUdIDe">/pages/xq3wmL0rf8lmm7PUdIDe</a></td></tr><tr><td><strong>Use Case 11: Accelerate PoCs &#x26; Pilots</strong><br>Deliver privacy-safe datasets fast to validate ideas, integrations, and workflows.</td><td><a href="/files/q1kuF8aCbmqiHC1gUDLz">/files/q1kuF8aCbmqiHC1gUDLz</a></td><td></td><td></td><td><a href="/pages/dRqPKvaPcwWLB81BtcED">/pages/dRqPKvaPcwWLB81BtcED</a></td></tr><tr><td><strong>Use Case 12: Training &#x26; Education</strong><br>Create safe, realistic datasets for onboarding, workshops, and hands-on training.</td><td><a href="/files/pRyl3u4klwSFcFnF2Lrk">/files/pRyl3u4klwSFcFnF2Lrk</a></td><td></td><td></td><td><a href="/pages/pGk7k01EA5f5uwtQyyfs">/pages/pGk7k01EA5f5uwtQyyfs</a></td></tr></tbody></table>

<details>

<summary>Baseline workflow (applies to every use case)</summary>

Use this checklist to go from “use case” to a repeatable job.

{% stepper %}
{% step %}

### Prerequisites

Confirm access, schema alignment, and environment readiness.

* [Prerequisites](/overview/get-started/prerequisites)
  {% endstep %}

{% step %}

### Create a workspace

Pick the source + destination, then choose a workspace mode that matches your starting point.

* [Create a workspace](/setup-workspaces/create-a-workspace)
* [Workspace modes](/setup-workspaces/create-a-workspace/workspace-modes)
  {% endstep %}

{% step %}

### Configure generators

Start from the simplest approach that meets the goal.

* [Introduction to data generators](/overview/get-started/introduction-to-data-generators)
* [Generators](/configure-a-data-generation-job/configure-column-settings)
  {% endstep %}

{% step %}

### Handle keys and relationships (relational schemas)

Make FK behavior explicit before your first big run.

* [Referential integrity & foreign keys](/overview/get-started/syntho-bootcamp/6.-referential-integrity-and-foreign-keys)
* [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys)
* [Key generators](/configure-a-data-generation-job/configure-column-settings/key-generators)
  {% endstep %}

{% step %}

### Validate and sync

Validate early, then resync whenever the schema drifts.

* [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace)
  {% endstep %}

{% step %}

### Tune generation settings

Optimize performance and reduce write errors before scaling up.

* [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings)
* [Large workloads](/overview/get-started/syntho-bootcamp/9.-large-workloads)
  {% endstep %}
  {% endstepper %}

</details>


# Use Case 1: Application & API testing

Privacy-safe, production-like test data for application and API tests in non-production.

Use this use case when you need privacy-safe, production-like data for application and API testing. You keep realistic system behavior. You avoid exposing PII/PHI in non-production.

### What problem this use case solves

Teams need stable test data for UI flows, integration tests, and API tests. Production copies are often blocked by privacy regulations. Manual test data creation is slow and not representative.

### When to choose this use case

Pick this when you need production-like test data without privacy risks:

* You test applications and APIs in `dev`, `test`, or `accept`.
* Your tests need valid formats (email, UUID, IBAN, dates).
* Your tests need stable joins across tables and refreshes.
* Your schema changes over time and you need reproducible failures.

### When to avoid this use case

Skip this use case when functional testing is not your main goal:

* You mainly need load or stress testing at huge volumes. Use [Use Case 2: Load & Stress](/overview/get-started/use-cases-and-configuration/use-case-2-load-and-stress-testing).
* You do not have production data available yet. Use [Use Case 5: Feature Development](/overview/get-started/use-cases-and-configuration/use-case-5-feature-development).
* You need maximum protection for external data sharing. Use [Use Case 9: Data Sharing & Monetization](/overview/get-started/use-cases-and-configuration/use-case-9-data-sharing-and-monetization).

### Recommended Syntho configuration

This setup is optimized for **DTAP-style application and API testing**. You keep schemas and relationships intact. You replace identifiers and free text that can leak PII/PHI

{% stepper %}
{% step %}

#### Prerequisites

* Use the [Prerequisites](/overview/get-started/prerequisites) checklist.

**Checklist**

* [ ] Source is **non-production** (snapshot/copy), not live production.
* [ ] Destination is isolated (separate DB or schema).
* [ ] You know which columns must change (PII/PHI + quasi-identifiers).
* [ ] You have a small smoke test to validate the output.

{% hint style="warning" %}
Never write generated data back into production. Keep the destination isolated.
{% endhint %}
{% endstep %}

{% step %}

#### Source & destination management

Create one workspace per non-production environment. Examples: `dev`, `test`, `accept`.

#### Baseline rules

* Keep the **source stable**. Prefer snapshots or back-ups.
* Avoid a **live production** source for iterative work.
* Keep the **destination isolated**. Never write into production.
* Keep **schemas aligned** between source, workspace and destination.
* Use **views** when you need only a subset of the original database.

#### Lifecycle rule of thumb

* Keep the source connection when you expect schema changes.
* Remove the source connection when you expect a new run only much later.
* Revalidate after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).

**Nuances for this use case**

* Use the **exact production schema** (types, constraints, indexes). Tests often fail on subtle schema drift.
* Avoid a shared destination schemas. Collisions between teams create flaky tests.
* Dropped join keys break joins and consistent mapping.
  {% endstep %}

{% step %}

#### Configure generators

**Workspace initialization mode**

Choose a [workspace mode](/setup-workspaces/create-a-workspace/workspace-modes). It applies baseline generator suggestions during workspace creation. For DTAP, you start from an existing schema and data.

Recommended modes for this use case:

* **De-identify** when you have a production-like copy and only need to replace PII/PHI while keeping non-PII intact.
* **From scratch** when you only test a few tables and want full manual control.

**AI-generated synthesis**

Use this when you have indirect identifiers in your data that need masking. For example, a column with gender, age, and weight. AI synthesis will generate new values while preserving correlations.

**Rule-based generation**

Use this when tests must hit **explicit branches** and **edge cases**. Use [Calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns) to inject controlled exceptions.

**Example (negative API tests with a stable base):** create an `EDGE_FLAG` and only corrupt one validator field on those rows.

```excel-formula
// New column: EDGE_FLAG (≈1% of rows)
RAND() < 0.01
```

```excel-formula
// Override: postal_code (break format only for edge rows)
IF([EDGE_FLAG], "XX-INVALID", [postal_code])
```

**Masking**

Use this when you need **format-preserving replacements** and **stable keys/relationships** across tables and refreshes. This is the default workhorse for API/UI validators.

**Example:** apply **Mask → Email** on `customers.email`, **Mask → UUID** on `users.external_id`, and enable **Consistent mapping** for shared identifiers so joins and app flows stay intact

**Hybrid**

Use this when you need **relational correctness** plus **deterministic business logic**. Hybrid is the default for serious app testing. It maps to the patterns in [Example data generation scenarios](/overview/get-started/syntho-bootcamp/example-data-generation-scenarios).

**Example (deterministic relations + safe identifiers):** enforce “gender → name style” while keeping joins intact.

1. Keep PK/FK behavior stable with key generators and consistent mapping where needed.
2. Mask/replace direct identifiers (emails, phone numbers) for validator safety.
3. Use calculated columns to enforce deterministic relations inside a table.

```excel-formula
// New column: first_name_generated (deterministic by gender)
IFS(
  [gender] = "M", MOCK_FIRST_NAME_MALE,
  [gender] = "F", MOCK_FIRST_NAME_FEMALE,
  TRUE,          MOCK_FIRST_NAME
)
```

**Minimal configuration steps**

1. Run a [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner).
2. Set identifier columns to **Mask** or **Mock.**
3. Enable [Consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping) only for shared identifiers.
4. Use [Free text PII obfuscation](/overview/get-started/syntho-bootcamp/5.-generators/free-text-de-identification) only on the few text columns that need it.

{% hint style="warning" %}
Consistent mapping increases linkability. Enable it only for columns needed for joins and flows.
{% endhint %}

* [Automatic PII discovery with PII scanner](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner)
* [Manage personally identifiable information (PII)](/configure-a-data-generation-job/manage-personally-identifiable-information-pii)

<details>

<summary>Concrete example: from PII scan output to generator choices</summary>

Example workspace name: `test-api-contracts` (stable dataset refreshed per sprint).

Example PII scan findings you should expect to review (illustrative):

```
customers.email            -> EMAIL_ADDRESS (confidence 0.97)
customers.phone_number     -> PHONE_NUMBER  (confidence 0.93)
customers.date_of_birth    -> DATE_OF_BIRTH (confidence 0.88)
customers.notes            -> FREE_TEXT      (confidence 0.74)
orders.billing_iban        -> IBAN          (confidence 0.96)
```

Example generator mapping for API-friendly realism:

* `customers.email`: **Mask → Email** (keeps a valid email structure for validators).
* `customers.phone_number`: **Mask → Phone** (keeps country/format patterns).
* `orders.billing_iban`: **Mask → IBAN** (format-preserving, prevents checksum failures).
* `customers.date_of_birth`: **Mock** (often safer than masking when age distribution is not tested).
* `customers.notes`: **Free text PII obfuscation** (only on this column, not the whole table).

</details>
{% endstep %}

{% step %}

#### Handle keys and relationships (relational schemas)

If your test dataset is a **single table** with no joins, you can skip this step. Most app and API test failures are broken PK/FK chains.

Verify [foreign key inheritance](/configure-a-data-generation-job/manage-foreign-keys/foreign-key-inheritance). Add [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys) where the database doesn’t define them.
{% endstep %}

{% step %}

#### Validate and sync

Validate source via [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace). Run a small smoke suite. Include at least one API happy path.

Re-run validation whenever schemas drift. DTAP schemas drift often.
{% endstep %}

{% step %}

#### Tune generation settings

Tune for repeatability, not just speed. Keep settings stable across runs. This makes test failures debuggable. This also reduces flaky tests.

Use [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings) and [Large workloads](/overview/get-started/syntho-bootcamp/9.-large-workloads) tuning once the config is correct.
{% endstep %}
{% endstepper %}

### Common pitfalls & misconfigurations

#### Use case-specific pitfalls

* Masking values but breaking API validators (emails, UUIDs, date formats).
* De-identifying large text fields without scoping. It can increase runtime. See [Free text PII obfuscation](/overview/get-started/syntho-bootcamp/5.-generators/free-text-de-identification).
* Tuning only at full scale. Start small, then refine configuration.

<details>

<summary>General pitfalls</summary>

These pitfalls show up in most projects:

* Running full-scale jobs before a small validation run.
* Skipping workspace validation/sync after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).
* Breaking relational integrity (missing PK/FK setup, missing foreign keys, missing virtual foreign keys). Start with [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys) and [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys).
* Leaving sensitive columns on [**Duplicate**](/configure-a-data-generation-job/configure-column-settings/duplicate), or trusting the [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner) without reviewing false positives/negatives.
* Overusing [**Consistent mapping**](/configure-a-data-generation-job/configure-column-settings/consistent-mapping) (it slows down data generation and increases linkability).

</details>

### Governance, compliance, and automation

#### Use case-specific recommendations

* Use a stable seed per environment (`dev`, `test`, `accept`) for reproducible failures.
* Automate refreshes per environment.
* Treat FK/virtual FK changes as breaking changes. Require sync/validation after schema migrations before testing.

<details>

<summary>General recommendations</summary>

Use these recommendations for most workspaces.

#### Ownership and change control

* Assign a single **workspace owner** (data steward / privacy lead / DBA).
* Require a ticket or change request for generator changes.
* Duplicate a workspace before large edits. Keep the previous version as rollback.

#### Access control

* Default to **read-only** access for source connections.
* Restrict **who can view source data** in the UI.
* Use separate workspaces per environment or audience.

#### Automation (baseline)

* Use the [Syntho REST API](/syntho-api/syntho-rest-api) to standardize scans and runs.
* Automate data generation not workspace configuration.
* Keep job logs for failed runs. This reduces back-and-forth during support.

</details>


# Use Case 2: Load & stress testing

Generate large volumes and edge cases for performance testing without sensitive production data.

Use this use case when performance testing needs realistic-looking data at scale. This often includes generating more rows than the original dataset.

### What problem this use case solves

Teams need to validate performance under peak load. They need predictable schemas and realistic distributions.

Classic anonymization can reduce realism and change distributions. It can also keep direct identifiers unless handled carefully.

### When to choose this use case

Pick this when the question is “will it perform at scale?”.

* You need more rows than the source has.
* You need realistic distributions for throughput and latency tests.
* You need repeatable profiles (baseline vs peak vs worst-case).
* You want to inject heavy rows to trigger worst-case behavior.
* Add edge cases with [Calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns).

### When to avoid this use case

Skip this when correctness is more important than scale.

* You need record-level parity, reconciliation, or preserving specific original values. Use [Use Case 4: ETL & Data Pipeline Testing](/overview/get-started/use-cases-and-configuration/use-case-4-etl-and-data-pipeline-testing).
* You need strict multi-table correctness for business logic andtesting. Use [Use Case 1: Application & API Testing](/overview/get-started/use-cases-and-configuration/use-case-1-application-and-api-testing).
* You need a smaller dataset, not a larger. Use [Use Case 10: Data Subsetting](/overview/get-started/use-cases-and-configuration/use-case-10-data-subsetting).

### Recommended Syntho configuration

This setup is optimized for **volume expansion and repeatable load profiles**. You generate more rows than the source. You keep schemas stable so performance results are comparable.

{% stepper %}
{% step %}

#### Prerequisites

* Use the [Prerequisites](/overview/get-started/prerequisites) checklist.

**Checklist**

* [ ] Target scale is defined (rows, GB, or TPS).
* [ ] Benchmark goal is clear (ingest, query, write throughput).
* [ ] PII handling decided before scaling.

{% hint style="warning" %}
Masking does **not** increase row counts. Use AI synthesis for upsampling.
{% endhint %}
{% endstep %}

{% step %}

#### Source & destination management

Create one workspace per performance profile. Examples: `baseline`, `peak-load`. Pick a destination that matches the target platform. Performance issues are platform-specific.

#### Baseline rules

* Keep the **source stable**. Prefer snapshots or back-ups.
* Avoid a **live production** source for iterative work.
* Keep the **destination isolated**. Never write into production.
* Keep **schemas aligned** between source, workspace and destination.
* Use **views** when you need only a subset of the original database.

#### Lifecycle rule of thumb

* Keep the source connection when you expect schema changes.
* Remove the source connection when you expect a new run only much later.
* Revalidate after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).

**Nuances for this use case**

* Don’t mix profiles in one destination. Use distinct schemas or databases per load test type.
* Masking is not upsampling. It does not create new rows.
* Heavy indexing and constraints can impact data generation throughput. Disable or minimize them when you measure app bottlenecks.
  {% endstep %}

{% step %}

#### Configure generators

**Workspace initialization mode**

Choose a [workspace mode](/setup-workspaces/create-a-workspace/workspace-modes). It applies baseline generator suggestions during workspace creation.

Recommended modes for this use case:

* **De-identify** when you mainly need production-like multi-table behavior (joins, constraints) and performance parity.
* **From scratch** when you only load-test a small subset of tables and want to skip generator suggestions.

**AI-generated synthesis**

Use this when you need **more rows** while keeping realistic distributions for a single table. This is the primary method for load and stress on single tables.

**Example (10M event rows):** pick `events` as the entity table, set **Rows to generate** to `10,000,000`, apply **AI synthesize** on non-key columns, and keep key generation on **Generate** so oversampling works.

**Rule-based generation**

Use this when you must inject **worst-case rows** that impact performance (large payloads, high cardinality keys, null-heavy records). Use [Calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns) to control the rate.

**Example (payload spikes):** add `EDGE_FLAG` (≈0.1%), then override `request_payload_size_bytes` to a large range only when `EDGE_FLAG` is true. Validate on 100k rows before scaling.

```excel-formula
// New column: EDGE_FLAG (≈0.1% of rows)
RAND() < 0.001
```

```excel-formula
// Override: request_payload_size_bytes
IF([EDGE_FLAG], RANDBETWEEN(500000, 2000000), [request_payload_size_bytes])
```

**Masking**

Use this when the workload needs **format-valid data** for ingestion validators, or when IDs must match a specific shape (UUID, IBAN, codes). It does not increase row count.

**Example (format-safe ingestion):** mask `session_id` to UUID format and `country_code` to an allowed code list so your ingestion pipeline doesn’t reject rows during stress tests.

**Hybrid**

Use this when you want **AI synthesis for volume**, plus **rule-based extremes** to trigger bottlenecks. This is the “background population + edge injection” pattern from [Example data generation scenarios](/overview/get-started/syntho-bootcamp/example-data-generation-scenarios).

**Example (peak + worst-case):**

1. AI synthesize the entity table to the target row count.
2. Add `EDGE_FLAG` and override 1–2 columns that drive worst-case performance.
3. Keep overrides rare enough to not dominate the benchmark.

Here’s a concrete “hot partition key” pattern that often reveals bottlenecks:

```excel-formula
// Override: partition_key (create skew/hotspot on purpose)
IF([EDGE_FLAG], "HOT_TENANT", [partition_key])
```

**Minimal configuration steps**

1. Pick an entity table/view.
2. Set target row count (start with `100,000`, then scale).
3. Apply AI synthesis to non-key columns and keep key generation on **Generate**.
4. Inject edge cases with calculated columns only when needed.

<details>

<summary>Concrete example: configuring a “peak load” profile</summary>

Example workspace name: `peak-load-v1`.

1. Pick the main workload table (entity table). Example: `events`.
2. In [Table view](/configure-a-data-generation-job/configure-table-settings), set **Rows to generate** for `events` to `10,000,000`.
3. Apply **AI synthesize** on non-key columns. Keep key generation on **Generate** so oversampling is supported.
4. Validate distributions on a small run before scaling (e.g., generate `100,000` rows first).

</details>
{% endstep %}

{% step %}

#### Handle keys and relationships (relational schemas)

If you generate a **single entity table**, you can skip this step.

Decide if the performance test truly needs multi-table joins. AI synthesis has known limits for cross-table consistency.

If you need multiple tables, read [Cross-table relationships limitations](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation/table-relationships). Prefer a single entity table, or switch to de-identification.

If you still generate relational outputs, plan new PKs and stable FKs.

* [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys)
* [Key generators](/configure-a-data-generation-job/configure-column-settings/key-generators)
  {% endstep %}

{% step %}

#### Validate and sync

Validate configuraiton before scaling. If you scale first, you may have correct later.
{% endstep %}

{% step %}

#### Tune generation settings

This use case fails on throughput first. Tune write settings before running at full target size.

Use [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings) and [Large workloads](/overview/get-started/syntho-bootcamp/9.-large-workloads) tuning. Reduce batch size if you hit parameter-limit write errors.
{% endstep %}
{% endstepper %}

### Common pitfalls & misconfigurations

#### Use case-specific pitfalls

* Using masking for upsampling.
* Using AI synthesis for complex multi-table consistency requirements. See [Cross-table relationships limitations](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation/table-relationships).
* Upsampling without validating that rare/edge values are actually present at target rates. Rebalance or inject edge cases using a hybrid approach. See [Example data generation scenarios](/overview/get-started/syntho-bootcamp/example-data-generation-scenarios).
* Running into write failures on big jobs due to batch sizing. If you hit parameter-limit errors, reduce the batch size. See [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings).

<details>

<summary>General pitfalls</summary>

These pitfalls show up in most projects:

* Running full-scale jobs before a small validation run.
* Skipping workspace validation/sync after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).
* Breaking relational integrity (missing PK/FK setup, missing foreign keys, missing virtual foreign keys). Start with [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys) and [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys).
* Leaving sensitive columns on [**Duplicate**](/configure-a-data-generation-job/configure-column-settings/duplicate), or trusting the [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner) without reviewing false positives/negatives.
* Overusing [**Consistent mapping**](/configure-a-data-generation-job/configure-column-settings/consistent-mapping) (it slows down data generation and increases linkability).

</details>

### Governance, compliance, and automation

#### Use case-specific recommendations

* Treat each performance profile as a versioned artifact (`baseline_v1`, `peak_v1`).
* Record benchmark parameters alongside the dataset (row count, batch size, connections, destination platform).
* Automate a run: generate dataset → run benchmark → capture metrics + job logs → publish a short run report.

<details>

<summary>General recommendations</summary>

Use these recommendations for most workspaces.

#### Ownership and change control

* Assign a single **workspace owner** (data steward / privacy lead / DBA).
* Require a ticket or change request for generator changes.
* Duplicate a workspace before large edits. Keep the previous version as rollback.

#### Access control

* Default to **read-only** access for source connections.
* Restrict **who can view source data** in the UI.
* Use separate workspaces per environment or audience.

#### Automation (baseline)

* Use the [Syntho REST API](/syntho-api/syntho-rest-api) to standardize scans and runs.
* Automate data generation not workspace configuration.
* Keep job logs for failed runs. This reduces back-and-forth during support.

</details>


# Use Case 3: Demo data

Realistic demo data that contains no real identifiers and can be shared safely.

Use this use case when you need realistic demo data without sensitive information. The focus is stable, shareable demo scenarios.

### What problem this use case solves

Sales and pre-sales teams need high-quality datasets to demonstrate workflows. Data must be safe to share in demo environments.

Classic anonymization can degrade the demo experience. It also requires careful handling of indirect identifiers.

### When to choose this use case

Pick this when you need shareable demo data that still “feels real”.

If you’re unsure, start with **Mock or mask all** and only enable [Consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping) for a small set of key entities.

* You demo workflows and UI journeys with realistic values.
* You need zero real identifiers in the demo environment.
* You need stable “hero customers” across tables and refreshes.
* You need realistic formats for validators and UI flows.
* Use [Mock](/configure-a-data-generation-job/configure-column-settings/mockers) for narrative fields.
* Use [Mask](/configure-a-data-generation-job/configure-column-settings/mask) for format-critical fields.

### When to avoid this use case

Skip this when demos are not the goal.

* You need external sharing with documented privacy evidence and low linkability. Use [Use Case 9: Data Sharing & Monetization](/overview/get-started/use-cases-and-configuration/use-case-9-data-sharing-and-monetization).
* You need DTAP-style test data for apps and APIs (not demo narratives). Use [Use Case 1: Application & API Testing](/overview/get-started/use-cases-and-configuration/use-case-1-application-and-api-testing).
* You need upsampling or load generation at scale. Use [Use Case 2: Load & Stress](/overview/get-started/use-cases-and-configuration/use-case-2-load-and-stress-testing).

### Recommended Syntho configuration

This setup is optimized for **stable demo narratives**. You want repeatable “storytelling” values. You want zero real identifiers, even indirectly.

{% stepper %}
{% step %}

#### Prerequisites

* Use the [Prerequisites](/overview/get-started/prerequisites) checklist.

**Checklist**

* [ ] Demo journeys listed (happy path + 1–2 alternates).
* [ ] “Hero entities” defined (small, intentional set).
* [ ] Destination is isolated and easy to reset.

{% hint style="warning" %}
Treat demo data as shareable by default. Avoid any real identifiers.
{% endhint %}
{% endstep %}

{% step %}

#### Source & destination management

Create one workspace per demo dataset or audience. Examples: `sales-basic`, `partner-demo`, `industry-demo`.

* Duplicate a working workspace before big changes. This gives you a rollback point.
* Use simple versioned names like `v1`, `v2`, `baseline`, or `pilot-partner-x`.

#### Baseline rules

* Keep the **source stable**. Prefer snapshots or back-ups.
* Avoid a **live production** source for iterative work.
* Keep the **destination isolated**. Never write into production.
* Keep **schemas aligned** between source, workspace and destination.
* Use **views** when you need only a subset of the original database.

#### Lifecycle rule of thumb

* Keep the source connection when you expect schema changes.
* Remove the source connection when you expect a new run only much later.
* Revalidate after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).

**Nuances for this use case**

* Use a stable seed when you can. It makes storytelling predictable.
* Treat demos as shareable by default.
* Keep “hero entities” stable intentionally. Use consistent mapping sparingly.
* Don’t generate into a schema that already contains old demo data. You may get inconsistent data across sessions.
  {% endstep %}

{% step %}

#### Configure generators

**Workspace initialization mode**

Choose a [workspace mode](/setup-workspaces/create-a-workspace/workspace-modes). It applies baseline generator suggestions during workspace creation.

Recommended modes for this use case:

* **Mock or mask all** when you want “looks real” data with zero identifiers.
* **Mock all** when you have little to no source data and you’re building a demo dataset from scratch.
* **De-identify** when you start from a production-like schema and want to keep non-PII columns mostly intact.

**AI-generated synthesis**

Use this when your demo must feel **statistically realistic** (distributions, correlations). It’s useful when you have an single entity table or demo view.

**Example (realistic churn patterns):** create a `demo_customer_features_view` (tenure, orders, segment), then use **AI synthesize** so churn-related features look realistic while still being unlinkable.

**Rule-based generation**

Use this when a demo needs **guaranteed scenarios**. Use [Calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns) to enforce “must exist” states.

**Example (always-have VIP customers):** create a tier that drives demo flows (pricing, permissions, entitlements).

```excel-formula
// New column: demo_tier (simple, stable ratios)
SWITCH(TRUE,
  RAND() < 0.05, "VIP",
  RAND() < 0.25, "PREMIUM",
  "STANDARD"
)
```

**Masking**

Use this when UI flows require **valid formats** (email, phone, IBAN) and you want stable data across tables.

**Example (stable hero customer):** mask `email` and `phone_number` to valid formats, then enable **Consistent mapping** only for `customer_name` and `company_name` so the same “ACME” appears across orders and invoices.

**Hybrid**

Use this when you need both **narrative control** and **realistic variation**.

It maps well to the “deterministic relations” and “absolute calculations” patterns in [Example data generation scenarios](/overview/get-started/syntho-bootcamp/example-data-generation-scenarios).

**Example (order totals and margin always consistent):** if your demo shows “margin” or “savings”, make the math hold.

1. AI synthesize or mock `order_total` and `fulfillment_cost`.
2. Use a calculated column for `gross_margin` so it’s always correct.

```excel-formula
// New column: gross_margin (absolute calculation)
[order_total] - [fulfillment_cost]
```

**Minimal configuration steps**

1. Run a [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner).
2. Use **Mock** for narrative fields. Use **Mask** for validator formats.
3. Enable consistent mapping only for the direct identifiers.

{% hint style="info" %}
Use consistent mapping sparingly. It helps storytelling, but increases linkability.
{% endhint %}

<details>

<summary>Concrete example: a “customer journey” demo narrative</summary>

Goal: a dataset that always has a few customers in each lifecycle stage so demo flows never get stuck.

Example workspace name: `sales-basic`.

Example “story columns” you can enforce with calculated columns:

```excel-formula
// New column: journey_stage
IFS(
  [last_order_date] >= DATEADD(TODAY(), -30, "day"),  "ACTIVE",
  [last_order_date] >= DATEADD(TODAY(), -180, "day"), "AT_RISK",
  TRUE,                                               "CHURNED"
)
```

```excel-formula
// New column: is_premium (ensure some premium customers exist)
IFS(
  [journey_stage] = "ACTIVE",  RAND() < 0.20,
  [journey_stage] = "AT_RISK", RAND() < 0.10,
  TRUE,                        RAND() < 0.05
)
```

Tip: only apply consistent mapping to the “hero entities” you showcase (top accounts, key products). This keeps the story stable while limiting linkability.

</details>
{% endstep %}

{% step %}

#### Handle keys and relationships (relational schemas)

If your demo is **single-table** (or you share one flattened table), you can skip this step.

Demo flows break on missing relationships. Validate FKs before polishing the data.

Use [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys) and add [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys) if the source schema is incomplete.

* [Key generators](/configure-a-data-generation-job/configure-column-settings/key-generators)
  {% endstep %}

{% step %}

#### Validate and sync

Run the demo journeys against the generated dataset. Fix the exact tables and columns that break flows.

Resync whenever the product schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace). Demos go stale fast without validation.

* [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace)
  {% endstep %}

{% step %}

#### Tune generation settings

Prioritize fast reset times. Demo environments get rebuilt often.

Use [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings) once the storytelling rules are stable.

* [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings)
  {% endstep %}
  {% endstepper %}

### Common pitfalls & misconfigurations

#### Use case-specific pitfalls

* Treating demo datasets as “internal-only” and skipping privacy review steps.
* Applying consistent mapping broadly instead of only on direct identifiers

<details>

<summary>General pitfalls</summary>

These pitfalls show up in most projects:

* Running full-scale jobs before a small validation run.
* Skipping workspace validation/sync after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).
* Breaking relational integrity (missing PK/FK setup, missing foreign keys, missing virtual foreign keys). Start with [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys) and [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys).
* Leaving sensitive columns on [**Duplicate**](/configure-a-data-generation-job/configure-column-settings/duplicate), or trusting the [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner) without reviewing false positives/negatives.
* Overusing [**Consistent mapping**](/configure-a-data-generation-job/configure-column-settings/consistent-mapping) (it slows down data generation and increases linkability).

</details>

### Governance, compliance, and automation

#### Use case-specific recommendations

* Maintain a list of approved tables/fields. Don’t add columns ad-hoc right before a demo.
* Separate internal sales demos from partner demos. Use separate workspaces and destinations to prevent accidental sharing.
* Automate a pre-demo reset: regenerate → run 2–3 scripted demo journeys → confirm no PII columns remain on **Duplicate**.

<details>

<summary>General recommendations</summary>

Use these recommendations for most workspaces.

#### Ownership and change control

* Assign a single **workspace owner** (data steward / privacy lead / DBA).
* Require a ticket or change request for generator changes.
* Duplicate a workspace before large edits. Keep the previous version as rollback.

#### Access control

* Default to **read-only** access for source connections.
* Restrict **who can view source data** in the UI.
* Use separate workspaces per environment or audience.

#### Automation (baseline)

* Use the [Syntho REST API](/syntho-api/syntho-rest-api) to standardize scans and runs.
* Automate data generation not workspace configuration.
* Keep job logs for failed runs. This reduces back-and-forth during support.

</details>


# Use Case 4: ETL & data pipeline testing

End-to-end pipeline testing without using production data.

Use this use case when you need an end-to-end test dataset for pipelines. Focus on correctness across tables and across systems.

### What problem this use case solves

Teams need to validate transformations and integrations. They need stable keys, relationships, and formats.

Classic anonymization can over-generalize values and reduce realism. In relational databases, you also need explicit handling of keys and foreign keys to keep referential integrity.

### When to choose this use case

Pick this when you test data pipelines, not just tables.

If you’re unsure, start with **de-identify** and enable [Consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping) for join keys. Then run [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).

* You validate ETL/ELT transformations end-to-end.
* Your tests rely on stable joins and key coverage.
* Your pipeline runs on a cadence (CI, nightly, releases).
* Downstream systems validate formats and constraints.
* Make joins explicit with [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys).

### When to avoid this use case

Skip this when data pipelines and ETL testing are not the target.

* You only test one curated table with no joins. Use [Use Case 1: Application & API Testing](/overview/get-started/use-cases-and-configuration/use-case-1-application-and-api-testing).
* You mainly need performance and volume testing. Use [Use Case 2: Load & Stress](/overview/get-started/use-cases-and-configuration/use-case-2-load-and-stress-testing).
* You validate workflows during a migration between platforms. Use [Use Case 8: Cloud & Data Migration](/overview/get-started/use-cases-and-configuration/use-case-8-cloud-and-data-migration).

### Recommended Syntho configuration

This setup is optimized for **join-correct, repeatable pipeline runs**. Your goal is functional correctness across transforms. Broken keys are a test failure.

{% stepper %}
{% step %}

#### Prerequisites

**Checklist**

* [ ] Pipeline checkpoints listed (counts, joins, null rules).
* [ ] Source snapshot fixed for the run.
* [ ] Key columns identified (business keys + join keys).
* [ ] PII handling decided at the right stage (ingest vs curated).

{% hint style="info" %}
Aim for repeatability first. Optimize later.
{% endhint %}

* Use the [Prerequisites](/overview/get-started/prerequisites) checklist.
  {% endstep %}

{% step %}

#### Source & destination management

Create one workspace per pipeline (or per environment). This keeps generator changes traceable to pipeline changes.

#### Baseline rules

* Keep the **source stable**. Prefer snapshots or back-ups.
* Avoid a **live production** source for iterative work.
* Keep the **destination isolated**. Never write into production.
* Keep **schemas aligned** between source, workspace and destination.
* Use **views** when you need only a subset of the original database.

#### Lifecycle rule of thumb

* Keep the source connection when you expect schema changes.
* Remove the source connection when you expect a new run only much later.
* Revalidate after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).

**Nuances for this use case**

* If you validate row-level equality, freeze the source snapshot. Don’t regenerate from moving extracts.
* If you change a view definition, resync before blaming the pipeline. Otherwise tests fail for the wrong reason.
* Don’t write test inputs into schemas used by prod-like data. Keep a dedicated namespace for pipeline test inputs.
* [Create a workspace](/setup-workspaces/create-a-workspace)
  {% endstep %}

{% step %}

#### Configure generators

**Workspace initialization mode**

Choose a [workspace mode](/setup-workspaces/create-a-workspace/workspace-modes). It applies baseline generator suggestions during workspace creation.

Recommended modes for this use case:

* **De-identify** when you want stable row-level behavior and predictable outputs for assertions.
* **Mock or mask all** when you want to remove more than just PII but still keep formats stable.
* **From scratch** when you only test a small number of pipeline-relevant tables.

**AI-generated synthesis**

Usually not the default for pipeline testing. It can change row-level behavior, which makes deterministic assertions harder.

**Example (non-assertive smoke runs):** synthesize a single `staging_events_view` to generate a larger, privacy-safe stream and validate pipeline robustness (parsing, scaling). Avoid using it for exact record-level checks.

**Rule-based generation**

Use this for **known-good** and **known-bad** rows. Use [Calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns) to inject boundary cases deterministically.

**Example (type-conversion boundary):** add a calculated override that sets `amount = -1` for `BAD_ROW_FLAG` rows, and `amount = 0` for another slice. Assert the pipeline rejects or routes them correctly.

```excel-formula
// New column: BAD_ROW_FLAG (≈0.5% of rows)
RAND() < 0.005
```

```excel-formula
// Override: amount (negative values trigger validation logic)
IF([BAD_ROW_FLAG], -ABS([amount]), [amount])
```

**Masking**

Use this when you need **format-preserving replacements** while keeping stable joins across stages and systems.

**Example (stable business keys):** enable **Consistent mapping** for `customer_id` and `order_id`, then mask `email` and `iban` so joins remain stable from ingest → curated, without leaking PII.

**Hybrid**

Use this when you need stable joins (mask/de-identify) plus **rule-driven guarantees** for transformation correctness. It maps to “absolute calculations” from [Example data generation scenarios](/overview/get-started/syntho-bootcamp/example-data-generation-scenarios).

**Example (pipeline invariant always holds):** if your pipeline derives `net_amount = gross_amount - tax_amount`, enforce that invariant in the input so you can test transformation drift.

```excel-formula
// Override: net_amount (guaranteed identity)
[gross_amount] - [tax_amount]
```

**Minimal configuration steps**

1. Run a [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner) on the pipeline inputs.
2. Set join keys to stable handling (key generators + consistent mapping where needed).
3. Add calculated-column assertions for the transformations you care about.
4. Validate on a small slice before scaling.

{% hint style="warning" %}
If joins break, you are no longer testing the pipeline. Fix PK/FK first.
{% endhint %}

* [Automatic PII discovery with PII scanner](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner)
* [Manage personally identifiable information (PII)](/configure-a-data-generation-job/manage-personally-identifiable-information-pii)

<details>

<summary>Concrete example: creating “known bad” rows for pipeline assertions</summary>

Use a dedicated workspace when you need intentional “bad data”. Example: `etl-negative-cases`.

If [Calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns) are available in your version, inject a small percentage of erroneous rows:

```excel-formula
// New column: BAD_ROW_FLAG (≈0.5% of rows)
RAND() < 0.005
```

```excel-formula
// Example override: postal_code (invalid format to test parsing)
IF([BAD_ROW_FLAG], "XX-INVALID", [postal_code])
```

```excel-formula
// Example override: amount (negative to test validation rules)
IF([BAD_ROW_FLAG], -ABS([amount]), [amount])
```

</details>
{% endstep %}

{% step %}

#### Handle keys and relationships (relational schemas)

If your pipeline test is based on a **single curated table** (no joins), you can skip this step.

Make foreign keys explicit before the first full run. Otherwise you test the wrong join behavior.

Use [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys). Add [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys) where the source schema is incomplete.

* [Key generators](/configure-a-data-generation-job/configure-column-settings/key-generators)

If you don’t have FKs in the database, start with the [foreign key scanner](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/use-foreign-key-scanner). Then validate key coverage on real joins.
{% endstep %}

{% step %}

#### Validate and sync

Validate early on a subset. Confirm row counts and join cardinalities at key stages.

Re-run validation whenever schemas change. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace). Pipeline tests depend on schema stability.

* [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace)

Also validate *intermediate outputs* in the pipeline:

* Snapshot row counts per stage (ingest → staging → curated).
* Compare distinct counts of business keys after dedup steps.
* Assert null-rate expectations for critical columns.
  {% endstep %}

{% step %}

#### Tune generation settings

Tune for repeatable runtime and stable write behavior. Pipeline tests often run in CI/CD.

Use [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings) after the join graph is correct.

* [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings)
  {% endstep %}
  {% endstepper %}

### Common pitfalls & misconfigurations

#### Use case-specific pitfalls

* Testing transformations without including representative edge-case inputs.

<details>

<summary>General pitfalls</summary>

These pitfalls show up in most projects:

* Running full-scale jobs before a small validation run.
* Skipping workspace validation/sync after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).
* Breaking relational integrity (missing PK/FK setup, missing foreign keys, missing virtual foreign keys). Start with [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys) and [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys).
* Leaving sensitive columns on [**Duplicate**](/configure-a-data-generation-job/configure-column-settings/duplicate), or trusting the [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner) without reviewing false positives/negatives.
* Overusing [**Consistent mapping**](/configure-a-data-generation-job/configure-column-settings/consistent-mapping) (it slows down data generation and increases linkability).

</details>

### Governance, compliance, and automation

#### Use case-specific recommendations

* Align workspace versions with pipeline versions (example: `etl_orders_v3`). Don’t reuse a workspace across major pipeline rewrites.
* Automate pipeline checks against the generated dataset (row counts, PK uniqueness, null-rate expectations, join cardinalities).
* Keep join-key strategy explicit and reviewed (which keys are duplicated/hashed/generated). Document it with the pipeline test plan.
* If you inject “known bad rows”, keep the flags and percentages stable across runs. Otherwise tests become flaky.

<details>

<summary>General recommendations</summary>

Use these recommendations for most workspaces.

#### Ownership and change control

* Assign a single **workspace owner** (data steward / privacy lead / DBA).
* Require a ticket or change request for generator changes.
* Duplicate a workspace before large edits. Keep the previous version as rollback.

#### Access control

* Default to **read-only** access for source connections.
* Restrict **who can view source data** in the UI.
* Use separate workspaces per environment or audience.

#### Automation (baseline)

* Use the [Syntho REST API](/syntho-api/syntho-rest-api) to standardize scans and runs.
* Automate data generation not workspace configuration.
* Keep job logs for failed runs. This reduces back-and-forth during support.

</details>


# Use Case 5: Feature development

Shift-left testing with realistic synthetic data when production data is unavailable or restricted.

Use this use case when you need data to build and test features early. This can include scenarios with no or limited input data.

### What problem this use case solves

Teams need realistic data to build features. Often, production data access is limited or not available yet.

Classic anonymization requires access to production-like data. It may also be slow when requirements change frequently.

### When to choose this use case

Pick this when you need data early, before production access exists.

If you’re unsure, start with **Mock all**, configure PK/FK with [key generators](/configure-a-data-generation-job/configure-column-settings/key-generators), and add rules later.

* You have no or limited production-like data.
* You need “new-shaped” data for new features and enums.
* You iterate fast and configs change often.
* You need dev data that is safe by default.
* Use **From scratch** when only a few tables matter and you want tight control.

### When to avoid this use case

Skip this when you need production parity.

* You must preserve original relationships for reconciliation. Use [Use Case 4: ETL & Data Pipeline Testing](/overview/get-started/use-cases-and-configuration/use-case-4-etl-and-data-pipeline-testing).
* You need production-parity behavior across many related tables. Use [Use Case 1: Application & API Testing](/overview/get-started/use-cases-and-configuration/use-case-1-application-and-api-testing).
* You mainly need performance testing at huge volumes. Use [Use Case 2: Load & Stress](/overview/get-started/use-cases-and-configuration/use-case-2-load-and-stress-testing).
* If the dataset will leave your team, treat it like a controlled release. Avoid stable pseudonyms by default.

### Recommended Syntho configuration

This setup is optimized for **fast iteration when production data is missing**. You generate values quickly. You encode rules only where the feature depends on them.

{% stepper %}
{% step %}

#### Prerequisites

**Checklist**

* [ ] Schema is stable enough for this iteration.
* [ ] “Must-have” rules listed (enums, ranges, required relationships).
* [ ] PK/FK strategy decided (key generators + foreign keys).

- Use the [Prerequisites](/overview/get-started/prerequisites) checklist.
  {% endstep %}

{% step %}

#### Source & destination management

Create one workspace per team or feature area. This prevents generator churn across teams.

* Duplicate a working workspace before big changes. This gives you a rollback point.
* Use simple versioned names like `v1`, `v2`, `baseline`, or `pilot-partner-x`.

#### Baseline rules

* Keep the **source stable**. Prefer snapshots or back-ups.
* Avoid a **live production** source for iterative work.
* Keep the **destination isolated**. Never write into production.
* Keep **schemas aligned** between source, workspace and destination.
* Use **views** when you need only a subset of the original database.

#### Lifecycle rule of thumb

* Keep the source connection when you expect schema changes.
* Remove the source connection when you expect a new run only much later.
* Revalidate after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).

**Nuances for this use case**

* “Mock-first” is common. Use no source data, or only small reference seeds.
* Treat dev destinations as disposable. Reset often. Don’t patch generated data by hand.
* Don’t share one workspace across teams. Generator edits will churn and break others’ flows.
* “Mock all” won’t preserve relational behavior. Keys and relationships still need explicit setup.
* [Create a workspace](/setup-workspaces/create-a-workspace)
* [Duplicate a workspace](/setup-workspaces/duplicate-a-workspace)
  {% endstep %}

{% step %}

#### Configure generators

**Workspace initialization mode**

Choose a [workspace mode](/setup-workspaces/create-a-workspace/workspace-modes). It applies baseline generator suggestions during workspace creation.

Recommended modes for this use case:

* **Mock all** when there is little/no reference data and you’re generating new values.
* **From scratch** when only a few tables matter and you want tight control.
* **Mock or mask all** when you have a baseline dataset but want to replace most fields quickly.

**AI-generated synthesis**

Use this when you have enough rows and want realistic distributions quickly, especially for analytics-like features used during development.

**Example (realistic transaction amounts):** if you have a small seed table `payments_seed` with representative columns, apply **AI synthesize** to generate realistic `amount`, `currency`, and `merchant_category` values for UI and backend feature testing.

**Rule-based generation**

Use this when new features require **explicit business rules** or new enums that don’t exist in any source yet. Use [Calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns) for maintainable logic.

**Example (new status enum):** add a calculated `review_status` column that produces `PENDING_REVIEW`, `APPROVED`, `REJECTED` at fixed ratios. Use it to test UI filters and backend branching.

```excel-formula
// New column: review_status (weighted distribution)
SWITCH(TRUE,
  RAND() < 0.10, "PENDING_REVIEW",
  RAND() < 0.85, "APPROVED",
  "REJECTED"
)
```

**Masking**

Use this when dev environments require **format-valid identifiers** (UUIDs, emails) and you want to keep relational joins stable.

**Example (format + joins):** mask `user_email` and `external_reference` to valid formats, and keep consistent mapping on `account_number` so a user’s account references remain stable across tables during repeated dev resets.

**Hybrid**

Use this when you need mock-first speed, plus a few **hard business rules**. It maps to “new data creation” in [Example data generation scenarios](/overview/get-started/syntho-bootcamp/example-data-generation-scenarios).

**Example (tenant-style dev data with realistic emails):**

1. Mock names and basic attributes.
2. Use a calculated column to build an email from those generated values.

```excel-formula
// New column: user_email (derived from generated names)
LOWER(CONCATENATE([first_name], ".", [last_name], "@", MOCK_FREE_EMAIL_DOMAIN_0))
```

**Minimal configuration steps**

1. Apply mockers for most columns.
2. Configure PK/FK via [key generators](/configure-a-data-generation-job/configure-column-settings/key-generators) and [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys).
3. Add calculated columns only for the behaviors your feature relies on.

<details>

<summary>When you need “new-shaped” data</summary>

Use explicit generators when the source cannot contain the new behavior:

* New enums/statuses (`PENDING_REVIEW`, `ESCALATED`, `CANCELLED_BY_USER`).
* JSON payload shapes.
* Free-text fields that may contain identifiers.

Relevant docs:

* [JSON de-identification](/configure-a-data-generation-job/configure-column-settings/json-de-identification)
* [Free text PII obfuscation](/overview/get-started/syntho-bootcamp/5.-generators/free-text-de-identification)

</details>

<details>

<summary>Practical example: versioning generator configurations</summary>

Feature work is experimental. Treat generator configs like versions.

Recommended pattern:

* Keep a stable baseline workspace: `feature-payments_v1`.
* Duplicate before large edits: `feature-payments_v2`.
* Roll back by reusing the previous workspace if tests fail.

This is also how you can compare two approaches (e.g., “more rules” vs “more mock”) without losing a known-good setup.

</details>
{% endstep %}

{% step %}

#### Handle keys and relationships (relational schemas)

If you only test a **single table** (no joins), you can skip this step.

Mockers can’t generate PK/FK columns. You must configure keys explicitly or your app won’t work.

Use [key generators](/configure-a-data-generation-job/configure-column-settings/key-generators) for primary keys and enforce relationships via [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys).

* [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys)
* [Key generators](/configure-a-data-generation-job/configure-column-settings/key-generators)
  {% endstep %}

{% step %}

#### Validate and sync

Validate after every schema change via [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace). Feature development is constant schema drift.

Validate before you blame failing tests on the feature.

* [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace)
  {% endstep %}

{% step %}

#### Tune generation settings

Optimize for small, frequent runs. This matches a dev workflow and keeps feedback fast.

Use [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings) when job time becomes a bottleneck.

* [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings)
  {% endstep %}
  {% endstepper %}

### Common pitfalls & misconfigurations

#### Use case-specific pitfalls

* Expecting “Mock all” to preserve original distributions or relationships.
* Forgetting that mockers cannot be applied to PK/FK columns.
  * Use [Key generators](/configure-a-data-generation-job/configure-column-settings/key-generators).

<details>

<summary>General pitfalls</summary>

These pitfalls show up in most projects:

* Running full-scale jobs before a small validation run.
* Skipping workspace validation/sync after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).
* Breaking relational integrity (missing PK/FK setup, missing foreign keys, missing virtual foreign keys). Start with [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys) and [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys).
* Leaving sensitive columns on [**Duplicate**](/configure-a-data-generation-job/configure-column-settings/duplicate), or trusting the [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner) without reviewing false positives/negatives.
* Overusing [**Consistent mapping**](/configure-a-data-generation-job/configure-column-settings/consistent-mapping) (it slows down data generation and increases linkability).

</details>

### Governance, compliance, and automation

#### Use case-specific recommendations

* Treat feature work data as disposable. Create workspaces per feature area or team. Delete/retire when work merges.
* When you introduce new enums/statuses, document the allowed values and generation ratios. This prevents UI/test drift.
* Enforce a hard rule: no production source connections for early feature work. Use mock-first or approved seeds only.
* Automate “fast feedback” runs: small generate → validate → scale only when the feature needs it.

<details>

<summary>General recommendations</summary>

Use these recommendations for most workspaces.

#### Ownership and change control

* Assign a single **workspace owner** (data steward / privacy lead / DBA).
* Require a ticket or change request for generator changes.
* Duplicate a workspace before large edits. Keep the previous version as rollback.

#### Access control

* Default to **read-only** access for source connections.
* Restrict **who can view source data** in the UI.
* Use separate workspaces per environment or audience.

#### Automation (baseline)

* Use the [Syntho REST API](/syntho-api/syntho-rest-api) to standardize scans and runs.
* Automate data generation not workspace configuration.
* Keep job logs for failed runs. This reduces back-and-forth during support.

</details>


# Use Case 6: ML model development

Generate feature datasets when real data is scarce or sensitive.

Use this use case when you need synthetic feature datasets for ML development.

### What problem this use case solves

Teams need datasets for model development and validation. Data may be scarce, sensitive, or slow to access.

Classic anonymization can reduce the statistical utility needed for ML. It can also keep indirect signals that are still privacy-sensitive.

### When to choose this use case

Pick this when you build ML models and need statistical utility.

If you’re unsure, start with **Synthesize all** on a single training table (entity table or view) and run the [QA report](broken://pages/tYZauKu0GdWCgMEjSKzN) before training.

* You need synthetic feature datasets for training and evaluation.
* You want new rows without 1:1 links to real people.
* You can train on an entity table or training view.
* You want privacy-safe iteration without production access.

### When to avoid this use case

Skip this when you need strict correctness or reversibility.

* You need deterministic, join-correct multi-table datasets for reconciliation or regression assertions. Use [Use Case 4: ETL & Data Pipeline Testing](/overview/get-started/use-cases-and-configuration/use-case-4-etl-and-data-pipeline-testing).
* You need 100% constraint adherence or stable pseudonyms that can be traced across tables. Use [Use Case 1: Application & API Testing](/overview/get-started/use-cases-and-configuration/use-case-1-application-and-api-testing).
* You need an analyst sandbox for exploration and BI. Use [Use Case 7: Analytics Sandboxes](/overview/get-started/use-cases-and-configuration/use-case-7-analytics-sandboxes).
* You need dev data for feature work, not modeling utility. Prefer mock-first generation and a small scope.

### Recommended Syntho configuration

This setup is optimized for **model development utility with strong privacy**. You generate new rows. You avoid any 1:1 link to original records.

{% stepper %}
{% step %}

#### Prerequisites

**Checklist**

* [ ] Training table/view is defined (entity table).
* [ ] Inputs vs targets vs leakage columns are decided.
* [ ] Direct identifiers removed from features.

{% hint style="warning" %}
Avoid leakage. Exclude post-outcome timestamps and human decisions from features.
{% endhint %}

* Use the [Prerequisites](/overview/get-started/prerequisites) checklist.
* Follow [AI synthesis: Data pre-processing](/overview/get-started/syntho-bootcamp/10.-ai-synthesis-data-pre-processing-when-using) when the source is not an entity table yet.
  {% endstep %}

{% step %}

#### Source & destination management

Create one workspace per feature dataset or model track. This keeps training and evaluation reproducible.

Use separate workspaces for different privacy settings. Privacy settings are part of your model governance.

#### Baseline rules

* Keep the **source stable**. Prefer snapshots or back-ups.
* Avoid a **live production** source for iterative work.
* Keep the **destination isolated**. Never write into production.
* Keep **schemas aligned** between source, workspace and destination.
* Use **views** when you need only a subset of the original database.

#### Lifecycle rule of thumb

* Keep the source connection when you expect schema changes.
* Remove the source connection when you expect a new run only much later.
* Revalidate after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).

**Nuances for this use case**

* Use a view to reduce leakage risk. Keep post-outcome fields out of the training cut.
* Prefer clean dataset versioning (`features_v1`, `features_v2`). Avoid a shared “analytics” schema that loses lineage.
* Don’t default to de-identification. For ML, you often need stronger unlinkability than de-identification provides.
* [Create a workspace](/setup-workspaces/create-a-workspace)
  {% endstep %}

{% step %}

#### Configure generators

**Workspace initialization mode**

Choose a [workspace mode](/setup-workspaces/create-a-workspace/workspace-modes). It applies baseline generator suggestions during workspace creation.

Recommended modes for this use case:

* **Synthesize all** for model development datasets (best default when you have an entity table).
* **De-identify** only when you must preserve multi-table behavior and don’t need maximum unlinkability.
* **From scratch** when you’re curating a very specific feature table and want manual control.

**AI-generated synthesis**

This is the primary method here. Use it when you need **statistical utility** and **strong privacy** without 1:1 record links.

**Example (training table via view):** build `training_entity_view` (features + label), then apply **AI synthesize** to generate a training dataset for modeling. Run the [QA report](broken://pages/tYZauKu0GdWCgMEjSKzN) before training.

**Rule-based generation**

Use this to enforce **feature constraints**, **bucketing**, or **label logic** that must be explicit (or to remove leakage). Use [Calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns) for transparent, auditable rules.

**Example (leakage-safe bucketing):** create a calculated `age_band` (`0–17`, `18–34`, `35–54`, `55+`) and drop raw `date_of_birth`. Train on the band to reduce leakage and privacy risk.

```excel-formula
// New column: age_band (train on band, then exclude raw date_of_birth)
IFS(
  YEAR(TODAY()) - YEAR([date_of_birth]) < 18,  "0-17",
  YEAR(TODAY()) - YEAR([date_of_birth]) < 35,  "18-34",
  YEAR(TODAY()) - YEAR([date_of_birth]) < 55,  "35-54",
  TRUE,        "55+"
)
```

**Masking**

Use this only for columns that must stay **format-valid** for downstream tooling. Avoid masking identifiers for ML unless strictly required.

**Example (pipeline contract):** if your training pipeline validates an `email` format, apply **Mask → Email** but exclude the column from the model features. Keep it as a non-training field for compatibility only.

**Hybrid**

Use this when you want AI synthesis for utility, plus explicit rules for stability and governance.

**Example (utility + business rules):** AI synthesize core features, then add a deterministic segmentation flag (matches “absolute calculations” style of thinking).

```excel-formula
// New column: is_high_value (business rule segment)
IF([spend_90d] >= 1000, TRUE, FALSE)
```

**Minimal configuration steps**

1. Build `training_entity_view` (one row per entity).
2. Apply **AI synthesize** and validate with the [QA report](broken://pages/tYZauKu0GdWCgMEjSKzN).
3. Add calculated columns for bucketing or governance flags only.

* [Automatic PII discovery with PII scanner](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner)
* [Manage personally identifiable information (PII)](/configure-a-data-generation-job/manage-personally-identifiable-information-pii)

<details>

<summary>Optional: feature engineering</summary>

* Prefer engineered features over raw identifiers and raw notes.
* Drop or recompute derived columns to avoid leakage.
* If you keep raw text, use [Free text PII obfuscation](/overview/get-started/syntho-bootcamp/5.-generators/free-text-de-identification) and scope it tightly.

If your real data is relational, create a training view first. See [Use SQL views as input tables](/setup-workspaces/create-a-workspace/use-sql-views-as-input-tables).

</details>
{% endstep %}

{% step %}

#### Handle keys and relationships (relational schemas)

This use case typically trains on a **single entity table**. If you already have that table (or a view), you can skip PK/FK configuration.

If your source is relational, decide what becomes the entity table.

Use [Cross-table relationships limitations](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation/table-relationships) to decide whether to reshape to a single entity table or use de-identification for relationship-heavy schemas.
{% endstep %}

{% step %}

#### Validate and sync

Run the [QA report](broken://pages/tYZauKu0GdWCgMEjSKzN) when available. Use it to sanity-check utility and privacy before training.

If you update the schema or feature set, revalidate. Small schema changes can invalidate a model comparison.
{% endstep %}

{% step %}

#### Tune generation settings

Tune for training stability. Prefer fewer reruns with stable outputs over maximum speed.

Apply [Additional privacy controls](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation/privacy-controls) before publishing datasets outside the model team.
{% endstep %}
{% endstepper %}

### Common pitfalls & misconfigurations

#### Use-case specific pitfalls

* Starting AI synthesis without an entity-table style dataset.
* Expecting AI synthesis to preserve cross-system consistency across multiple systems.
* Treating QA results as optional when the output is used for model validation.
* Training on redundant or derived columns (e.g. totals derived from components).
  * Remove derived columns first. See [AI synthesize](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation).

<details>

<summary>General pitfalls</summary>

These pitfalls show up in most projects:

* Running full-scale jobs before a small validation run.
* Skipping workspace validation/sync after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).
* Breaking relational integrity (missing PK/FK setup, missing foreign keys, missing virtual foreign keys). Start with [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys) and [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys).
* Leaving sensitive columns on [**Duplicate**](/configure-a-data-generation-job/configure-column-settings/duplicate), or trusting the [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner) without reviewing false positives/negatives.
* Overusing [**Consistent mapping**](/configure-a-data-generation-job/configure-column-settings/consistent-mapping) (it slows down data generation and increases linkability).

</details>

### Governance, compliance, and automation

#### Use-case specific recommendations

* Version datasets like model inputs (`features_v1`, `features_v2`). Store generation settings + QA report with the experiment.
* Separate training vs evaluation datasets. Don’t generate both from the same workspace settings without intent.
* Gate model training on a QA review (utility + privacy sanity check). Capture acceptance criteria in the ticket.
* If outputs leave the ML team, require an explicit privacy review and apply additional privacy controls before distribution.

<details>

<summary>General recommendations</summary>

Use these recommendations for most workspaces.

#### Ownership and change control

* Assign a single **workspace owner** (data steward / privacy lead / DBA).
* Require a ticket or change request for generator changes.
* Duplicate a workspace before large edits. Keep the previous version as rollback.

#### Access control

* Default to **read-only** access for source connections.
* Restrict **who can view source data** in the UI.
* Use separate workspaces per environment or audience.

#### Automation (baseline)

* Use the [Syntho REST API](/syntho-api/syntho-rest-api) to standardize scans and runs.
* Automate data generation not workspace configuration.
* Keep job logs for failed runs. This reduces back-and-forth during support.

</details>


# Use Case 7: Analytics sandboxes

Secure sandboxes for exploratory analytics and data science.

Use this use case when analysts need access to data while keeping privacy risk controlled. The focus is utility for exploration with privacy-by-design controls.

### What problem this use case solves

Teams need safe environments for exploration. They need distributions and correlations that remain useful.

Classic anonymization can remove detail and distort distributions. It can also require multiple iterations to meet privacy needs.

### When to choose this use case

Pick this when analysts need access, but production is restricted.

If you’re unsure, start with **Synthesize all** on a curated entity table and restrict access via [Share a workspace](/setup-workspaces/share-a-workspace).

* You need exploration and BI dashboards without production access.
* You need correlations and distributions to stay useful.
* Many users need the same refreshable dataset.
* You need access control and privacy controls built-in.
* Use **De-identify** when analysts need production-like multi-table joins.

### When to avoid this use case

Skip this when exploration is not the goal.

* You need strict rule compliance for every row. Use [Use Case 4: ETL & Data Pipeline Testing](/overview/get-started/use-cases-and-configuration/use-case-4-etl-and-data-pipeline-testing).
* You need feature datasets for ML training and evaluation. Use [Use Case 6: ML Model Development](/overview/get-started/use-cases-and-configuration/use-case-6-ml-model-development).
* You need external data sharing with approvals and evidence. Use [Use Case 9: Data Sharing & Monetization](/overview/get-started/use-cases-and-configuration/use-case-9-data-sharing-and-monetization).
* You mainly need load and stress testing at scale. Focus on volume profiles and throughput tuning instead of analyst UX.

### Recommended Syntho configuration

This setup is optimized for **exploration with controlled privacy risk**. You preserve correlations and distributions. You reduce re-identification risk through privacy controls and access control.

{% stepper %}
{% step %}

#### Prerequisites

**Checklist**

* [ ] Audience and access policy defined (analysts vs data science).
* [ ] Refresh cadence defined (daily/weekly).
* [ ] Destination schema strategy chosen (blue/green if needed).

- Use the [Prerequisites](/overview/get-started/prerequisites) checklist.
  {% endstep %}

{% step %}

#### Source & destination management

Create one workspace per audience or policy. Example: `sandbox-analysts` vs `sandbox-data-science`.

#### Baseline rules

* Keep the **source stable**. Prefer snapshots or back-ups.
* Avoid a **live production** source for iterative work.
* Keep the **destination isolated**. Never write into production.
* Keep **schemas aligned** between source, workspace and destination.
* Use **views** when you need only a subset of the original database.

#### Lifecycle rule of thumb

* Keep the source connection when you expect schema changes.
* Remove the source connection when you expect a new run only much later.
* Revalidate after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).

**Nuances for this use case**

* Use roles and sharing to enforce access boundaries. Only a small group should change generators.
* Prefer blue/green schemas for refresh. Avoid breaking dashboards mid-refresh.
* Avoid in-place refreshes. Users see partial data and inconsistent aggregates.
* Don’t hide join keys in source views. Analysts will rebuild them manually and create privacy risk.
* [Create a workspace](/setup-workspaces/create-a-workspace)
* [Share a workspace](/setup-workspaces/share-a-workspace)
  {% endstep %}

{% step %}

#### Configure generators

**Workspace initialization mode**

Choose a [workspace mode](/setup-workspaces/create-a-workspace/workspace-modes). It applies baseline generator suggestions during workspace creation.

Recommended modes for this use case:

* **Synthesize all** when you can provide an entity table and you want strong utility for exploration.
* **De-identify** when analysts need multi-table joins that behave like production (and you mainly replace identifiers).
* **Mock or mask all** for “safe-by-default” sandboxes with minimal dependency on the original distributions.

**AI-generated synthesis**

Use this when analysts need **correlations and distributions** to stay useful for exploration.

**Example (BI-ready entity view):** create `sandbox_sales_entity_view` (customer segment, channel, order totals), then AI synthesize it into a single fact table for dashboards without exposing production data.

**Rule-based generation**

Use this when you must enforce **reporting conventions** or guarantee certain slices exist for dashboards. Use [Calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns) to keep dashboards stable.

**Example (stable “last 30 days” charts):** add a calculated `EDGE_RECENT` flag (e.g., 20% of rows), then set `order_date` to a random value in the last 30 days when `EDGE_RECENT` is true. This avoids empty “recent activity” charts after refresh.

```excel-formula
// New column: EDGE_RECENT (≈20% of rows)
RAND() < 0.20
```

```excel-formula
// Override: order_date (force recent rows for dashboards)
IF([EDGE_RECENT], DATEADD(TODAY(), -RANDBETWEEN(0, 30), "day"), [order_date])
```

**Masking**

Use this when BI tooling expects **format-valid codes** or when analysts need stable join keys in a de-identified relational sandbox.

**Example (stable dimension joins):** de-identify identifiers, keep consistent mapping for `product_id`, and mask `postal_code` to valid formats so geography dashboards and joins behave predictably.

**Hybrid**

Use this when you need both **utility** and **operational stability** for many users.

**Example (hierarchy correctness for geography dashboards):** enforce “city → province → country” in a dimension table (matches the “hierarchical relationship” scenario).

```excel-formula
// New column: province (derived from city)
SWITCH(UPPER(TRIM([city])),
  "TORONTO",  "ONTARIO",
  "MONTREAL", "QUEBEC",
  "VANCOUVER","BRITISH_COLUMBIA",
  "OTHER"
)
```

**Minimal configuration steps**

1. Create one curated entity view (BI-friendly).
2. Prefer **AI synthesize** for the entity table.
3. Apply masking/de-identification for identifiers that remain.

* [Automatic PII discovery with PII scanner](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner)
* [Manage personally identifiable information (PII)](/configure-a-data-generation-job/manage-personally-identifiable-information-pii)

<details>

<summary>Optional: BI-friendly dataset shape</summary>

* Prefer one fact table + small dimensions.
* Keep common filters (`country`, `segment`, `channel`).
* Avoid raw free-text unless needed.
* If you want one query-friendly table, create a view first. See [Use SQL views as input tables](/setup-workspaces/create-a-workspace/use-sql-views-as-input-tables).

</details>
{% endstep %}

{% step %}

#### Handle keys and relationships (relational schemas)

If you publish a **single sandbox table** (no joins), you can skip this step.

If the sandbox needs joins, make FKs explicit. Analysts will join tables in unpredictable ways.

If you do not need joins, flatten into an entity table before synthesis. This reduces privacy risk and simplifies validation.

* [Key generators](/configure-a-data-generation-job/configure-column-settings/key-generators)
  {% endstep %}

{% step %}

#### Validate and sync

Use the [QA report](broken://pages/tYZauKu0GdWCgMEjSKzN) when available to validate utility and privacy.

Revalidate after each refresh cycle. Sandbox users notice drift quickly.
{% endstep %}

{% step %}

#### Tune generation settings

Tune for interactive performance. Sandboxes are query-heavy.

Apply [Additional privacy controls](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation/privacy-controls) before widening access to more users.

Use [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings) when query latency becomes the bottleneck.

**Refresh and rollback strategy (low disruption)**

Avoid breaking dashboards during refreshes:

* Keep two destination schemas: `sandbox_blue` and `sandbox_green`.
* Refresh the inactive schema, validate dashboards, then switch BI connections.
* If something breaks, roll back by switching back to the previous schema.
  {% endstep %}
  {% endstepper %}

### Common pitfalls & misconfigurations

#### Use-case specific pitfalls

* Publishing sandboxes that still contain sensitive identifiers.
* Using entity tables that are too small for stable results.
* Over-sharing sandbox workspaces.
  * Use roles and data access controls. See [Share a workspace](/setup-workspaces/share-a-workspace).

<details>

<summary>General pitfalls</summary>

These pitfalls show up in most projects:

* Running full-scale jobs before a small validation run.
* Skipping workspace validation/sync after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).
* Breaking relational integrity (missing PK/FK setup, missing foreign keys, missing virtual foreign keys). Start with [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys) and [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys).
* Leaving sensitive columns on [**Duplicate**](/configure-a-data-generation-job/configure-column-settings/duplicate), or trusting the [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner) without reviewing false positives/negatives.
* Overusing [**Consistent mapping**](/configure-a-data-generation-job/configure-column-settings/consistent-mapping) (it slows down data generation and increases linkability).

</details>

### Governance, compliance, and automation

#### Use-case specific recommendations

* Use strict roles: many **Readers**, very few **Editors**. Analysts should not change generators.
* Use blue/green refresh for sandboxes. Automate refresh into the inactive schema, validate, then switch.
* Publish a lightweight data dictionary and refresh timestamp with every refresh. Analysts need lineage to trust results.
* Automate drift checks on key aggregates (top segments, null rates, distinct counts). Alert when the sandbox changes materially.

<details>

<summary>General recommendations</summary>

Use these recommendations for most workspaces.

#### Ownership and change control

* Assign a single **workspace owner** (data steward / privacy lead / DBA).
* Require a ticket or change request for generator changes.
* Duplicate a workspace before large edits. Keep the previous version as rollback.

#### Access control

* Default to **read-only** access for source connections.
* Restrict **who can view source data** in the UI.
* Use separate workspaces per environment or audience.

#### Automation (baseline)

* Use the [Syntho REST API](/syntho-api/syntho-rest-api) to standardize scans and runs.
* Automate data generation not workspace configuration.
* Keep job logs for failed runs. This reduces back-and-forth during support.

</details>


# Use Case 8: Cloud & data migration

Validate data workflows during migrations.

Use this use case when you need privacy-safe data to validate migrations. This includes scenarios where schemas and infrastructure are changing.

### What problem this use case solves

Teams need to validate data generation workflows across environments. They often need repeatable runs as schemas evolve.

Classic anonymization can over-generalize values and reduce realism. It can also be slow to repeat at scale.

### When to choose this use case

Pick this when you validate a migration across environments.

If you’re unsure, start with **De-identify** and keep generator configs identical across both environments. Validate after every schema change.

* You compare behavior across old and new platforms.
* You expect schema drift and need repeatable re-sync cycles.
* You need privacy-safe data in temporary migration environments.
* You need the same masking logic on both targets.

### When to avoid this use case

Skip this when migration validation is not the target.

* You need an analytics sandbox for exploration. Use [Use Case 7: Analytics Sandboxes](/overview/get-started/use-cases-and-configuration/use-case-7-analytics-sandboxes).
* You mainly need load testing at scale. Use [Use Case 2: Load & Stress](/overview/get-started/use-cases-and-configuration/use-case-2-load-and-stress-testing).
* You need end-to-end pipeline testing in a stable environment (not a platform migration). Use [Use Case 4: ETL & Data Pipeline Testing](/overview/get-started/use-cases-and-configuration/use-case-4-etl-and-data-pipeline-testing).
* You need ML training data. Prefer an entity-table style dataset and [AI synthesize](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation).

### Recommended Syntho configuration

This setup is optimized for **repeatable migration validation**. You expect schema drift. You need the same generation logic on old and new platforms.

{% stepper %}
{% step %}

#### Prerequisites

**Checklist**

* [ ] “Migration success” checks defined (counts, null rates, key distincts).
* [ ] Same source snapshot will be used for both targets.
* [ ] Generator config will be identical across workspaces.
* [ ] Privacy posture decided for each target environment.

{% hint style="warning" %}
Don’t compare two targets fed by two different source snapshots. Drift looks like migration defects.
{% endhint %}

* Use the [Prerequisites](/overview/get-started/prerequisites) checklist.
  {% endstep %}

{% step %}

#### Source & destination management

Create separate workspaces per target platform or environment. Example: `onprem-test` and `cloud-test`.

#### Baseline rules

* Keep the **source stable**. Prefer snapshots or back-ups.
* Avoid a **live production** source for iterative work.
* Keep the **destination isolated**. Never write into production.
* Keep **schemas aligned** between source, workspace and destination.
* Use **views** when you need only a subset of the original database.

#### Lifecycle rule of thumb

* Keep the source connection when you expect schema changes.
* Remove the source connection when you expect a new run only much later.
* Revalidate after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).

**Nuances for this use case**

* Use the same source snapshot across environments. Otherwise drift looks like migration defects.
* Keep generator configs aligned across the two workspaces. Masking differences can look like migration defects.
* Prefer behavioral equivalence checks. Avoid row-level equality unless you truly need it.
* [Create a workspace](/setup-workspaces/create-a-workspace)
* [Workspace modes](/setup-workspaces/create-a-workspace/workspace-modes)
  {% endstep %}

{% step %}

#### Configure generators

**Workspace initialization mode**

Choose a [workspace mode](/setup-workspaces/create-a-workspace/workspace-modes). It applies baseline generator suggestions during workspace creation.

Match the mode to your migration phase. Early phases need iteration. Later phases need locked configs.

Recommended modes for this use case:

* **De-identify** when you need migration parity across many related tables and you compare behavior across platforms.
* **Mock or mask all** for fast iteration when you mainly test schema compatibility and basic application flows.
* **Synthesize all** only when migration validation is based on a single entity table and you’re not comparing record-level behavior.

**AI-generated synthesis**

Use this only when migration validation is **behavioral** and scoped to a **single entity table**. Avoid it when you need parity across many joined tables.

**Example (warehouse-only validation):** synthesize a `migr_orders_entity_view` into both old and new platforms to validate query performance and datatype behavior without requiring record-level parity.

**Rule-based generation**

Use this to stress migration edge cases like **length truncation**, **nullability**, and **timezone boundaries**. Use [Calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns) to inject boundary values.

**Example (precision boundary):** enforce a destination-safe numeric scale and inject a small % of boundary values.

```excel-formula
// New column: MIG_BOUNDARY (≈0.5% of rows)
RAND() < 0.005
```

```excel-formula
// New column: order_amount_2dp (common migration pitfall: precision/scale)
IF([MIG_BOUNDARY], ROUND([order_amount], 2), [order_amount])
```

**Masking**

This is the default for migration comparisons. It preserves **formats** and keeps multi-table behavior close to production while removing identifiers.

**Example (same masking on both targets):** mask `email`, `iban`, and `phone_number` using identical generator settings in both workspaces. Enable consistent mapping for join keys so old vs new comparisons are fair.

**Hybrid**

Use this when you want parity for the core schema, plus targeted stress cases.

**Example (parity + boundary tests):** de-identify the full relational dataset for both targets, then inject the same boundary values in both workspaces so comparisons stay fair.

```excel-formula
// Override: created_at (inject migration boundary dates)
IF([MIG_BOUNDARY], DATE(YEAR(TODAY()), 3, 31), [created_at])
```

**Minimal configuration steps**

1. Keep both workspaces aligned (same tables, same generators).
2. Use de-identification + masking for multi-table parity.
3. Add boundary injections (precision, dates) with calculated columns.
4. Validate and compare using behavioral checks (counts, nulls, distributions).

* [Automatic PII discovery with PII scanner](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner)
* [Manage personally identifiable information (PII)](/configure-a-data-generation-job/manage-personally-identifiable-information-pii)
  {% endstep %}

{% step %}

#### Handle keys and relationships (relational schemas)

If your migration validation is **single-table** (or you validate only a curated layer), you can skip this step.

Migration failures often show up as broken joins. Make FK behavior explicit.

Use [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys) and add [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys) when needed. Keep FK configs consistent across the two workspaces.

* [Key generators](/configure-a-data-generation-job/configure-column-settings/key-generators)
  {% endstep %}

{% step %}

#### Validate and sync

Schema drift is normal during migration. Treat validation as part of every run.

Run validation after schema changes and before comparing results across platforms. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).

* [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace)

**Compare old vs new (quick checklist)**

* Row counts per table/partition.
* Null rates and distinct counts for key business fields.
* Schema-level constraints that changed (nullable, lengths, precision).

<details>

<summary>Optional: deeper migration checks</summary>

Focus on equivalence of behavior, not identical rows.

Common migration edge cases to watch:

* `VARCHAR` length truncation (destination max length < source).
* Timestamp timezone behavior differences (`TIMESTAMP` vs `TIMESTAMPTZ`).
* Numeric precision/scale mismatches (e.g., `DECIMAL(18,2)` vs `FLOAT`).

If you need a quick “same-ness” signal, compute checksums on a stable projection of columns in each environment and compare those checksums per table or per partition.

</details>
{% endstep %}

{% step %}

#### Tune generation settings

Tune for “migrate-and-validate” loops. You need stable runtime and predictable writes.

Use [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings) and [Large workloads](/overview/get-started/syntho-bootcamp/9.-large-workloads) tuning once the config is correct.

* [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings)
* [Large workloads](/overview/get-started/syntho-bootcamp/9.-large-workloads)
  {% endstep %}
  {% endstepper %}

### Common pitfalls & misconfigurations

#### Use-case specific pitfalls

* Treating a migration run as “one-off” when the schema is still evolving.
* Datatype mismatches between old and new platforms.
  * Validate schema alignment early. See [Prerequisites](/overview/get-started/prerequisites).

<details>

<summary>General pitfalls</summary>

These pitfalls show up in most projects:

* Running full-scale jobs before a small validation run.
* Skipping workspace validation/sync after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).
* Breaking relational integrity (missing PK/FK setup, missing foreign keys, missing virtual foreign keys). Start with [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys) and [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys).
* Leaving sensitive columns on [**Duplicate**](/configure-a-data-generation-job/configure-column-settings/duplicate), or trusting the [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner) without reviewing false positives/negatives.
* Overusing [**Consistent mapping**](/configure-a-data-generation-job/configure-column-settings/consistent-mapping) (it slows down data generation and increases linkability).

</details>

### Governance, compliance, and automation

#### Use-case specific recommendations

* Run paired workspaces per target (`onprem`, `cloud`). Keep generator configs in lockstep by policy.
* Automate side-by-side checks: row counts, null rates, distinct counts for keys, and a checksum over a stable projection.
* Require “same snapshot” as an explicit prerequisite in the migration checklist. Otherwise comparisons are meaningless.
* Log every schema drift event with a resync/validation run. Treat it as part of the migration timeline.

<details>

<summary>General recommendations</summary>

Use these recommendations for most workspaces.

#### Ownership and change control

* Assign a single **workspace owner** (data steward / privacy lead / DBA).
* Require a ticket or change request for generator changes.
* Duplicate a workspace before large edits. Keep the previous version as rollback.

#### Access control

* Default to **read-only** access for source connections.
* Restrict **who can view source data** in the UI.
* Use separate workspaces per environment or audience.

#### Automation (baseline)

* Use the [Syntho REST API](/syntho-api/syntho-rest-api) to standardize scans and runs.
* Automate data generation not workspace configuration.
* Keep job logs for failed runs. This reduces back-and-forth during support.

</details>


# Use Case 9: Data sharing & monetization

Share data with strong privacy protection.

Use this use case when data must be made available beyond the original production boundary. Approvals by privacy officers or boards may be needed.

### What problem this use case solves

Teams need to make data available to other consumers. They need strong privacy protection with measurable quality.

Classic anonymization can leak secondary identifiers. It can also reduce utility through broad generalization.

### When to choose this use case

Pick this when data leaves your team or organization.

If you’re unsure, start with **Synthesize all**, avoid [Consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping), and run the [QA report](broken://pages/tYZauKu0GdWCgMEjSKzN) before each release.

* You share data with partners, vendors, or customers.
* You need sign-off and a clear privacy posture.
* You need privacy and utility evidence per delivery.
* You need low linkability across releases and recipients.
* Avoid stable pseudonyms unless required by contract.

### When to avoid this use case

Skip this when the data stays internal.

* You only need internal demo datasets. Use [Use Case 3: Demo Data](/overview/get-started/use-cases-and-configuration/use-case-3-demo-data).
* You only need internal training datasets. Use [Use Case 12: Training & Education](/overview/get-started/use-cases-and-configuration/use-case-12-training-and-education).
* You need stable, refreshable access for internal analysts. Use [Use Case 7: Analytics Sandboxes](/overview/get-started/use-cases-and-configuration/use-case-7-analytics-sandboxes).
* You need reversible mapping back to original records for internal debugging. Prefer de-identification with controlled access and tight auditing.

### Recommended Syntho configuration

This setup is optimized for **external sharing with strong privacy posture**. You minimize linkability. You validate privacy and utility before distribution.

{% stepper %}
{% step %}

#### Prerequisites

**Checklist**

* [ ] Recipient, purpose, retention period defined.
* [ ] Share contract defined (view or extract).
* [ ] Approval/evidence requirements defined (QA report, privacy board).

{% hint style="warning" %}
Avoid stable pseudonyms unless contractually required. They increase linkability across releases.
{% endhint %}

* Use the [Prerequisites](/overview/get-started/prerequisites) checklist.
  {% endstep %}

{% step %}

#### Source & destination management

Create one workspace per external consumer or per policy. This prevents accidental reuse of a “less strict” config.

#### Baseline rules

* Keep the **source stable**. Prefer snapshots or back-ups.
* Avoid a **live production** source for iterative work.
* Keep the **destination isolated**. Never write into production.
* Keep **schemas aligned** between source, workspace and destination.
* Use **views** when you need only a subset of the original database.

#### Lifecycle rule of thumb

* Keep the source connection when you expect schema changes.
* Remove the source connection when you expect a new run only much later.
* Revalidate after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).

**Nuances for this use case**

* Lock down who can change generators. Treat generator changes like a release.
* Prefer “pull once, publish many”. Generate into staging, then distribute from the output.
* Don’t share the same database boundary as production. It blurs governance and access control.
* Don’t reuse a workspace for two recipients. One recipient’s requirements can silently weaken another’s posture.
* [Create a workspace](/setup-workspaces/create-a-workspace)
* [Share a workspace](/setup-workspaces/share-a-workspace)
  {% endstep %}

{% step %}

#### Configure generators

**Workspace initialization mode**

Choose a [workspace mode](/setup-workspaces/create-a-workspace/workspace-modes). It applies baseline generator suggestions during workspace creation.

Recommended modes for this use case:

* **Synthesize all** when you want maximum unlinkability and strong statistical utility (best default for external sharing).
* **Mock all** when you must avoid learning from the source altogether (highest separation, often lower utility).
* **De-identify** only when the sharing contract explicitly requires production-like multi-table behavior and you can accept higher linkage risk.

**AI-generated synthesis**

This is typically the best option for external sharing. Use it for **strong unlinkability** with good statistical utility.

**Example (partner dataset as one table):** create `partner_share_view` with only approved columns, then AI synthesize it into a single share table. Apply rare-category protection before delivery.

**Rule-based generation**

Use this to enforce **release policy** rules such as bucketing, top-coding, or redaction of sensitive derived fields. Use [Calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns) to encode these policies.

**Example (k-anonymity-friendly bucketing):** replace `age` with calculated `age_band` and replace `income` with top-coded buckets. This reduces re-identification risk while keeping utility.

```excel-formula
// New column: age_band
IFS(
  [age] < 18,  "0-17",
  [age] < 35,  "18-34",
  [age] < 55,  "35-54",
  TRUE,        "55+"
)
```

```excel-formula
// New column: income_bucket (top-coded)
SWITCH(TRUE,
  [income] >= 200000, "200k+",
  [income] >= 100000, "100-199k",
  [income] >= 50000,  "50-99k",
  "0-49k"
)
```

**Masking**

Use this only when you must keep **format-valid values** for consumer systems. Be careful with stable pseudonyms across releases.

**Example (format contract without linkability):** mask `customer_reference` to a UUID format **without** consistent mapping so the same person cannot be linked across deliveries.

For text columns, use [Rare Category Protection](/configure-a-data-generation-job/configure-column-settings/mask/text#rare-category-protection) to replace infrequent values. You can also enable it with **Shuffle**.

**Hybrid**

Use this when you need synthesis for unlinkability, plus explicit policy constraints. It mirrors the “rare scenario / policy guardrails” mindset from [Example data generation scenarios](/overview/get-started/syntho-bootcamp/example-data-generation-scenarios).

**Example (synthesize + guardrails):**

1. AI synthesize a single “share contract” view (entity table).
2. Add calculated columns for bucketing and top-coding.
3. Use rare category protection and additional privacy controls before release.

One practical guardrail is removing day-level timestamps while keeping time utility:

```excel-formula
// New column: event_month (coarsen timestamps for sharing)
CONCATENATE(YEAR([event_date]), "-", MONTH([event_date]))
```

Then exclude the original `event_date` from the shared dataset.

**Minimal configuration steps**

1. Build a share-contract view with approved columns only.
2. Prefer **AI synthesize** on that view.
3. Add calculated columns for bucketing/top-coding.
4. Run the [QA report](broken://pages/tYZauKu0GdWCgMEjSKzN) and capture evidence for release.

* [Automatic PII discovery with PII scanner](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner)
* [Manage personally identifiable information (PII)](/configure-a-data-generation-job/manage-personally-identifiable-information-pii)

{% hint style="warning" %}
Avoid [Consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping) unless the sharing contract explicitly requires stable pseudonyms.
{% endhint %}
{% endstep %}

{% step %}

#### Handle keys and relationships (relational schemas)

If you share a **single flattened table** (common), you can skip this step.

Only keep multi-table relationships if the consumer truly needs joins. Every join key increases linkage risk.

If you must keep multiple tables, keep join keys consistent and document them. Otherwise, prefer a flattened entity table for sharing.

* [Key generators](/configure-a-data-generation-job/configure-column-settings/key-generators)

**Single-table sharing is the common path**

Many external consumers want one table that is easy to import and understand.

Typical approach:

1. Create a **view** that represents the shareable dataset (entity table).
2. Use that view as the input to Syntho.
3. Publish the generated output as a single table or file export.

See [Use SQL views as input tables](/setup-workspaces/create-a-workspace/use-sql-views-as-input-tables).
{% endstep %}

{% step %}

#### Validate and sync

Run the [QA report](broken://pages/tYZauKu0GdWCgMEjSKzN) when available. Treat the QA output as part of your release evidence.

Revalidate on every delivery. A small schema change can reintroduce sensitive fields.

**Privacy metrics and approvals (what to document)**

Treat sharing like a controlled release. Privacy reviewers typically expect:

* The intended recipient, purpose, and retention period.
* A list of removed/transformed identifiers and any stable pseudonyms used.
* Utility and privacy evidence (QA report where applicable, plus privacy controls used).
* A clear threshold statement (example): “k-anonymity ≥ 10 on {age\_band, region, product} after rare category protection”.

If your organization has a privacy board, capture sign-off before distributing the dataset.

**Package and distribute (practical)**

Make the dataset easier (and safer) to consume:

* Include a short data dictionary (column meaning, units, allowed values).
* Include metadata: refresh date, version, and known limitations.
* Use explicit licensing/usage terms when data leaves your organization.
  {% endstep %}

{% step %}

#### Tune generation settings

Apply [Additional privacy controls](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation/privacy-controls) before shipping. These controls matter more here than runtime.

Tune performance only after privacy posture is accepted. Speed is not the primary constraint for sharing.
{% endstep %}
{% endstepper %}

### Common pitfalls & misconfigurations

#### Use-case specific pitfalls

* Using consistent mapping for identifiers in external shares.
* Shipping outputs without capturing QA/validation evidence.
* Accidentally leaking indirect identifiers through rare categories.
  * For AI-generated categorical columns, see [Rare category protection](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#rare-category-protection).
  * For masked text columns, see [Rare Category Protection](/configure-a-data-generation-job/configure-column-settings/mask/text#rare-category-protection).

<details>

<summary>General pitfalls</summary>

These pitfalls show up in most projects:

* Running full-scale jobs before a small validation run.
* Skipping workspace validation/sync after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).
* Breaking relational integrity (missing PK/FK setup, missing foreign keys, missing virtual foreign keys). Start with [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys) and [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys).
* Leaving sensitive columns on [**Duplicate**](/configure-a-data-generation-job/configure-column-settings/duplicate), or trusting the [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner) without reviewing false positives/negatives.
* Overusing [**Consistent mapping**](/configure-a-data-generation-job/configure-column-settings/consistent-mapping) (it slows down data generation and increases linkability).

</details>

<details>

<summary>Governance, compliance, and automation</summary>

#### Governance, access control, and audit evidence

Keep the workspace configuration as a controlled artifact. Treat it like “test data release”.

#### Recommended roles

* **Workspace Owner**: data steward or privacy lead. Approves generator choices and sharing.
* **Workspace Editor**: data engineer or platform engineer. Implements configuration changes.
* **Workspace Reader**: testers, analysts, or trainees. Can run jobs but should not change rules.

See [Workspace & user management](/overview/get-started/syntho-bootcamp/8.-workspace-and-user-management) and [Share a workspace](/setup-workspaces/share-a-workspace).

#### Access control checklist

* Use **read-only** access to the **source** database for day-to-day users.
* Restrict **who can view source data** in the UI. Don’t default to broad access.
* Use a **dedicated destination** per environment (`dev`, `test`, `accept`, `sandbox`).
* Keep external recipients in a **separate workspace** with stricter settings.

#### Evidence for auditors (lightweight but useful)

Capture these items per delivery or refresh:

* Workspace name, owner, and intended audience.
* PII scan results and the final list of “PII columns + applied generator type”.
* Any enabled privacy controls (e.g., rare category protection, free-text de-identification scope).
* Validation output and/or QA report (when applicable).
* Approval notes (ticket link, privacy board sign-off, or risk acceptance).

#### Automation and deployment (reference)

You can automate workspace setup, scans, and generation runs via the [Syntho REST API](/syntho-api/syntho-rest-api).

</details>


# Use Case 10: Data subsetting

Reduce data footprint while preserving integrity.

Use this use case when you need a smaller, representative dataset that still behaves like production.

### What problem this use case solves

Teams need smaller non-production datasets. They want faster jobs and lower storage costs.

Working with full-size databases can be time-consuming. Large workloads can be infeasible without reducing scope.

### When to choose this use case

Pick this when your non-prod dataset is too big to handle.

If you’re unsure, start with an entity-based subset (example: “recent customers”), then follow [Configure subsetting](/subsetting/configure-subsetting) and apply [Mask](/configure-a-data-generation-job/configure-column-settings/mask) to sensitive columns.

* Full-size copies are too slow or too expensive.
* You need a smaller dataset that still supports key joins.
* You need faster refresh cycles for dev and test.
* You can define deterministic subset criteria.
* Run a [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner) on the retained slice to catch forgotten identifiers.

### When to avoid this use case

Skip this when you cannot safely slice the dataset.

* You need all tables and all rows (full copy), or you can’t define subset rules without breaking flows. Use [Use Case 1: Application & API Testing](/overview/get-started/use-cases-and-configuration/use-case-1-application-and-api-testing).
* You need to increase row counts (upsampling). Use [Use Case 2: Load & Stress](/overview/get-started/use-cases-and-configuration/use-case-2-load-and-stress-testing).
* You need strict statistical fidelity for modeling. Use [Use Case 6: ML Model Development](/overview/get-started/use-cases-and-configuration/use-case-6-ml-model-development).

### Recommended Syntho configuration

This setup is optimized for **right-sizing non-production datasets**. You reduce runtime and storage. You keep relationships so the subset remains usable.

{% stepper %}
{% step %}

#### Prerequisites

**Checklist**

* [ ] Target size and acceptance criteria defined.
* [ ] Subset criteria is deterministic (cohort, timeframe, region).
* [ ] FK graph is understood (what must be retained to keep joins).
* [ ] PII handling decided for the retained slice.

- Use the [Prerequisites](/overview/get-started/prerequisites) checklist.

{% hint style="info" %}
If your Syntho version shows “Coming soon” in the Subsetting UI, treat subsetting as a **pre-step**.

Create a smaller source dataset first (database-native subsetting, extraction, or limiting the scope of tables). Then run de-identification or generation on that reduced dataset.
{% endhint %}

<details>

<summary>Optional: if subsetting is “Coming soon” in your UI</summary>

You can still run the use case by preparing a subset in the database first:

1. Pick one target entity (often `customers`, `patients`, or `accounts`).
2. Extract a deterministic set of IDs (region, cohort, last activity).
3. Copy the entity rows plus linked tables needed for your flows.

</details>
{% endstep %}

{% step %}

#### Source & destination management

Keep subsetting workspaces separate from full-copy de-identification workspaces.

#### Baseline rules

* Keep the **source stable**. Prefer snapshots or back-ups.
* Avoid a **live production** source for iterative work.
* Keep the **destination isolated**. Never write into production.
* Keep **schemas aligned** between source, workspace and destination.
* Use **views** when you need only a subset of the original database.

#### Lifecycle rule of thumb

* Keep the source connection when you expect schema changes.
* Remove the source connection when you expect a new run only much later.
* Revalidate after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).

**Nuances for this use case**

* Your destination size is part of the goal. Prefer a destination that enforces “smaller by design”.
* Validate the FK graph before trusting the subset. Missing FKs cause silent data loss.
* Don’t write subsets into schemas that already contain full tables. You’ll create mixed-scale datasets and confusing joins.
* Don’t forget “dimension” tables. Otherwise you keep most of the database by accident.
* [Create a workspace](/setup-workspaces/create-a-workspace)
  {% endstep %}

{% step %}

#### Configure generators

**Workspace initialization mode**

Choose a [workspace mode](/setup-workspaces/create-a-workspace/workspace-modes). It applies baseline generator suggestions during workspace creation.

Recommended modes for this use case:

* **De-identify** when the subset will still be production-like and must preserve joins.
* **Mock or mask all** when you’re building a smaller dataset mainly for fast iteration and you don’t need original distributions.

**AI-generated synthesis**

Not usually the first choice for subsetting. Use it only when the subset becomes an **analytics-style entity table** and you want stronger unlinkability.

**Example (subset + synthesize a shareable slice):** subset to “last 90 days orders”, then synthesize a flattened `orders_entity_view` so the smaller dataset can be shared internally without row-level links.

**Rule-based generation**

Use this to enforce subset properties (coverage, deterministic criteria) and to add test-friendly flags. Use [Calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns) for deterministic labeling.

**Example (deterministic cohort label):** tag rows so teams can validate “this subset still matches our criteria”.

```excel-formula
// New column: subset_cohort (based on recency)
IF([last_login_date] >= DATEADD(TODAY(), -90, "day"), "RECENT_ACTIVE", "OLDER")
```

**Masking**

This is the common path: keep the smaller dataset production-like, but remove identifiers while preserving joins.

**Example (subset stays relational):** de-identify `customers` + linked tables, mask `email` and `phone`, and enable consistent mapping for `customer_id` so foreign keys remain valid in the reduced dataset.

**Hybrid**

Use this when you want a relational subset for testing, plus an analysis-friendly table for convenience.

**Example (two outputs, two purposes):**

1. Keep the relational subset for app testing (de-identify + consistent mapping for join keys).
2. Build a flattened `subset_summary_view` for analysis.
3. AI synthesize the flattened view for stronger unlinkability (single-table sweet spot).

If you need a stable “subset label” in the flattened view, add it with a calculated column:

```excel-formula
// New column: subset_label
IF([order_date] >= DATEADD(TODAY(), -30, "day"), "RECENT_30D", "HISTORIC")
```

**Minimal configuration steps**

1. Define the subset criteria (entity + selection rule).
2. Keep required linked tables to preserve joins.
3. Apply de-identification/masking to the retained slice.
4. Validate joins with real application queries.

* [Automatic PII discovery with PII scanner](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner)
* [Manage personally identifiable information (PII)](/configure-a-data-generation-job/manage-personally-identifiable-information-pii)
  {% endstep %}

{% step %}

#### Handle keys and relationships (relational schemas)

If your reduced dataset is **one table only**, you can skip this step.

Subsets fail on missing relationships. Fix foreign keys before you trust the slice.

Use [foreign key inheritance](/configure-a-data-generation-job/manage-foreign-keys/foreign-key-inheritance). Add [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys) when the database doesn’t define them. This ensures linked tables are pulled correctly.

* [Verify foreign keys](/subsetting/verify-foreign-keys)
* [Key generators](/configure-a-data-generation-job/configure-column-settings/key-generators)
  {% endstep %}

{% step %}

#### Validate and sync

Validate the subset with real application queries. Confirm that joins return expected results.

If you iterate on the schema or FK graph, re-run validation in [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace) before the next extraction.
{% endstep %}

{% step %}

#### Tune generation settings

Right-size early and iterate fast. This is the main ROI of subsetting.

Use [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings) and [Large workloads](/overview/get-started/syntho-bootcamp/9.-large-workloads) tuning when your subset job becomes the bottleneck.

* [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings)
* [Large workloads](/overview/get-started/syntho-bootcamp/9.-large-workloads)
  {% endstep %}
  {% endstepper %}

### Common pitfalls & misconfigurations

#### Use-case specific pitfalls

* Expecting “5% of target table” to equal 5% of the full database.

<details>

<summary>General pitfalls</summary>

These pitfalls show up in most projects:

* Running full-scale jobs before a small validation run.
* Skipping workspace validation/sync after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).
* Breaking relational integrity (missing PK/FK setup, missing foreign keys, missing virtual foreign keys). Start with [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys) and [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys).
* Leaving sensitive columns on [**Duplicate**](/configure-a-data-generation-job/configure-column-settings/duplicate), or trusting the [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner) without reviewing false positives/negatives.
* Overusing [**Consistent mapping**](/configure-a-data-generation-job/configure-column-settings/consistent-mapping) (it slows down data generation and increases linkability).

</details>

<details>

<summary>Governance, compliance, and automation</summary>

#### Governance, access control, and audit evidence

Keep the workspace configuration as a controlled artifact. Treat it like “test data release”.

#### Recommended roles

* **Workspace Owner**: data steward or privacy lead. Approves generator choices and sharing.
* **Workspace Editor**: data engineer or platform engineer. Implements configuration changes.
* **Workspace Reader**: testers, analysts, or trainees. Can run jobs but should not change rules.

See [Workspace & user management](/overview/get-started/syntho-bootcamp/8.-workspace-and-user-management) and [Share a workspace](/setup-workspaces/share-a-workspace).

#### Access control checklist

* Use **read-only** access to the **source** database for day-to-day users.
* Restrict **who can view source data** in the UI. Don’t default to broad access.
* Use a **dedicated destination** per environment (`dev`, `test`, `accept`, `sandbox`).
* Keep external recipients in a **separate workspace** with stricter settings.

#### Evidence for auditors (lightweight but useful)

Capture these items per delivery or refresh:

* Workspace name, owner, and intended audience.
* PII scan results and the final list of “PII columns + applied generator type”.
* Any enabled privacy controls (e.g., rare category protection, free-text de-identification scope).
* Validation output and/or QA report (when applicable).
* Approval notes (ticket link, privacy board sign-off, or risk acceptance).

#### Automation and deployment (reference)

You can automate workspace setup, scans, and generation runs via the [Syntho REST API](/syntho-api/syntho-rest-api).

</details>


# Use Case 11: Accelerate PoCs & pilots

Deliver privacy-safe datasets fast to validate ideas, integrations, and workflows.

Use this use case when you need to validate a new idea fast. This includes internal proof-of-concepts and external pilots.

The focus is **speed** and **safe collaboration**. You want data that behaves like production. You do not want production risk.

### What problem this use case solves

PoCs and pilots often stall on data access.

Teams wait on approvals, exports, and manual data prep. When data finally arrives, it is incomplete or unrealistic.

You need a repeatable way to provision privacy-safe, production-like datasets. You also need to share them with stakeholders.

### When to choose this use case

Pick this when speed is the primary constraint.

If you’re unsure, start with **De-identify**, keep scope to one “must-work” flow, and only mask fields validated by the app/API.

* You need a usable dataset in days, not weeks.
* You test one or two “must-work” workflows end-to-end.
* You collaborate across teams, vendors, or partners.
* Privacy rules block production copies.
* Run a [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner) before sharing outputs.
* Duplicate the workspace before big changes. See [Duplicate a workspace](/setup-workspaces/duplicate-a-workspace).

### When to avoid this use case

Skip this when you need a long-lived, governed setup.

* You need stable DTAP baselines for many teams. Use [Use Case 1: Application & API Testing](/overview/get-started/use-cases-and-configuration/use-case-1-application-and-api-testing).
* You need stable, repeatable demo narratives for sales or pre-sales. Use [Use Case 3: Demo Data](/overview/get-started/use-cases-and-configuration/use-case-3-demo-data).
* You need formal external data sharing approvals. Use [Use Case 9: Data Sharing & Monetization](/overview/get-started/use-cases-and-configuration/use-case-9-data-sharing-and-monetization).
* If you mainly need upsampling, focus on [AI synthesize](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation) and performance tuning.

### Recommended Syntho configuration

This setup is optimized for **fast iteration with safe sharing**.

You start simple. You get a working dataset. You then tighten rules where the PoC depends on them.

{% stepper %}
{% step %}

#### Prerequisites

**Checklist**

* [ ] “Must-work” workflow defined (1–2 flows).
* [ ] Sharing boundary defined (internal vs external pilot).
* [ ] Reset cadence defined (ad-hoc vs weekly).

- Use the [Prerequisites](/overview/get-started/prerequisites) checklist.

<details>

<summary>Quick start checklist: first 48 hours</summary>

Use this when you need momentum fast and you don’t want to over-design early.

**Day 0–1 (get a first dataset):**

* Choose one “must-work” flow (e.g., login → search → checkout).
* Agree on success metrics (example: “demo the flow with 0 PII leakage” and “refresh in <30 minutes”).
* Create a workspace via [Create a workspace](/setup-workspaces/create-a-workspace) and run a first [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner).
* Apply masking for validator-critical fields (emails, UUIDs, IBANs).
* Generate a small dataset first (smoke test), then scale up.

**Day 1–2 (iterate with stakeholders):**

* Review the dataset with the PoC team and privacy lead.
* Capture issues as a short backlog (missing tables, broken joins, unrealistic values).
* Duplicate the workspace before large changes. See [Duplicate a workspace](/setup-workspaces/duplicate-a-workspace). This makes rollbacks easy.

</details>
{% endstep %}

{% step %}

#### Source & destination management

Create one workspace per PoC or pilot. Examples: `poc-crm-integration`, `pilot-partner-x`.

* Duplicate a working workspace before big changes. This gives you a rollback point.
* Use simple versioned names like `v1`, `v2`, `baseline`, or `pilot-partner-x`.

#### Baseline rules

* Keep the **source stable**. Prefer snapshots or back-ups.
* Avoid a **live production** source for iterative work.
* Keep the **destination isolated**. Never write into production.
* Keep **schemas aligned** between source, workspace and destination.
* Use **views** when you need only a subset of the original database.

#### Lifecycle rule of thumb

* Keep the source connection when you expect schema changes.
* Remove the source connection when you expect a new run only much later.
* Revalidate after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).

**Nuances for this use case**

* For external pilots, treat the environment like an external share. Restrict access and default to stronger unlinkability.
* Don’t reuse a PoC workspace for a new initiative. Old generator decisions silently carry over.
* Clean up connectors and access after the pilot. Make it part of close-out.
* [Create a workspace](/setup-workspaces/create-a-workspace)
* [Duplicate a workspace](/setup-workspaces/duplicate-a-workspace)
  {% endstep %}

{% step %}

#### Configure generators

**Workspace initialization mode**

Choose a [workspace mode](/setup-workspaces/create-a-workspace/workspace-modes). It applies baseline generator suggestions during workspace creation.

Recommended modes for this use case:

* **De-identify** when you start from a production-like copy and need fast, safe parity.
* **Mock or mask all** when you want stronger separation from the source but still need realistic formats.
* **Mock all** when you have no usable source data yet (early product work).

**AI-generated synthesis**

Use this when the PoC needs **realistic distributions** or **more rows** quickly, and you’re not asserting row-level parity.

**Example (integration at scale):** synthesize a `poc_contacts_entity_view` to generate 5× more contacts so you can validate connector throughput and UI pagination without using real identifiers.

**Rule-based generation**

Use this to guarantee the PoC has the exact scenarios stakeholders will test. Use [Calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns) to keep demos repeatable.

**Example (must-have workflow states):** assign a stable stage based on a numeric key so every stage appears across refreshes.

```excel-formula
// New column: onboarding_stage (stable by id modulo)
SWITCH(MOD([customer_id], 3),
  0, "NEW",
  1, "IN_PROGRESS",
  2, "COMPLETED",
  "NEW"
)
```

Use an integer-like key column here. If your IDs are UUIDs, use a numeric surrogate key for the lab.

**Masking**

Use this when partner systems validate formats, and you need stable keys/joins during multiple PoC refreshes.

**Example (API contract fields):** mask `email`, `phone`, and `external_id` to valid formats, and enable consistent mapping for identifiers used across tables so the integration doesn’t break between runs.

**Hybrid**

Use this when you need speed plus just enough realism and governance to share safely.

**Example (fast baseline + scenario control):**

1. De-identify a production-like snapshot for quick parity.
2. Add calculated “scenario columns” that drive the PoC story (workflow stage, flags).
3. Publish a flattened stakeholder view and synthesize it if you need unlinkability.

A simple “demo label” trick that makes filtering obvious in UIs:

```excel-formula
// New column: demo_account_label
CONCATENATE(PROPER(MOCK_COMPANY_NAME), " - ", [onboarding_stage])
```

**Minimal configuration steps**

1. Start with de-identification for parity (or mock-first if no source exists).
2. Mask only the fields the integration validates (emails, UUIDs, IBANs).
3. Add 1–2 calculated “scenario” columns to drive the PoC story.
4. Run a [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner) before sharing.

* [Automatic PII discovery with PII scanner](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner)
* [Manage personally identifiable information (PII)](/configure-a-data-generation-job/manage-personally-identifiable-information-pii)
  {% endstep %}

{% step %}

#### Handle keys and relationships (relational schemas)

If your PoC uses a **single table** (no joins), you can skip this step.

PoCs fail on broken relationships.

Validate primary keys and foreign keys early. Add virtual keys if the schema is incomplete.

Use [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys) and add [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys) when the database schema is incomplete.

* [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys)
* [Key generators](/configure-a-data-generation-job/configure-column-settings/key-generators)
  {% endstep %}

{% step %}

#### Validate and sync

Validate quickly on a subset.

Run the “happy path” scenario the PoC exists to prove.

Re-run validation whenever schemas or requirements change. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).

* [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace)
  {% endstep %}

{% step %}

#### Tune generation settings

Optimize for short feedback loops.

Keep settings stable so results are comparable across iterations.

Use [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings) when runtime becomes the bottleneck.

* [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings)
  {% endstep %}
  {% endstepper %}

### Common pitfalls & misconfigurations

#### Use-case specific pitfalls

* Starting the PoC without a clear success definition.
* Using real production data in pilot environments.
* Enabling consistent mapping for external sharing without a privacy review.
* Over-modeling the dataset.
  * Get the critical flows working first.

<details>

<summary>General pitfalls</summary>

These pitfalls show up in most projects:

* Running full-scale jobs before a small validation run.
* Skipping workspace validation/sync after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).
* Breaking relational integrity (missing PK/FK setup, missing foreign keys, missing virtual foreign keys). Start with [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys) and [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys).
* Leaving sensitive columns on [**Duplicate**](/configure-a-data-generation-job/configure-column-settings/duplicate), or trusting the [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner) without reviewing false positives/negatives.
* Overusing [**Consistent mapping**](/configure-a-data-generation-job/configure-column-settings/consistent-mapping) (it slows down data generation and increases linkability).

</details>

<details>

<summary>Governance, compliance, and automation</summary>

#### Governance, access control, and audit evidence

Keep the workspace configuration as a controlled artifact. Treat it like “test data release”.

#### Recommended roles

* **Workspace Owner**: data steward or privacy lead. Approves generator choices and sharing.
* **Workspace Editor**: data engineer or platform engineer. Implements configuration changes.
* **Workspace Reader**: testers, analysts, or trainees. Can run jobs but should not change rules.

See [Workspace & user management](/overview/get-started/syntho-bootcamp/8.-workspace-and-user-management) and [Share a workspace](/setup-workspaces/share-a-workspace).

#### Access control checklist

* Use **read-only** access to the **source** database for day-to-day users.
* Restrict **who can view source data** in the UI. Don’t default to broad access.
* Use a **dedicated destination** per environment (`dev`, `test`, `accept`, `sandbox`).
* Keep external recipients in a **separate workspace** with stricter settings.

#### Evidence for auditors (lightweight but useful)

Capture these items per delivery or refresh:

* Workspace name, owner, and intended audience.
* PII scan results and the final list of “PII columns + applied generator type”.
* Any enabled privacy controls (e.g., rare category protection, free-text de-identification scope).
* Validation output and/or QA report (when applicable).
* Approval notes (ticket link, privacy board sign-off, or risk acceptance).

#### Automation and deployment (reference)

You can automate workspace setup, scans, and generation runs via the [Syntho REST API](/syntho-api/syntho-rest-api).

</details>


# Use Case 12: Training & Education

Create safe, realistic datasets for onboarding, workshops, and hands-on training.

Use this use case when you need **safe, realistic data** for onboarding, workshops, and hands-on learning.

The focus is **repeatable training scenarios** without exposing real customer data.

### What problem this use case solves

Training requires data that feels real.

Real production data is usually blocked by privacy, security, and access controls. Manually created demo data often lacks realism and breaks workflows.

You need a dataset that supports realistic exercises. You also need a quick reset between sessions.

### When to choose this use case

Pick this when humans learn hands-on using realistic data.

If you’re unsure, start with **Mock or mask all**, keep the dataset small, and duplicate the workspace before every session.

* You run onboarding, enablement, or workshops.
* You need stable scenarios that always work.
* Many trainees share the same dataset.
* You need quick resets during sessions.
* Use [Consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping) only for storytelling.

### When to avoid this use case

Skip this when training is not the purpose.

* You share data externally beyond the training boundary. Use [Use Case 9: Data Sharing & Monetization](/overview/get-started/use-cases-and-configuration/use-case-9-data-sharing-and-monetization).
* You need analytics-grade statistical utility. Use [Use Case 7: Analytics Sandboxes](/overview/get-started/use-cases-and-configuration/use-case-7-analytics-sandboxes).
* You need stable demo narratives for product walkthroughs. Use [Use Case 3: Demo Data](/overview/get-started/use-cases-and-configuration/use-case-3-demo-data).
* If you need load testing at scale, prioritize volume profiles and destination tuning.

### Recommended Syntho configuration

This setup is optimized for **repeatable training exercises**. You want datasets that reset fast. You want stable examples for step-by-step instructions.

{% stepper %}
{% step %}

#### Prerequisites

**Checklist**

* [ ] Learning goals defined (PII scan, generators, FK handling).
* [ ] Dataset size kept small (fast resets).
* [ ] Trainees’ access decided (no source access by default).

- Use the [Prerequisites](/overview/get-started/prerequisites) checklist.
  {% endstep %}

{% step %}

#### Source & destination management

Create one workspace per training track. Examples: `training-basics`, `training-pii`, `training-foreign-keys`.

* Duplicate a working workspace before big changes. This gives you a rollback point.
* Use simple versioned names like `v1`, `v2`, `baseline`, or `pilot-partner-x`.

#### Baseline rules

* Keep the **source stable**. Prefer snapshots or back-ups.
* Avoid a **live production** source for iterative work.
* Keep the **destination isolated**. Never write into production.
* Keep **schemas aligned** between source, workspace and destination.
* Use **views** when you need only a subset of the original database.

#### Lifecycle rule of thumb

* Keep the source connection when you expect schema changes.
* Remove the source connection when you expect a new run only much later.
* Revalidate after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).

**Nuances for this use case**

* Prefer mock-first sources. Don’t use production copies for training.
* Keep datasets small. Resets should be minutes, not hours.
* Don’t give trainees source access. Training should not be a backdoor to production-like data.
* Don’t mix “storytelling” and “privacy” goals in one dataset. Use separate tracks or separate workspaces.
* [Create a workspace](/setup-workspaces/create-a-workspace)
* [Duplicate a workspace](/setup-workspaces/duplicate-a-workspace)
  {% endstep %}

{% step %}

#### Configure generators

**Workspace initialization mode**

Choose a [workspace mode](/setup-workspaces/create-a-workspace/workspace-modes). It applies baseline generator suggestions during workspace creation.

Recommended modes for this use case:

* **Mock or mask all** when you want safe, realistic values with minimal reliance on the source.
* **Mock all** when you want trainees to learn configuration from scratch without any production-like input.
* **De-identify** when you have a production-like training dataset and you want to teach privacy-safe replacement patterns.

**AI-generated synthesis**

Use this when training includes analytics/ML concepts and you want realistic correlations without exposing real people.

**Example (training on “churn”):** synthesize a `training_churn_features_view` so participants can build a simple model or dashboard with realistic feature relationships.

**Rule-based generation**

Use this to make exercises deterministic and repeatable. Use [Calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns) to keep labs stable.

**Example (scripted “bad rows” lab):** inject a small, known set of malformed values learners must find and fix.

```excel-formula
// New column: LAB_BAD_ROW (≈2% of rows)
RAND() < 0.02
```

```excel-formula
// Override: email (invalid only for lab rows)
IF([LAB_BAD_ROW], "not-an-email", [email])
```

**Masking**

Use this when labs require format-valid fields for validation exercises and relational joins.

**Example (PII lab):** mask `email` and `phone_number`, then enable consistent mapping for `customer_id` so learners can see that joins still work after de-identification.

**Hybrid**

Use this when you want safe realism, plus scripted teaching scenarios.

**Example (progressive lab setup):** follow the hybrid patterns in [Example data generation scenarios](/overview/get-started/syntho-bootcamp/example-data-generation-scenarios).

1. Mock names/addresses for safety.
2. Mask format-critical fields for validators (emails, UUIDs).
3. Use calculated columns to inject edge cases (LAB\_BAD\_ROW) and deterministic relations (e.g., gender → name).

If you want trainees to practice a classic “absolute calculation”, add this exercise:

```excel-formula
// New column: trial_end_date (derived field exercise)
DATEADD([signup_date], 14, "day")
```

**Minimal configuration steps**

1. Run a PII scan and review findings.
2. Apply mock/mask for the exercise scope.
3. Use calculated columns to inject lab tasks or derived fields.

* [Automatic PII discovery with PII scanner](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner)
* [Manage personally identifiable information (PII)](/configure-a-data-generation-job/manage-personally-identifiable-information-pii)

<details>

<summary>Progressive training scenarios (recommended)</summary>

Design training so learners build confidence, then complexity.

**Scenario A (Basics):** PII scan + safe replacements

* Goal: identify PII and apply mock/mask correctly.
* Exercise: run PII scan, fix one false positive and one false negative, then generate.

**Scenario B (Relational correctness):** keys + foreign keys

* Goal: keep joins working.
* Exercise: add one virtual FK, validate, and re-run generation.

**Scenario C (Edge cases):** inject rare cases for testing

* Goal: produce rows that trigger special logic.
* Exercise: add an `EDGE_FLAG` and override one column.

```excel-formula
// New column: EDGE_FLAG (≈2% of rows)
RAND() < 0.02
```

```excel-formula
// Example override: subscription_status
IF([EDGE_FLAG], "PAST_DUE", [subscription_status])
```

</details>
{% endstep %}

{% step %}

#### Handle keys and relationships (relational schemas)

If the training dataset is **single-table**, you can skip this step.

Training breaks fast on missing relationships.

Validate foreign keys early. Use [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys). Add [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys) if the schema is incomplete.

* [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys)
* [Key generators](/configure-a-data-generation-job/configure-column-settings/key-generators)
  {% endstep %}

{% step %}

#### Validate and sync

Validate a small slice first.

Run the exercises end-to-end as a trainee would.

Re-run validation whenever the training schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).

* [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace)
  {% endstep %}

{% step %}

#### Tune generation settings

Prioritize fast reset times.

Stable settings make labs reproducible.

Use [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings) once the exercises are stable.

* [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings)
  {% endstep %}
  {% endstepper %}

### Common pitfalls & misconfigurations

#### Use-case specific pitfalls

* Using production copies for training environments.
* Making datasets too big.
  * Training should reset in minutes.
* Changing generator configs right before a session.
  * Duplicate a working workspace instead. See [Duplicate a workspace](/setup-workspaces/duplicate-a-workspace).
* Using consistent mapping by default.
  * Decide based on training goals: stable storytelling vs strict unlinkability.

<details>

<summary>General pitfalls</summary>

These pitfalls show up in most projects:

* Running full-scale jobs before a small validation run.
* Skipping workspace validation/sync after schema changes. Use [Validate and synchronize workspace](/configure-a-data-generation-job/generation-and-validation/validate-and-synchronize-workspace).
* Breaking relational integrity (missing PK/FK setup, missing foreign keys, missing virtual foreign keys). Start with [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys) and [virtual foreign keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/add-virtual-foreign-keys).
* Leaving sensitive columns on [**Duplicate**](/configure-a-data-generation-job/configure-column-settings/duplicate), or trusting the [PII scan](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner) without reviewing false positives/negatives.
* Overusing [**Consistent mapping**](/configure-a-data-generation-job/configure-column-settings/consistent-mapping) (it slows down data generation and increases linkability).

</details>

<details>

<summary>Governance, compliance, and automation</summary>

#### Governance, access control, and audit evidence

Keep the workspace configuration as a controlled artifact. Treat it like “test data release”.

#### Recommended roles

* **Workspace Owner**: data steward or privacy lead. Approves generator choices and sharing.
* **Workspace Editor**: data engineer or platform engineer. Implements configuration changes.
* **Workspace Reader**: testers, analysts, or trainees. Can run jobs but should not change rules.

See [Workspace & user management](/overview/get-started/syntho-bootcamp/8.-workspace-and-user-management) and [Share a workspace](/setup-workspaces/share-a-workspace).

#### Access control checklist

* Use **read-only** access to the **source** database for day-to-day users.
* Restrict **who can view source data** in the UI. Don’t default to broad access.
* Use a **dedicated destination** per environment (`dev`, `test`, `accept`, `sandbox`).
* Keep external recipients in a **separate workspace** with stricter settings.

#### Evidence for auditors (lightweight but useful)

Capture these items per delivery or refresh:

* Workspace name, owner, and intended audience.
* PII scan results and the final list of “PII columns + applied generator type”.
* Any enabled privacy controls (e.g., rare category protection, free-text de-identification scope).
* Validation output and/or QA report (when applicable).
* Approval notes (ticket link, privacy board sign-off, or risk acceptance).

#### Automation and deployment (reference)

You can automate workspace setup, scans, and generation runs via the [Syntho REST API](/syntho-api/syntho-rest-api).

</details>


# Sample datasets

To provide users with practical examples for testing and analytics, we have selected datasets optimized for various scenarios. These datasets are sourced from well-known repositories and are designed to help users get started with Syntho's features effectively. For testing purposes, you can access a **multi-table dataset**, while for analytics, there is a **single-table dataset**. These datasets serve as a practical starting point for exploring Syntho's features and capabilities:

## **Census dataset**

* **Use Case**: Ideal for analytics and AI model training.
* **Description**: Contains demographic information, including age, education, occupation, and income classification.
* **Source**: [UCI Machine Learning Repository - Adult Dataset](https://archive.ics.uci.edu/dataset/2/adult).

<figure><img src="/files/lBPMSZjfonKF290PdfnF" alt=""><figcaption><p>A screenshot from census dataset</p></figcaption></figure>

Click below link to download `.csv` file.

{% file src="/files/RO3muY2hNLcw74HNwQnv" %}
Census dataset
{% endfile %}

## **COVID-19 dataset**

* **Use Case**: Useful for testing synthetic data generation on multi-table healthcare-related datasets.
* **Description**: Includes tables such as patients, conditions, encounters etc. simulated for COVID-19 scenarios.
* **Source**: [Synthea COVID Patients Dataset](https://synthea.mitre.org/downloads).

<figure><img src="/files/agUZXlWk23tHQOcPN7e2" alt=""><figcaption><p>A screenshot from patients table</p></figcaption></figure>

Click below link to download `.zip` file for 10k patient records with COVID-19 in the CSV format. If you would like to download 100k patient records version, please click [here](https://mitre.box.com/shared/static/wk3560f962ozlg7sd2oj1zxk73ayqvm0.zip).

{% file src="/files/ks7hURZnT9B8eyBrk8g7" %}
Covid datasets with 10k records
{% endfile %}

## **Baseball dataset**

* **Use Case**: Suitable for analytics and relational dataset exploration.
* **Description**: Features player statistics and seasonal performance data.
* **Source**: [Lahman Baseball Dataset](https://lahman.r-forge.r-project.org/).

<figure><img src="/files/NSKhBlgyJtuwA5g90PGp" alt=""><figcaption><p>A screenshot from players table</p></figcaption></figure>

<figure><img src="/files/IpRhkjCn4Xxt6zPONKlh" alt=""><figcaption><p>A screenshot from seasons table</p></figcaption></figure>

Click below link to download `.zip` file.

{% file src="/files/mIABrc9U4TcMf1OdDoEt" %}
Baseball dataset
{% endfile %}


# Introduction to data generators

The Syntho platform offers various data generators for diverse scenarios, taking into account the data's nature, privacy concerns, and specific use cases, allowing users to select the most appropriate options. The summary table provides an overview of these methods, detailing their relevance and use-case scenarios below. You can select any of the data generators to be forwarded to the detailed user guide sections.

The below features are key for the smart de-identification and rule-based synthetic data methods.

<table data-full-width="false"><thead><tr><th>Key feature</th><th>Description</th><th>When to use</th><th>When not to use</th></tr></thead><tbody><tr><td><a href="/pages/TmyJdbwGRi9FCjnkVClu">AI-generated synthetic data</a></td><td>Training a generative AI model on the original data to generate new rows that mimic, but have no 1-to-1 relation with original rows.</td><td><ul><li>To generate synthetic feature dataset for ML model development</li><li>When statistical accuracy and maximum privacy are needed</li><li>To expand dataset rows while maintaining original statistical properties</li></ul></td><td><ul><li>When working with multiple related tables</li><li>When data consistency across systems is required</li><li>When you need to be able to revert to original records</li><li>If entirely new, unseen text values must be generated</li></ul></td></tr><tr><td><a href="/pages/5zxQAQDDY05BVgAvv4wT">Mockers</a></td><td>Generating entirely new, user-defined values</td><td>For custom data generation without regard to preserving original column value relationships</td><td>When you need to maintain relationships with original data</td></tr><tr><td><a href="/pages/f9F95DCiA9VPEWsv1ucc">Consistent Mapping with Mockers</a></td><td>To generate mock values that are consistently mapped from original values (e.g. Hank always becomes Jeffrey)</td><td>To ensure data consistency across tables, systems and data generation jobs</td><td>If fully random data, without consistency is desired</td></tr><tr><td><a href="/pages/uf8zm3z9Gk3PBDx61Ejc">Mask</a></td><td>Anonymizes data by modifying values directly while preserving the format</td><td>When data needs to remain recognizable in format. For anonymizing PII fields in non-production environments</td><td>When preserving exact relationships or values is required.</td></tr><tr><td><a href="/pages/IGE7PdfKclq1J0KY2yYN">Calculated Columns</a></td><td>Generating user-defined values based on custom logic</td><td>For complex data manipulations requiring specific business logic</td><td>For simple data generation tasks that don't need custom logic</td></tr><tr><td><a href="/pages/m9EiJVbPUi3W7cA0YmL4">Key generators</a></td><td>Generates unique keys to ensure referential integrity across related tables.</td><td>When working with multiple tables needing unique keys for foreign-key relationships.</td><td>If relationships or foreign keys are not required.</td></tr><tr><td><a href="/pages/a9SJ3fDWLeP1Yd66rAcP">PII scanner</a></td><td>Automatic discovery of most sensitive (i.e. PII/PHI) columns in you database</td><td>To discover most sensitive columns (i.e. PII / PHI)</td><td>When your data is not sensitive</td></tr></tbody></table>

## Comparison of data generated with different generators

We demonstrate the application of each generators on a real baseball dataset, which includes **players** and **seasons** tables.

### [AI-generated synthetic data](/overview/get-started/ai-generated-synthetic-data) is applied to players table

<figure><img src="/files/IBxmHYRqRRFs9oTccn2O" alt="" width="563"><figcaption><p>AI-generated synthetic data is applied to players table</p></figcaption></figure>

\
In the first example, we see that an entirely new synthetic dataset was generated by the generative AI model based on the original dataset. The synthetic dataset preserves the statistics of the original dataset, but there is no 1:flag\_to:1 correspondence of synthetic records and original records. Note that for AI-generated synthetic data, a rare category replacement value of 10 was applied. This means that any name appearing fewer than 10 times in the `nameFirst`and `nameLast` columns was replaced with an asterisk to protect privacy.

### [Mockers](/configure-a-data-generation-job/configure-column-settings/mockers) are applied to players table

<figure><img src="/files/s06zquf4Rta53OjEb1Z4" alt="" width="563"><figcaption><p>Mocker is applied to players table</p></figcaption></figure>

Mockers are applied to specific columns in the players table, which are highlighted in yellow in the table above: 'country', 'birthDate', 'deathDate', 'nameFirst', and 'nameLast'.

### [Consistent Mapping with Mockers](/configure-a-data-generation-job/configure-column-settings/consistent-mapping) is applied to players table

<figure><img src="/files/a98xRq8pVUDzQRKmW9p7" alt="" width="375"><figcaption><p>Enabling Consistent Mapping "nameFirst" in players table</p></figcaption></figure>

If you enable [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping), the values will be consistently mapped to the same value across the tables. For example, we enabled consistent mapping for two columns: "nameFirst" and "nameLast". We want to generate the same synthetic names and surnames (mockers) for the original names. See the illustrations from MySQL tables below, where mockers with consistent mapping map the name "Bill Kennedy" to "Danielle Olson".

Please note that other names can also be mapped to "Danielle" or "Olson"; however, whenever Syntho detects "Bill", it will always replace it with a mocker first name "Danielle". The same applies to "Kennedy" and "Olson" in the last name column. Consistency can be verified with other columns since they are duplicated without any change from source to destination, allowing original and synthetic tables to be matched for a better understanding of consistency.

<figure><img src="/files/wkLwwcuaAesEUwiS6UY6" alt="" width="563"><figcaption><p>Consistent mapping with mockers is applied to players tables</p></figcaption></figure>

### [Calculated Columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns) is applied to players table

[Calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns) allow users to perform a broad spectrum of operations on data, ranging from simple arithmetic to complex logical and statistical computations. In below illustration, the following operation is applied:

```dax
IF([Gender] = 'M', MOCK_FIRST_NAME, IF([Gender] = 'F', MOCK_FIRST_NAME_FEMALE, 'nothing'))
```

<figure><img src="/files/2P4XJqzFEQnmQcVLbHwW" alt="" width="375"><figcaption><p>Preview of the result of above Calculated Column function</p></figcaption></figure>

### [Mask](/configure-a-data-generation-job/configure-column-settings/mask) is applied to players table

The Mask generator modifies values directly without creating new records or altering the original dataset structure. This approach allows data to remain recognizable in its format while being anonymized, which is particularly useful for fields containing identifiable attributes. In this example, the Mask generator is applied to specific columns in the `players` table to ensure sensitive information is anonymized. Columns like `country` (Random Character Swap), `birthDate` (Datetime Noise), `deathDate` (Hasher), `nameFirst` (Format Preserving Encryption) and `nameLast` (Random Character Swap) are anonymized using the Mask generator with respective masking functions. These columns contain sensitive information that could potentially identify individuals. When consistent mapping is enabled in the Mask settings, identical input values across records will always map to the same masked output values.

<figure><img src="/files/ARX8PcAVtNIy24V5xdTH" alt="" width="563"><figcaption><p>Mask is applied to players table</p></figcaption></figure>

### Hash is applied to players table

In this example, the Hash generator is applied to key columns in the `players` table to ensure unique identifiers while preserving referential integrity. Key column, which is `id`, is hashed using a consistent algorithm to create anonymized yet unique values. This method ensures that identifiers in the `players` table can be anonymized without compromising the ability to link related records across tables. With unique hashing enabled, the output value will always be unique.

<figure><img src="/files/8VODc9iYoaY1O9B4Mvc8" alt="" width="563"><figcaption><p>Hash is applied to players table</p></figcaption></figure>

### [Key generator](/configure-a-data-generation-job/configure-column-settings/key-generators) is applied to players table

In this example, Key Generators are used to generate unique keys or duplicate keys in the `players` table, ensuring referential integrity across related tables.

* **Duplicate** copies the original key values exactly as they appear in the source data, preserving the relationships between primary and foreign keys. This ensures the key structure remains intact.
* **Hash** converts original key values into hashed representations while preserving relationships across tables. The hashed values are obfuscated and irreversible, ensuring relational integrity is maintained.
* **Generate** creates new, synthetic key values that do not correspond to the original keys. It produces entirely new keys and does not preserve the order or relationships of the source data.

<figure><img src="/files/OZrtRageHAjAfyLH28wr" alt=""><figcaption><p>3 different key generators are applied to id column of players table</p></figcaption></figure>


# AI-generated synthetic data

This guide provides the step-by-step procedures for AI-generated synthetic data for a single entity table.

The diagram below illustrates a workflow for AI-generated synthetic data. Detailed information about each step shown in the diagram is provided throughout this page.

<figure><img src="/files/7IMbMYLNXkDXbuWEsPvk" alt=""><figcaption><p>Workflow of AI-generated synthetic data process</p></figcaption></figure>

Before starting AI-generated synthetic data use case, check the video below that provides a short introduction to data generators.

{% embed url="<https://youtu.be/668vK84zCD8>" %}
A short introduction to data generators
{% endembed %}

For this key use case, a single table, named **census**, containing census-collected data, requires to be synthesized using Syntho's AI synthesize generator. Maximum privacy, alongside the generation of highly realistic data that statistically reflects the original dataset, is crucial for AI and analytics. The initial step involves creating a workspace in Syntho, linked to a census database. Once established, this workspace will feature the **census** table exclusively. Below, the table's columns and a sample of the contained data are presented.

<figure><img src="/files/YzUNEqUKBO8ID3oMFY11" alt=""><figcaption><p>Table census</p></figcaption></figure>

***

## [Prerequisites](/overview/get-started/prerequisites)

For prerequisites check [Prerequisites](/overview/get-started/prerequisites) or watch the video below.

{% embed url="<https://youtu.be/BveahcOYYVk>" %}
Prerequisites for AI generated synthetic data
{% endembed %}

***

## Preparing your data

For AI synthesize, ensure your data is fit to synthesize. Syntho expects your data to be stored in an **entity table** that adheres to specific guidelines:

1. Maintain a minimum **column-to-row ratio of 1:500** for privacy and algorithmic generalization. With 15 columns, aim for at least 7,500 rows; our example database exceeds this with 48,842 rows (see below illustration).
2. Describe **each entity in a single row**, ensuring row independence without sequential information. That means each row can be treated independently. The order of the rows does not convey any information. The contents of one row also do not affect other rows.
3. **Remove columns that are derived directly from other columns and not contain additional information**. For example, you may have a redundant duration column that is derived from the start\_time and end\_time columns. For categorical columns, there could be hierarchical relationships, such as a redundant Treatment category column referring to a Treatment Type column. Removing such columns containing redundant information will simplify the modeling process and will lead to higher quality synthetic data. If not removed, such columns constructed with [calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns) feature.
4. Avoid column names with privacy-sensitive information, like patient\_a\_medications, patient\_b\_medications, etc. Instead, simply have a patient column with the names in it. This prevents patient names from being exposed in metadata or bypass rare category protection (e.g., there’s a patient\_a column, but this patient only appeared five times in the whole dataset).

Our example table fully meets these criteria.

<figure><img src="/files/GBAb01PFjoMO7WuC4WX8" alt="" width="375"><figcaption><p>Calculating rows</p></figcaption></figure>

{% embed url="<https://youtu.be/i--iXsJNWag>" %}
Preparing your data
{% endembed %}

## Configuring Column Settings

Under **Column settings > Generation Method**, select **AI synthesize** for Syntho's ML models to synthesize data.

{% embed url="<https://youtu.be/w0ABWEI51ss>" %}
Configure table settings
{% endembed %}

## [Rare category protection](#rare-category-protection)

This feature helps hide sensitive or rare observations, like specific occupations in the **census** table, by replacing them with a user-defined value, enhancing data privacy. Adjust the **rare category protection threshold** and **replacement value** in **Column settings > Encoding type > Advanced settings**.

<figure><img src="/files/2qZhMcCOCtTqkavZrBvC" alt="" width="186"><figcaption><p>Column occupation</p></figcaption></figure>

* **Rare category protection threshold**: All column values that occur as frequently or less than the rare category protection threshold are automatically replaced.
* **Rare category replacement value**: All column values that occur as frequently or less than the rare category protection threshold are automatically replaced by this replacement value.

For example, occupations appearing **as frequently or less than 15**, will be replaced with the sign asterisk or “\*”. The number and replacement value are voluntarily and can be defined per user request (See below illustration).

<figure><img src="/files/zOeZQXrqn6P7AwKvRp8E" alt="" width="375"><figcaption><p><strong>Rare category protection</strong></p></figcaption></figure>

{% embed url="<https://youtu.be/IK-9IFyvB80>" %}
Rare category protection
{% endembed %}

## [PII scanner](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner) and [Mockers](/configure-a-data-generation-job/configure-column-settings/mockers)

{% hint style="info" %}
The PII scanner provides a starting point for PII detection. Users should perform additional reviews to identify and handle any other sensitive data that may not be detected by the scanner.
{% endhint %}

### [PII scanner](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner)

In the **PII tab**, you can add new columns to the list of PII columns, either manually or by using Syntho's **PII scanner**. You have the option to manually label columns containing PII by selecting the column name and optionally choosing a mocker to apply. Clicking "**Confirm**" will mark the column as containing PII and confirm the **mocker** selection.

<figure><img src="/files/U2YRW2H5wEX72BVZUXb6" alt="" width="563"><figcaption><p>Manual PII labelling</p></figcaption></figure>

Alternatively, deploy automatic PII discovery with the PII scanner. Launch a scan to detect PII across all database columns from the PII tab in the **Job Configuration** panel. Note that the scanner offers both **Shallow** and **Deep** scan modes:

* The **shallow scan** assesses columns using regular expression rules to identify PII, optimized for speed but with variable accuracy.
* The **deep scan** examines both metadata and data within columns for a thorough PII identification.

<figure><img src="/files/R3ukKo03Pf6jvPquGAsP" alt="" width="563"><figcaption><p>Automatic PII labelling</p></figcaption></figure>

### [Mockers](/configure-a-data-generation-job/configure-column-settings/mockers)

As with de-identification, you can use mockers or exclude to replace any PII columns. If not, those PII columns will be treated as categorical columns and processed by Syntho's “[Rare category protection](#rare-category-protection)”.

{% embed url="<https://youtu.be/JwoiuoH6Abc>" %}
PII scanner and Mockers
{% endembed %}

## [Advanced generator settings](#advanced-generator-settings)

In the job settings, under table settings, you can adjust generator-level settings, including the maximum number of rows for training to optimize speed. Leaving this setting as **None** utilizes all rows. The **Take random sample** option allows for sampling:

* **On**: Random rows are selected for training.
* **Off**: Top rows as per the database are used.

## Start data generation process

To start data generation, you can do the following:

1. On the **Job configuration** panel, select **Generate.**
2. On the **Job configuration summary** panel, adjust generation parameters as desired.
3. Finally, select **Start generating**.

## [Model parameters](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings)

Before initiating the generation process, you have the option to modify model parameters. Here's an overview:

* **Batch size:** The maximum datapoints Syntho reads, generates, and writes per batch.
* **Number of simultaneous connections:** Specifies how many tables Syntho can read from or write to in parallel.

<figure><img src="/files/3hZjBseRLkGcyypZlRin" alt="" width="375"><figcaption><p>Model parameters</p></figcaption></figure>

### Truncate tables before each new data generation job

Users are required to manually **TRUNCATE** their tables in the **DESTINATION** database before initiating each new data generation job. If truncation is hindered due to existing constraints, these constraints should be temporarily disabled before truncation and then re-enabled afterwards. For instance, to facilitate the truncation process when foreign key constraints prevent it, use the following SQL commands: First, disable the constraints by executing `SET FOREIGN_KEY_CHECKS = 0;`, then **TRUNCATE** the table, and finally, re-enable the constraints with `SET FOREIGN_KEY_CHECKS = 1;`. This sequence ensures that tables are properly prepared for data generation without constraint violations.

{% embed url="<https://youtu.be/0YCd5KQgpRw>" %}
Start a data generation process
{% endembed %}

## Evaluation

[SDMetrics ](https://docs.sdv.dev/sdmetrics)is an [open-source](https://github.com/sdv-dev/SDMetrics) **Python** library designed for evaluating tabular synthetic data to determine how closely it mimics the mathematical properties of real data, known as synthetic data fidelity, through selective metric evaluation, detailed results explanation, score visualization, and report sharing capabilities.

For full metrics and information, please check the SDMetrics documentation by clicking on this [link](https://docs.sdv.dev/sdmetrics).

In the notebook below, we compare some real and synthetic demo data using SDMetrics based on SDMetrics original notebook. Also, you will find a shareable report, and you can use it to discover some insights and create visual graphics.

{% file src="/files/X6I30YqNqQUjGPbPvViu" %}
Jupyter Notebook
{% endfile %}


# Database de-identification

This guide describes the steps for fully de-identifying a production database.

<figure><img src="/files/IcVBqOwyElBaU8S2joNU" alt=""><figcaption><p>Workflow of database de-identification process</p></figcaption></figure>

Before starting database de-identification use case, check the video below that provides a short introduction to data generators:

{% embed url="<https://youtu.be/668vK84zCD8>" %}
A short introduction to data generators
{% endembed %}

Syntho helps customers ensure columns containing **personally identifiable information (PII)** are properly managed and governed. It provides fast discovery and de-identification of PII columns, replacing their contents for entities such as person names, locations, social security numbers, phone numbers, financial/health data and more.

Below, you will find the most important features widely used by customers, which we will cover in this guide:

1. Use the **PII scanner** to identify sensitive columns.
2. De-identify PII using **Mockers** or **Exclude**.
3. Deploy **consistent mapping** with Syntho mockers.
4. Use the newest feature **calculated columns** to perform a wide range of operations on data.
5. Utilize the **foreign key scanner** to inherit foreign keys from the database.
6. Use the **Sync** button to sync the source schema with the workspace.

{% embed url="<https://youtu.be/xmR1ycrEDx4>" %}
What is covered in the database de-identification video guides?
{% endembed %}

***

## [Prerequisites](/overview/get-started/prerequisites)

For prerequisites check [Prerequisites](/overview/get-started/prerequisites) or watch the video below.

{% embed url="<https://youtu.be/dubPW24-4Jk>" %}
Prerequisites for De-identification
{% endembed %}

***

## Healthcare Database

Let’s assume the customer operates in the healthcare industry. The customer's database comprises medical data about their patients, medications, supplies, devices, etc. The below screenshot displays all tables residing in the database on the left side and a sample of rows in the **patients** table:

<figure><img src="/files/HbAydWrq4gWJZz49ao4D" alt=""><figcaption><p>All tables (on the left) and contents of patients table (in the middle)</p></figcaption></figure>

{% embed url="<https://youtu.be/bNgZnacMfFM>" %}
Setup a workspace
{% endembed %}

## [Workspace Default Settings menu](/setup-workspaces/workspace-default-settings)

You can **de-identify** by transforming column data to remove or mock PII via two options: **Mockers** and **Exclude**. The default column mode is **Duplicate**, meaning the column is copied directly without alteration. However, this setting can be changed to either mock data with **Mockers** or exclude specific columns.

1. **Create** or **open** **the workspace** with the columns that you want to de-identify.
2. To preserve all cross-table relationships, hold `CTRL + SHIFT + ALT + 0` to open the Workspace Default Settings menu. If this short key is reserved on your system, you can add `/workspace_settings` to the end of the workspace URL.
3. Under the `key_generation_method` entry, set the value either to:
   * “**duplicate**“: to preserve cross-table relationships and duplicate the original key values.
   * “**hash**“: to preserve cross-table relationships and hash the original key values.
4. On the **Job configuration** panel use `CTRL` or `SHIFT` to select multiple tables simultaneously.
5. Access the **column settings** for the selected table.
6. By default, the column mode is set to **Duplicate**.
7. Change the column mode to one of the following options:
   * **Mocker**: Use this option to fill the columns with mock data.
   * **Exclude**: Choose this option if you don’t want to include specific columns in the duplicated table.

Using these modes, you can safely de-identify PII by either replacing it with mock data (**Mockers**) or excluding it (**Exclude**) from the target database (for more information, see [Configure column settings](/configure-a-data-generation-job/configure-column-settings)).

For clarity, the below illustration shows the distinction between table and column configurations.

<figure><img src="/files/fSkqbL76unq2JV7LZnkx" alt=""><figcaption><p>Table and column configurations</p></figcaption></figure>

{% embed url="<https://youtu.be/6SBN-nqejEM>" %}
How to De-identify
{% endembed %}

## [Discovering and De-identifying columns](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner)

In the **PII tab**, you can add new columns to the list of **PII** columns, either **manually** or by using Syntho's **PII scanner**. You have the option to **manually label columns** containing PII by selecting the column name and optionally choosing a mocker to apply. Clicking "**Confirm**" will mark the column as containing PII and confirm the mocker selection.

<figure><img src="/files/W1g4sq0iZjSlhc7rSUGu" alt=""><figcaption><p>Selecting a mocker manually under generation method</p></figcaption></figure>

Alternatively, deploy automatic PII discovery with the PII scanner. Launch a scan to detect PII across all database columns from the PII tab in the **Job Configuration** panel. Note that the scanner offers both **Shallow** and **Deep** scan modes:

* The **shallow scan** assesses columns using regular expression rules to identify PII, optimized for speed but with variable accuracy.
* The **deep scan** examines both metadata and data within columns for a thorough PII identification.

Following a **deep scan**, Syntho may reveal columns likely containing PII, assigning a probability score to each (e.g., 80% for the "**ADDRESS**" column).

To delete unwanted configurations, click the delete icon on the right side of the panel.

Clicking "**Configure**" opens a new window for column settings, detailed in the subsequent section.

<figure><img src="/files/5Z3bk3pt3CHjtMCV3mMn" alt=""><figcaption><p>PII tab</p></figcaption></figure>

{% embed url="<https://youtu.be/uwxcCTb85i0>" %}
Discovering and De-identifying PII columns
{% endembed %}

For more information, please see - [Automatic PII discovery with PII scanner](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner).

## De-identify using [mockers](/configure-a-data-generation-job/configure-column-settings/mockers) & [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping)

After clicking the "**Configure**" button on PII tab, the window shown below will appear.

<figure><img src="/files/bUeONIPndlFlLUuUK2Jg" alt=""><figcaption><p>Column settings</p></figcaption></figure>

The above window can **also** be reached by clicking the column settings for the selected table. Please see below to find how to open the window.

<figure><img src="/files/8i1RfDZa0fpxGofguB25" alt=""><figcaption><p>Alternative way to open column settings</p></figcaption></figure>

For columns not identified as containing PII, such as the "**COUNTRY**" column, the default mode applied is **Duplicate**, meaning it can be safely duplicated to the **destination** database. However, for columns detected as containing PII, like "**NAME**", you can apply a **Mocker**.

To configure settings for the "**NAME**" column (as shown in the previous illustration), we opt for mock data over real names. The data type "**Name**" is automatically selected and we can also choose the "**unique**" option to ensure only unique values are generated.

**Consistent mapping**, an advanced feature, generates identical mock data for each set of original values every time it's applied. For instance, the mock name "Jack" will replace "Alan" consistently, **maintaining value consistency across** tables, databases, and jobs.

By clicking the "**Preview**" button, you can view a preview of the mock data with the defined settings.

<figure><img src="/files/umeDw1HB9wbnR2DjhSpW" alt=""><figcaption><p>Previewing mock values</p></figcaption></figure>

{% embed url="<https://youtu.be/s1PCO6HmNWM>" %}
De-identify using Mockers and Consistent Mapping
{% endembed %}

For more information, please check [mockers](/configure-a-data-generation-job/configure-column-settings/mockers) & [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).

### Truncate tables before each new data generation job

Users are required to manually **TRUNCATE** their tables in the **DESTINATION** database before initiating each new data generation job. If truncation is hindered due to existing constraints, these constraints should be temporarily disabled before truncation and then re-enabled afterwards. For instance, to facilitate the truncation process when foreign key constraints prevent it, use the following SQL commands: First, disable the constraints by executing `SET FOREIGN_KEY_CHECKS = 0;`, then **TRUNCATE** the table, and finally, re-enable the constraints with `SET FOREIGN_KEY_CHECKS = 1;`. This sequence ensures that tables are properly prepared for data generation without constraint violations.

## De-identify using [Calculated Columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns)

{% hint style="info" %}
**This feature is planned for release and not part of the Syntho platform yet. The calculated column function list will be rolled out in a phased approach.**

**Please contact your Syntho contact person if you have suggestions for this feature.**
{% endhint %}

Another example is the first name mocker. Imagine having a table with a column for first names. However, the user wants to generate male mock data for male names and female mock data for female names based on checking their gender in the gender column. This request can be expressed using the formula below:

```excel-formula
IF([Gender] = 'M', MOCK_FIRST_NAME, IF([Gender] = 'F', MOCK_LAST_NAME_FEMALE, 'nothing'))
```

<div align="left"><figure><img src="/files/EfyNJQxA6uZJKy2G93D2" alt="" width="563"><figcaption><p>Calculated column and its formula field</p></figcaption></figure></div>

For more information, please see - [Calculated columns](#calculated-columns).

## [Verify ](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys/use-foreign-key-scanner)or [Add Foreign Keys](/configure-a-data-generation-job/manage-foreign-keys/add-virtual-foreign-keys)

The **Foreign Key tab**, adjacent to the PII tab, shows Syntho's automatic inheritance of foreign keys from your **source** database. If not explicitly defined, you can add them through **import** (JSON files), manually or scanning.

<figure><img src="/files/duMK2kDFc8nqtMR57fZx" alt=""><figcaption><p>Foreign key tab</p></figcaption></figure>

For databases without explicit foreign key relationships, Syntho allows you to add virtual foreign keys manually. To do this, select the tables and columns for the foreign and primary keys under the **Foreign Keys** tab and click on "**Add foreign key**" to finalize.

To streamline setup, you can **import** foreign keys through a **JSON** file. Just click "**Upload foreign keys**", use the **Browse** button to select your file, and click "**Import**" to update your **Foreign Keys** list.

Syntho also offers a **foreign key scanner** for discovering potential virtual foreign keys, useful in large databases. To initiate a scan, go to the **Foreign Keys** tab, press "**Scan**," apply filters if needed, and confirm to start. You can then review, confirm, or delete any identified foreign key candidates.

{% embed url="<https://youtu.be/2xZa6qralAY>" %}
Verify and add foreign keys
{% endembed %}

For more information, please see [Manage foreign keys](/configure-a-data-generation-job/manage-foreign-keys).

## Keep your source database in [sync ](https://docs.syntho.ai/configure-a-data-generation-job/validate-and-synchronize-workspace#source-schema-synchronization)with your workspace

The **Sync** button is helpful for reflecting frequent schema changes in Syntho. It ensures the workspace mirrors the current state of the **source** database, accommodating additions, deletions, and modifications to the **source** database.

Let’s assume we have a **source** database called healthcare and the column “**Drivers**” was removed from the table **patients** in the **source MySQL database**. After removal of the column, when you press **Sync** button, it will show the current version of the **source** database.

<figure><img src="/files/iFf4nuLpbNU2IudITWcz" alt=""><figcaption><p>Schema changes can be reflected immediately in Syntho with sync button</p></figcaption></figure>


# Frequently asked questions

### Deployment

#### What permissions do I need to deploy Syntho?

Syntho requires an administrative user for installation. If you use **Docker** for the deployment, the user needs **sudo** rights for **docker/docker-compose**.

### Databases and generation jobs

#### How should I provision my databases for data generation?

The Syntho platform works with source and destination databases.

* The **source database** stores the input data for your data generation job and can be **read-only**.
* The **destination database** must have empty tables and columns, which are structurally identical to the **source** database. It requires **write access,** as the generated data will be written to it.

The **source** and **destination** databases can run on the same server, or be hosted on different servers.

#### How does Syntho handle constraints on my database?

Syntho uses relevant metadata stored in your **source** database for generating the data that is written into the **destination** database. For example, foreign key constraints are inferred to reproduce table relationships and referential integrity in the generated database.

Syntho is capable of handling scenarios where the constraints of the destination database are either enabled or disabled.

#### I have non-sensitive tables that I still want in my destination database. What should I do?

Non-sensitive tables (for example, definition tables such as language or product tables) can be copied as-is when writing to the destination database.

Use the **Duplicate** generator for the relevant columns.

For more information, see [Table view](/configure-a-data-generation-job/configure-table-settings#table-modes).

#### I do not have any data yet. Can I generate data from scratch?

Yes. If you do not have any data yet, you can generate data from scratch to fill an empty database.

You can do this in the Syntho platform as follows:

1. First, set up an empty **source** database and **destination** database with the tables and columns that you want to fill. The source database should be different from the destination database.
2. Ensure primary and foreign key columns in your **source** database are correctly configured and already have **some key values present**.
3. On the **Job configuration** panel, drag all relevant tables under **Include**.
4. Use the shortcut `CTRL + SHIFT + ALT + 0` and set the value under the entry "**key\_generation\_method**" to "**generate**".
5. For each column, go to **Column settings** and select [Mock](/configure-a-data-generation-job/configure-column-settings/mockers) as the generator.\
   Note: [Calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns) are currently not supported for generating data from scratch.
6. Configure the relevant **Mock** settings.
7. Select the **number of rows** to generate under the Table settings.
8. Finally, select **Generate** to generate your database.

#### Can I use Syntho for data de-identification, masking, or pseudonymization?

Yes, Syntho also allows you to mask or de-identify your most sensitive columns. The following section describes how this can be done:

[Database de-identification](/overview/get-started/database-de-identification)

### Performance and scaling

#### How can I speed up my data generation jobs?

The Syntho platform and its generative models are optimized to generate data fast and efficiently.

For example, task parallelization is applied throughout the pipeline, from reading source data to writing destination data.

Here are some additional tips to boost the speed of your data generation jobs:

1. [Exclude](/configure-a-data-generation-job/configure-column-settings/exclude) large tables from the generation process.
2. [Reduce the number of generated rows](/configure-a-data-generation-job/configure-table-settings#adjust-the-number-of-rows-to-generate) for large tables.
   * This only works if the table is not referenced by other tables.
3. For columns with [AI synthesize](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation) enabled:
   * Adjust the [advanced settings](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#advanced-settings).
   * Reduce the number of high-cardinality columns.
4. Disable [PII obfuscation](/configure-a-data-generation-job/configure-column-settings/duplicate/automatic-pii-discovery-and-de-identification-in-free-text-columns) for columns containing free text.
5. In some database types, **write performance** can become a bottleneck when working with larger tables. Syntho offers several options to optimize write performance:
   1. **Use parallel writing** by increasing the [**Number of simultaneous connections**](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings#advanced-generation-settings) to allow parallel writing across multiple tables.
   2. [Bypassing database checks](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings#data-generation-capabilities) (foreign key constraints, indexes, and identity/auto-increment behavior) can significantly reduce write time. This is especially effective with a [higher maximum number of connections](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings#advanced-generation-settings).
   3. Increasing the [batch size](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings#advanced-generation-settings) may modestly improve write speed.
   4. Write generated data directly to Parquet files in [Azure Data Lake Storage (ADLS)](/setup-workspaces/create-a-workspace/connect-to-a-database/azure-data-lake-storage-adls) or [Amazon Simple Storage Service (S3)](/setup-workspaces/create-a-workspace/connect-to-a-database/amazon-simple-storage-service-s3) to skip database checks.
   5. Consider synthesizing a representative subset of tables instead of the full dataset. You can prepare these subsets in your source database as *(MATERIALIZED) VIEWS*, which simplifies preprocessing and reduces storage costs. Syntho supports reading and processing views directly.

#### How can I optimize memory utilization of my cluster?

The Syntho platform offers several features to optimize memory utilization during data generation. Understanding how these features work can help you manage memory more effectively and prevent issues such as out-of-memory (OOM) errors.

1. [Batch size](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings): The number of data points processed in a single batch. Higher values can speed up generation, but increase memory usage.
2. [Number of simultaneous connections](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings#advanced-generation-settings) **(N)**: The number of tables that can be read or written in parallel. Higher values speed up generation, but increase memory usage.
3. [Maximum rows used for training](/configure-a-data-generation-job/configure-table-settings#advanced-table-settings) **(N)**: The number of source rows used to train the model. More rows can improve utility, but increase memory usage.
4. [Ray dashboard](/deploy-syntho/logs-and-monitoring): Monitor memory usage, CPU utilization, and other runtime metrics.
5. [OOM error logs](/deploy-syntho/logs-and-monitoring): Errors like `Workers (tasks/actors) killed due to memory pressure (OOM)` mean the cluster ran out of memory. Reduce batch size, reduce parallel connections, or lower training rows.

### Data handling

#### Does Syntho collect or store any data?

Syntho does not collect or store any data.

Syntho runs in the customer’s environment, and Syntho cannot access customer data.

For more information, see [Does Syntho collect any data?](/deploy-syntho/logs-and-monitoring/does-syntho-collect-any-data).

### AI synthesis

#### Can real data be found in AI-generated synthetic data?

No. Synthetic data does not directly replicate the real data it is based on.

It uses learned patterns and statistical properties to generate new records, independent of the source dataset.

[Privacy control mechanisms](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation/privacy-controls) such as [rare category protection](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#rare-category-protection) and [clipping thresholds](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#advanced-column-settings) further reduce the risk of exposing unique values or outliers.

#### Are there scenarios where similarities might appear between synthetic and real data?

In rare cases, synthetic data may contain values that match those in the original dataset. For example, if several individuals share identical characteristics in the real data, those attributes might also appear in the synthetic dataset. However, this does not pose a privacy risk, as these instances are not linked to specific individuals. Methods like K-anonymity are used to minimize such risks.

#### What measures are in place to prevent exposure of sensitive data?

Syntho implements various privacy-preserving techniques.

See [Additional privacy controls](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation/privacy-controls) for details.

#### How many training records do I need for AI synthesis?

To minimize privacy risks and improve the training algorithm's generalization ability, as a rule of thumb, a minimum **column-to-row ratio of 1:500** is recommended. For example, if your **source** table has 6 columns, it should contain a minimum of 3000 rows.

#### Does AI synthesis work for multiple tables?

No, AI synthesis only works well on single tables.

For best utility with minimal resources, [prepare your data as a single entity table](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#entity-tables).

If you need to anonymize multiple related tables while keeping consistency across tables, Syntho’s other [generators](/configure-a-data-generation-job/configure-column-settings) can be used.

#### With AI synthesis, what can I do to improve synthetic data utility?

There are several ways to improve the utility (a.k.a. 'quality' or 'fidelity') of the generated synthetic data. A few options:

1. Ensure your dataset is prepared as a single entity table. Follow [Preparing your data](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#preparing-your-data).
2. Increase [Maximum rows used for training](/configure-a-data-generation-job/configure-table-settings#advanced-table-settings). Consider using all rows.
   * If you limit training rows, enable [Take random sample](/configure-a-data-generation-job/configure-table-settings#advanced-table-settings) for a more representative subset.
3. For advanced users, you may consider tweaking [Workspace default settings](/setup-workspaces/workspace-default-settings#ai-synthesis-privacy).


# View workspaces

To view all workspaces you have access to, you can either:

* Select **Open an existing workspace** on the **Launch your workspace** panel,
* Select **Saved Workspaces** > **More...** on the left navigation panel.

{% embed url="<https://www.guidejar.com/guides/f9e85458-2642-49f4-8446-86594152c4b0>" %}

## Workspace details

The workspaces view shows the following details for each workspace:

* **Name**: The name of the workspace.
* **Data source**: The data source connector.
* **Status**: The status of the last job that was started using this workspace.
* **Your role:** Your role in relation to the workspace.
* **Owner**: The user who owns this workspace.
* **Created at**: The moment this workspace was created.
* **Actions menu:** Actions that can performed for the workspace. These include:
  * [Edit a workspace](/setup-workspaces/edit-a-workspace)
  * [Duplicate a workspace](/setup-workspaces/duplicate-a-workspace)
  * [Transfer ownership of a workspace](/setup-workspaces/transfer-workspace-ownership)
  * [Share a workspace](/setup-workspaces/share-a-workspace)
  * [Delete a workspace](/setup-workspaces/delete-a-workspace)

## What can you do

Besides viewing your workspaces, on the workspaces panel you can:

<table data-card-size="large" data-view="cards"><thead><tr><th></th><th data-hidden></th><th data-hidden></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Create a workspace</strong></td><td></td><td></td><td><a href="/pages/sKmBJproaXx52pm76j2t">/pages/sKmBJproaXx52pm76j2t</a></td></tr><tr><td><strong>Edit a workspace</strong></td><td></td><td></td><td><a href="https://github.com/shahin-hu/Translations/blob/main/English/setup-workspaces/broken-reference/README.md">https://github.com/shahin-hu/Translations/blob/main/English/setup-workspaces/broken-reference/README.md</a></td></tr><tr><td><strong>Delete a workspace</strong></td><td></td><td></td><td><a href="/pages/XPx5r0qCd5tvNJcAlKnc">/pages/XPx5r0qCd5tvNJcAlKnc</a></td></tr><tr><td><strong>Share a workspace</strong></td><td></td><td></td><td><a href="/pages/cCOwn6fCP1jvaNvioHHZ">/pages/cCOwn6fCP1jvaNvioHHZ</a></td></tr><tr><td><strong>Transfer ownership of a workspace</strong></td><td></td><td></td><td><a href="/pages/lB6fKMo7dmko57tegHkr">/pages/lB6fKMo7dmko57tegHkr</a></td></tr></tbody></table>


# Create a workspace

Create a new workspace using either options:

* On the **Launch your workspace** screen, click **Connect to a database**,
* In the left toolbar, click on **Create workspace**.

{% embed url="<https://www.guidejar.com/guides/d27734b5-703a-4aa7-9db8-f040717bb28a>" %}

**Note:** When creating a new workspace, the **source** and the **destination** database should be of the same database type. The **source** and **destination** database can be hosted on the same or on a different server.


# Connect to a database

## Define the source and destination database

Before you can synthesize your data, you must first create a workspace by connecting to a **source** and **destination** database. The **source** database contains the data that you want to synthesize. The **destination** database is the location where you want to write the synthetic data to.

Syntho supports connecting to a variety of different databases.

### Supported databases

<table data-view="cards"><thead><tr><th></th><th align="center"></th><th></th><th data-hidden data-card-target data-type="content-ref"></th><th data-hidden data-card-cover data-type="files"></th></tr></thead><tbody><tr><td></td><td align="center"><img src="/files/CNuH9ZhC4cXEE7O4SGZI" alt="" data-size="original"></td><td></td><td><a href="/pages/1AgoQHNXkEOFZHDZYoAM">/pages/1AgoQHNXkEOFZHDZYoAM</a></td><td></td></tr><tr><td></td><td align="center"><img src="/files/9InHAycy9uKNoCJmQUqt" alt="" data-size="original"></td><td></td><td><a href="/pages/pznUwXwGX0laz6Z8Vm28">/pages/pznUwXwGX0laz6Z8Vm28</a></td><td></td></tr><tr><td></td><td align="center"><img src="/files/X4UT7sCbuhaR9DWAn5Ow" alt=""></td><td></td><td><a href="https://github.com/shahin-hu/Translations/blob/main/English/setup-workspaces/create-a-workspace/connect-to-a-database/broken-reference/README.md">https://github.com/shahin-hu/Translations/blob/main/English/setup-workspaces/create-a-workspace/connect-to-a-database/broken-reference/README.md</a></td><td></td></tr><tr><td></td><td align="center"><img src="/files/dgFl0Em2sgyAYdTNHE6B" alt=""></td><td></td><td><a href="/pages/RynPf8Bl1w8VZOUWc3La">/pages/RynPf8Bl1w8VZOUWc3La</a></td><td></td></tr><tr><td></td><td align="center"><img src="/files/zHAzcsKmA8Lc8N9PV4Vt" alt=""></td><td></td><td><a href="/pages/xkIy1PQOBX7okiF6Qit0">/pages/xkIy1PQOBX7okiF6Qit0</a></td><td></td></tr><tr><td></td><td align="center"><img src="/files/6cP5UOewt66pJ90SddtY" alt=""></td><td></td><td><a href="/pages/Vxaa98oEtqQ7bbb2F8Ut">/pages/Vxaa98oEtqQ7bbb2F8Ut</a></td><td></td></tr><tr><td></td><td align="center"><img src="/files/ZAoG2FClzZ54bTViSWcz" alt=""></td><td></td><td><a href="/pages/tVqZES0tnUCTNmri20wx">/pages/tVqZES0tnUCTNmri20wx</a></td><td></td></tr></tbody></table>

### Supported filesystems (destination only)

<table data-view="cards"><thead><tr><th align="center"></th><th align="center"></th><th align="center"></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td align="center"></td><td align="center"><img src="/files/DjcjcG0N9R423RsCYsuz" alt=""></td><td align="center"></td><td><a href="/pages/tdYBoBDngVGXePMYIAu0">/pages/tdYBoBDngVGXePMYIAu0</a></td></tr><tr><td align="center"></td><td align="center"><img src="/files/Tgmql4IRatHvmMZOhtPs" alt="" data-size="original"></td><td align="center"></td><td><a href="/pages/ke8Vqm2KENKJdJ6HvE97">/pages/ke8Vqm2KENKJdJ6HvE97</a></td></tr></tbody></table>

## Batch size options

Syntho processes data in batches. **Batch size** controls the maximum datapoints Syntho reads, generates, and writes per batch. Lower values reduce memory usage for large databases. To change this setting, see [View and adjust generation settings](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings#advanced-generation-settings).

## Request a new connector

If Syntho doesn't have a native (built-in) connector for your data, you can request it with your Syntho contact person. Syntho regularly reviews requests by customers to help determine and prioritize what features should be added to the product.

## Connector considerations & limitations

This section outlines the general considerations and limitations associated with various connectors. For specific considerations or limitations of a particular connector, visit its respective page.

* All connectors, except for Microsoft SQL Server, write special Unicode characters as-is (e.g., `De\u200bC → De\u200bC`).
* For Microsoft SQL Server, behavior depends on the driver:
  * ODBC Driver: Interprets and removes special Unicode characters (`De\u200bC → DeC`).
  * FreeTDS Driver: Cannot interpret special characters, replacing them with `?` (`De\u200bC → De?C`).


# Databricks

<div data-full-width="false"><figure><img src="/files/lPr3Rv5I646TuSQetA8h" alt="" width="375"><figcaption></figcaption></figure></div>

Source and Destination Databases

{% hint style="info" %}
**Important**

This connector can only be used as a source database. The generated data can be written to Local [Filesystem](/setup-workspaces/create-a-workspace/connect-to-a-database/local-filesystem), [Azure Data Lake Storage (ADLS)](/setup-workspaces/create-a-workspace/connect-to-a-database/azure-data-lake-storage-adls) or [Amazon Simple Storage Service (S3)](/setup-workspaces/create-a-workspace/connect-to-a-database/amazon-simple-storage-service-s3) as Parquet files.
{% endhint %}

## Before you begin

Before you begin, gather this connection information:

* Name of the server that hosts the database you want to connect to and port number
* The name of the database that you want to connect to
* HTTP path to the data source
* Personal Access Token
* In Databricks, find your cluster server hostname and HTTP path using the instructions in [Construct the JDBC URL](https://docs.databricks.com/user-guide/bi/jdbc-odbc-bi.html#http-path) on the Databricks website.

If you first need to load data into Databricks, see [Importing Data into Databricks](/setup-workspaces/create-a-workspace/connect-to-a-database/databricks/importing-data-into-databricks).

## Connect and set up the workspace

1. Launch **Syntho** and select **Connect to a database** (or **Create workspace**).
2. Under **The connection details**, choose **Databricks** from the **Type** dropdown.
3. Fill in the required fields:
   * **Server hostname** → e.g. `adb-1111111111111111.0.azuredatabricks.net`
   * **Catalog name** → e.g. `demo_catalog`
   * **Database name** → e.g. `marketing_db`
   * **HTTP Path** → e.g. `sql/protocolv1/o/1234567890123456/0000-111111-demo123`
   * **Port number** → default is `443`
   * **Personal Access Token** → (See [Personal Access Tokens](https://docs.databricks.com/aws/en/dev-tools/auth/pat) on the **Databricks** website for information on access tokens.)
   * **Warehouse ID (optional)** → the SQL Warehouse to query through (recommended). Use this when possible. It is usually faster and more stable for large databases. If omitted, Syntho falls back to JDBC retrieval via the Spark driver.
4. Click **Create Workspace** to complete the setup.\
   If Syntho can't make the connection, verify that your credentials are correct. If issues persist, your computer may not be able to locate the server. Contact your **network administrator** or **database administrator** for support.

## Supported Databricks versions

The table below provides an overview of the supported Databricks versions and their corresponding Apache Spark versions.

| Databricks Version | Spark Version |
| ------------------ | ------------- |
| 16.2               | 3.5.0         |
| 15.4 LTS           | 3.5.0         |
| 14.3 LTS           | 3.5.0         |

**Note:** Version 13 is no longer supported.

## Supported data types

The following table summarizes the current support limitations for various data types when using connectors with **Databricks**. It indicates what's supported per generator type.

| Data Type      | AI-powered Generation                                                                                | Mockers                                                                                    | Mask                                                                                    | Calculated Columns                                                                                         |
| -------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| INT            | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| SMALLINT       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| TINYINT        | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| BIGINT         | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| DECIMAL        | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| FLOAT          | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| DOUBLE         | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| STRING         | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| BINARY         | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                     |
| BOOLEAN        | False                                                                                                | [True](/configure-a-data-generation-job/configure-column-settings/mockers/other)           | False                                                                                   | True\*                                                                                                     |
| DATE           | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      |
| TIMESTAMP      | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      |
| TIMESTAMP\_NTZ | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      |
| ARRAY          | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                     |
| STRUCT         | False                                                                                                | [True](/configure-a-data-generation-job/configure-column-settings/mockers/other#json)      | False                                                                                   | False                                                                                                      |
| MAP            | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                     |
| VARIANT        | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                     |
| OBJECT         | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                     |
| ENUM           | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      |

{% hint style="info" %}
**\*** Some data types are not actively supported. Some generators may still show `True` for these fields. This means you can apply the generator, even though the type is not actively supported. Duplication is fully supported for these data types.
{% endhint %}

## Limitations

* When entering catalog, database, or schema names, use lowercase letters. Names containing capital letters must be entered in lowercase to ensure a proper connection.
* Schema, table, and column names containing single quotes (`'`) or backticks (\`) are not supported.


# Importing Data into Databricks

Once your synthetic data is written as Parquet files to a storage location (Local Filesystem, Azure Data Lake Storage (ADLS), or Amazon S3), follow these steps to import it back into Databricks:

1. **Access Databricks workspace**: Go to your Databricks workspace and navigate to the Data tab.<br>
2. **Select your data source**:
   * For ADLS, select Azure Data Lake.
   * For Amazon S3, choose Amazon S3.
   * If using the Local Filesystem, upload your files to a cloud storage service like ADLS or S3 first.<br>
3. **Mount the storage**: Mount your cloud storage (ADLS or S3) to Databricks following the [Databricks mounting documentation](https://docs.databricks.com/data/data-sources/azure/azure-datalake-gen2.html) for ADLS or [S3](https://docs.databricks.com/data/data-sources/aws/amazon-s3.html).<br>
4. **Read the parquet files**: Use the Databricks Data tab or a notebook to load Parquet files into a DataFrame. For details, check the [Databricks guide on reading files](https://docs.databricks.com/data/data-sources/read-parquet.html).<br>
5. **Create or register a table**: Use Databricks SQL commands or the user interface to create a temporary or permanent table from the loaded data. Parquet files generated by Syntho can be loaded into Databricks using standard Databricks SQL commands. For example:

```sql
CREATE TABLE example_table
USING PARQUET
LOCATION 'dbfs:/FileStore/tables/example.parquet';
```

This command creates a table from the specified Parquet file. Refer to the [Databricks documentation](https://docs.databricks.com/data/tables.html) for more information on managing tables.


# DB2 for LUW

<figure><img src="/files/j06xxHiwqhmt62iw6fml" alt=""><figcaption><p>Source and Destination Databases</p></figcaption></figure>

## Before you begin <a href="#before-you-begin" id="before-you-begin"></a>

Before you begin, gather this connection information:

* Name of the server that hosts the database you want to connect to (Host name) and port number
* Database name
* User name (User ID) and password
* Are you connecting to an SSL server?

### Port settings <a href="#port-settings" id="port-settings"></a>

The port number is dependent on the type of server you are connecting to and whether you are connecting to an encrypted port. Generally, use 50000 for a non-encrypted port and 60000 for an encrypted port. It is possible that your server is configured to use a non-standard port. Contact your administrator if you don't know which port to connect to.

### SSL requirements <a href="#ssl-requirements" id="ssl-requirements"></a>

This connector enables you to connect to your server using SSL authentication. If you work in an SSL environment, your computer is probably already configured to support SSL. If you have trouble making an SSL connection with **Tableau**, make sure that the following IBM software is installed on your computer:

* IBM Data Server Runtime Client v10.5fp3 or above
* GSK8 SSL Library

Note that the IBM Data Server Runtime Client and GSK8 Library must have the same bitness. For example, both must be either 32-bit or 64-bit. For more information, see [Installation of the GSK8 SSL Library(Link opens in a new window)](https://www.ibm.com/support/knowledgecenter/SSEPGG_11.1.0/com.ibm.db2.luw.admin.sec.doc/doc/t0053518.html) on the IBM website.

## Connect and set up the workspace

Launch Syntho and select **Connect to a database**, or **Create workspace**. Then select **DB2** from **Type** under **The connection details**. For a complete list of data connections, click **Type** under **The connection details**. Then do the following:

1. Enter the name of the server that hosts the database that you want to connect to.
2. Enter the database name to connect to the contained database.
3. Enter the port number.
4. Enter user name and password.

   Select the **Require SSL** check box when connecting to an SSL server.
5. To connect to one schema only, enter its name in **Schema**.
6. Enable **Include views** to include views in the connection.
7. Select **Next**.\
   If Syntho can't make the connection, verify that your credentials are correct. If you still can't connect, your computer is having trouble locating the server. Contact your network administrator or database administrator.

{% hint style="warning" %}
When source and destination use the same database, select different schemas. Ensure no foreign keys span these schemas. Cross-schema foreign keys can cause data-generation issues.
{% endhint %}

## Limitations

* Batch reading and writing can be unstable and not as efficient as for other databases, but it will work in most cases.
* IBM DB2 only supported for LUW (Linux, Unix, and Windows)
* When entering database or schema names, use lowercase letters. Names containing capital letters must be entered in lowercase to ensure a proper connection.

### Supported data types

| Data Type  | AI-powered Generation                                                                                | Mockers                                                                                    | Mask                                                                                    | Calculated Columns                                                                                         | Other |
| ---------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | ----- |
| INTEGER    | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -     |
| SMALLINT   | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -     |
| BIGINT     | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -     |
| DECIMAL    | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -     |
| NUMERIC    | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -     |
| FLOAT      | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -     |
| REAL       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -     |
| DOUBLE     | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -     |
| DECFLOAT   | True\*                                                                                               | False                                                                                      | False                                                                                   | False                                                                                                      | -     |
| CHAR       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -     |
| VARCHAR    | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -     |
| CLOB       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -     |
| BINARY     | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      | -     |
| VARBINARY  | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      | -     |
| BLOB       | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      |       |
| DATE       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      | -     |
| TIME       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      | -     |
| TIMESTAMP  | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      | -     |
| BOOLEAN    | False                                                                                                | [True](/configure-a-data-generation-job/configure-column-settings/mockers/other)           | False                                                                                   | False                                                                                                      | -     |
| GRAPHIC    | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -     |
| VARGRAPHIC | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -     |
| DBCLOB     | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -     |
| XML        | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -     |

{% hint style="info" %}
**\***&#x53;ome data types are not actively supported; however, certain generators such as AI synthesize, mask, mockers, or calculated columns may still show 'True' for these fields. This means the generators can be applied, even though the types are not actively supported. Duplication is fully supported for these data types.
{% endhint %}


# Hive

{% hint style="info" %}
**Syntho Beta feature**
{% endhint %}

<figure><img src="/files/0a14uPJtfHBNoZwtXp4e" alt=""><figcaption><p>Source and Destination Databases</p></figcaption></figure>

{% hint style="info" %}
**Important**

This connector can only be used as a source database. The generated data can be written to [Azure Data Lake Storage (ADLS)](/setup-workspaces/create-a-workspace/connect-to-a-database/azure-data-lake-storage-adls) or [Amazon Simple Storage Service (S3)](/setup-workspaces/create-a-workspace/connect-to-a-database/amazon-simple-storage-service-s3) as Parquet files.
{% endhint %}

## Before you begin

Before you begin, gather this connection information:

* Name of the server that hosts the database you want to connect to and port number
* User name and password
* Are you connecting to an SSL server?

## Connect and set up the workspace

Launch Syntho and select **Connect to a database**, or **Create workspace**. Then select **Hive** from **Type** under **The connection details**. For a complete list of data connections, click **Type** under **The connection details**. Then do the following:

1. Enter the name of the server that hosts the database and the port number to use.
2. Optionally, enter the schema name.
3. Enter user name and password.

   Select the **Require SSL** check box when connecting to an SSL server.
4. Select **Create Workspace**.

   If Syntho can't make the connection, verify that your credentials are correct. If you still can't connect, your computer is having trouble locating the server. Contact your network administrator or database administrator.

## **Considerations: Handling Hive database partitioning**

{% hint style="info" %}
Hive only
{% endhint %}

In Hive databases, source tables are often partitioned based on three columns treated as index columns. These columns are used for ordering in queries, but they do not always form unique composites.

To address this, use the partitioning columns along with the additional columns specified through the "ORDER BY" dropdown. This approach ensures unique and consistent ordering, leveraging both the partitioning logic and user-defined columns. For more information, check [Table view](/configure-a-data-generation-job/configure-table-settings#order-by) in table settings.

## Supported data types

| Data Type | AI-powered Generation                                                                                | Mockers                                                                                    | Mask                                                                                    | Calculated Columns                                                                                         |
| --------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| TINYINT   | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| SMALLINT  | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| INT       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| BIGINT    | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| FLOAT     | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| DOUBLE    | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| DECIMAL   | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| TIMESTAMP | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      |
| DATE      | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      |
| STRING    | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| VARCHAR   | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| CHAR      | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| BOOLEAN   | False                                                                                                | [True](/configure-a-data-generation-job/configure-column-settings/mockers/other)           | False                                                                                   | False                                                                                                      |
| BINARY    | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      |
| ARRAY     | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| MAP       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| STRUCT    | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |

{% hint style="info" %}
**\***&#x53;ome data types are not actively supported; however, certain generators such as AI synthesize, mask, mockers, or calculated columns may still show 'True' for these fields. This means the generators can be applied, even though the types are not actively supported. Duplication is fully supported for these data types.
{% endhint %}


# Microsoft SQL Server

<figure><img src="/files/ZrhoPnQg4arFQHrGxRHm" alt=""><figcaption><p>Source and Destination Databases</p></figcaption></figure>

## Before you begin

Before you begin, gather this connection information:

* Name of the server that hosts the database you want to connect to and port number
* User name and password
* Are you connecting to an SSL server?
* Do you want to set the database isolation level to read uncommitted data?

## Connect and set up the workspace

Launch Syntho and select **Connect to a database**, or **Create workspace**. Then select **Microsoft SQL Server** from **Type** under **The connection details**. For a complete list of data connections, click **Type** under **The connection details**. Then do the following:

1. Enter the name of the server that hosts the database that you want to connect to.
2. Enter the database name to connect to the contained database.
3. Enter the port number.
4. Enter user name and password.

   Select the **Require SSL** check box when connecting to an SSL server.
5. To connect to one schema only, enter its name in **Schema**.
6. Enable **Include views** to include views in the connection.
7. Select **Next**.\
   If Syntho can't make the connection, verify that your credentials are correct. If you still can't connect, your computer is having trouble locating the server. Contact your network administrator or database administrator.

{% hint style="warning" %}
When source and destination use the same database, select different schemas. Ensure no foreign keys span these schemas. Cross-schema foreign keys can cause data-generation issues.
{% endhint %}

## Considerations & limitations

* **Handling sensitive data in encoded images**: The `VARBINARY` data type may be used to store encoded images, which can potentially include sensitive information. Due to the complexity of replicating synthetic twins of such images, our product, Syntho, replaces values stored in `VARBINARY` columns with a dummy character to ensure data privacy and integrity. This approach prevents the accidental leakage of sensitive data.
* **Identity columns in table names with dots**: Table names with dots in their names are not supported when such tables contain `identity` columns. This limitation is due to the way the driver interprets table names with dots. These tables will have to be completely excluded from processing.
* **SQL variant data type**: Database columns with the data type `sql_variant` cannot use the key generation method "Duplicate". This is due to the way the used driver converts values. More details on the `sql_variant` data type can be found in the [Microsoft Documentation](https://learn.microsoft.com/en-us/sql/t-sql/data-types/sql-variant-transact-sql?view=sql-server-ver16).
* **JSON and XML types**: For JSON and XML columns, it is recommended to disable any format validation in the destination database.
* **Timestamp Column Restrictions**:
  * **Foreign key constraints**: Columns of the `TIMESTAMP` data type cannot be used as the target of foreign key (FK) constraints. This restriction stems from the nature of `TIMESTAMP` as a data type primarily used for version-stamping rows rather than linking rows across tables.
  * **Primary key constraints**: A `TIMESTAMP` column cannot be used as the sole primary key (PK) in a table. If a `TIMESTAMP` column is included in a primary key, it must be combined with one or more other columns.
  * For `TIMESTAMP` columns, the user should be aware that timestamps reflect the moment of writing to the destination database with values of the current system time.
* When entering database or schema names, use lowercase letters. Names containing capital letters must be entered in lowercase to ensure a proper connection.

## **Fast execute many**

`fast_executemany` is an option in Microsoft ODBC drivers that allows for faster data writing to SQL Server databases by optimizing the execution of multiple insert statements. While it can significantly improve performance, it comes with some limitations, particularly around robust datatype handling.

This feature can be enabled directly on the SQL Server **target connector** by toggling `fast_executemany` when configuring the connection.

## **Dropping and recreating indexes**

{% hint style="info" %}
**Coming soon**
{% endhint %}

Tables with many indexes may experience blocking or halting during data writing without any error messages. In such cases, considering dropping indexes for that table can resolve the issue.\
Syntho has a feature called **default\_drop\_indexes** that introduces logic for dropping indexes before writing data to SQL Server tables and recreating them afterward. This process is managed on a per-table basis and is controlled by a new engine attribute.\
\
**default\_drop\_indexes**: A boolean attribute that determines whether indexes should be dropped before writing and recreated afterward. Default value is false and currently, only the MSSQL connector supports this feature.

#### **Enabling index dropping**

To enable this feature, hold **CTRL + SHIFT + ALT + 0** to open the **Workspace Default Settings** menu. There, set the value to **true** for the **default\_drop\_indexes**.

```
default_drop_indexes = true
```

#### **Permissions required**

To execute the DROP INDEX command, the following permissions are required:

* **ALTER permission**: At a minimum, ALTER permission on the table or view is required.
  * This permission is granted by default to the sysadmin fixed server role and the db\_ddladmin and db\_owner fixed database roles.

#### **Limitations**

* **Less robust datatype handling**: When enabled, `fast_executemany` may have less robust handling of certain datatypes, which can lead to issues. Due to these limitations and thorough testing to ensure compatibility with the data is advised.

#### **Recommendations**

* **Performance testing**: Before enabling this feature in a production environment, perform extensive testing to ensure that it does not adversely affect data integrity or application performance.

## Supported data types

The below table indicates what's supported per generator type, and if any additional processing is performed by Syntho to successfully generate and write the data.

| Data Type        | AI-powered Generation                                                                                | Mockers                                                                                    | Mask                                                                                    | Calculated Columns                                                                                         | Other                                      |
| ---------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | ------------------------------------------ |
| BIT              | False                                                                                                | [True](/configure-a-data-generation-job/configure-column-settings/mockers/other)           | False                                                                                   | False                                                                                                      | -                                          |
| TINYINT          | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                          |
| SMALLINT         | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                          |
| INT              | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                          |
| BIGINT           | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                          |
| NUMERIC          | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                          |
| DECIMAL          | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                          |
| FLOAT            | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                          |
| REAL             | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                          |
| DATE             | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      | -                                          |
| TIME             | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      | -                                          |
| DATETIME         | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      | -                                          |
| DATETIME2        | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      | -                                          |
| SMALLDATETIME    | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      | -                                          |
| DATETIMEOFFSET   | True\*                                                                                               | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                     | -                                          |
| CHAR             | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                          |
| NCHAR            | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                          |
| VARCHAR          | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                          |
| NVARCHAR         | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                          |
| TEXT             | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                          |
| NTEXT            | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                          |
| BINARY           | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      | -                                          |
| VARBINARY        | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      | -                                          |
| IMAGE            | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      | -                                          |
| MONEY            | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                          |
| SMALLMONEY       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                          |
| UNIQUEIDENTIFIER | False                                                                                                | [True](/configure-a-data-generation-job/configure-column-settings/mockers/other)           | [True](/configure-a-data-generation-job/configure-column-settings/mask/uuid)            | False                                                                                                      | -                                          |
| ROWVERSION       | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      | Completely excluded from writing           |
| TIMESTAMP        | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      | Completely excluded from writing           |
| XML              | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      | XMLs are written as `<blank>blank</blank>` |
| SQL\_VARIANT     | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                     | -                                          |
| HIERARCHYID      | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                     | -                                          |
| GEOMETRY         | False                                                                                                | [True](/configure-a-data-generation-job/configure-column-settings/mockers/other)           | False                                                                                   | False                                                                                                      | -                                          |
| GEOGRAPHY        | False                                                                                                | [True](/configure-a-data-generation-job/configure-column-settings/mockers/other)           | False                                                                                   | False                                                                                                      | -                                          |

{% hint style="info" %}
**\***&#x53;ome data types are not actively supported; however, certain generators such as AI synthesize, mask, mockers, or calculated columns may still show 'True' for these fields. This means the generators can be applied, even though the types are not actively supported. Duplication is fully supported for these data types.
{% endhint %}


# MySQL / MariaDB

Connect to a MySQL database and set up a workspace.

<figure><img src="/files/hdq53g49kWGNgbL6kHRZ" alt=""><figcaption><p>Source and Destination Databases</p></figcaption></figure>

## Before you begin

Before you begin, gather this connection information:

* Name of the server that hosts the database you want to connect to and port number
* User name and password
* Are you connecting to an SSL server?

## Connect and set up the workspace

Launch Syntho and select **Connect to a database**, or **Create workspace**. Then select **MySQL / MariaDB** from **Type** under **The connection details**. For a complete list of data connections, click **Type** under **The connection details**. Then do the following:

1. Enter the name of the server that hosts the database that you want to connect to.
2. Enter the database name to connect to the contained database.
3. Enter the port number.
4. Leave the protocol as-is. Only in very specific circumstances you may want to change this.
5. Enter user name and password.

   Select the **Require SSL** check box when connecting to an SSL server.
6. Select **Next**.\
   If Syntho can't make the connection, verify that your credentials are correct. If you still can't connect, your computer is having trouble locating the server. Contact your network administrator or database administrator.

## Limitations

* Columns with the data type SET are not supported.
* When entering database or schema names, use lowercase letters. Names containing capital letters must be entered in lowercase to ensure a proper connection.

### Connector Limitations for MySQL / MariaDB

Our MySQL and MariaDB connector, utilizing SQLAlchemy, presents certain limitations that may impact your operations:

#### Known issues

* **Backtick characters in names**: While MySQL supports the use of backtick characters (\`) in table and column names by escaping them, SQLAlchemy fails to escape these characters correctly. This can result in mismatches during table and column referencing in read or write operations.
* **Empty SET data type**: MySQL allows defining columns with an empty `SET` definition (`SET('')`). However, SQLAlchemy does not support this edge case, which may lead to issues when working with `SET` data types.

#### Supported data types

The table below outlines the support for various data types in terms of generator type, and if any additional processing is performed by Syntho to successfully generate and write the data.

| Data Type              | AI-powered Generation                                                                                | Mockers                                                                                    | Mask                                                                                    | Calculated Columns                                                                                         |
| ---------------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| **INTEGER**            | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| **TINYINT**            | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| **SMALLINT**           | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| **MEDIUMINT**          | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| **BIGINT**             | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| **FLOAT**              | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| **DOUBLE**             | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| **DOUBLE PRECISION**   | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| **DECIMAL**            | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| **CHAR**               | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| **VARCHAR**            | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| **TEXT**               | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| **LONGTEXT**           | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| **TINYTEXT**           | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| **MEDIUMTEXT**         | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| **BINARY**             | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      |
| **VARBINARY**          | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      |
| **BLOB**               | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      |
| **TINYBLOB**           | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      |
| **MEDIUMBLOB**         | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      |
| **LONGBLOB**           | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      |
| **DATE**               | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      |
| **TIME**               | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      |
| **DATETIME**           | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      |
| **TIMESTAMP**          | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      |
| **YEAR**               | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      |
| **BIT**                | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| **JSON**               | False                                                                                                | [True](/configure-a-data-generation-job/configure-column-settings/mockers/other#json)      | False                                                                                   | False                                                                                                      |
| **ENUM**               | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      |
| **SET**                | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      |
| **BOOL**               | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| **BOOLEAN**            | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| **POINT**              | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                     |
| **GEOMETRY**           | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                     |
| **LINESTRING**         | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                     |
| **POLYGON**            | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                     |
| **MULTIPOINT**         | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                     |
| **MULTILINESTRING**    | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                     |
| **MULTIPOLYGON**       | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                     |
| **GEOMETRYCOLLECTION** | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                     |

{% hint style="info" %}
**\***&#x53;ome data types are not actively supported; however, certain generators such as AI synthesize, mask, mockers, or calculated columns may still show 'True' for these fields. This means the generators can be applied, even though the types are not actively supported. Duplication is fully supported for these data types.
{% endhint %}


# Oracle

<figure><img src="/files/a7FzWHnhIp19bxWwlQke" alt=""><figcaption><p>Source and Destination Databases</p></figcaption></figure>

## Before you begin

Before you begin, gather this connection information:

* Name of the server that hosts the database you want to connect to and the Oracle service name and port
* User name and password
* Are you connecting to an SSL server? You must configure the Oracle client before you can use SSL

## Connect and set up the workspace

Launch Syntho and select **Connect to a database**, or **Create workspace**. Then select **Oracle** from **Type** under **The connection details**. For a complete list of data connections, click **Type** under **The connection details**. Then do the following:

1. Enter the server name.
   * Note: To find the correct service name, open the tnsnames.ora file (in the ORACLE\_HOME/network/admin directory by default) and search for the net\_service\_name section. Use the value listed for service\_name.
2. Enter the database name to connect to the contained database.
3. Enter the port number.
4. Enter user name and password.

   Select the **Require SSL** check box when connecting to an SSL server.
5. To connect to one schema only, enter its name in **Schema**.
6. Enable **Include views** to include views in the connection.
7. Select **Next**.\
   If Syntho can't make the connection, verify that your credentials are correct. If you still can't connect, your computer is having trouble locating the server. Contact your network administrator or database administrator.

{% hint style="warning" %}
When source and destination use the same database, select different schemas. Ensure no foreign keys span these schemas. Cross-schema foreign keys can cause data-generation issues.
{% endhint %}

## Limitations

* There is a possibility for some **Oracle DB** system schemas to appear after creating workspace. This shouldn't happen if database permissions and schemas are set properly, but if it does happen then user should manually exclude schemas from synthetization. See below the list of schema that are excluded by default.
* When entering database or schema names, use lowercase letters. Names containing capital letters must be entered in lowercase to ensure a proper connection.

<details>

<summary>List of automatically excluded tables in Oracle</summary>

* anonymous
* appqossys
* audsys
* dbsfwuser
* dbsnmp
* dgpdb\_int
* dip
* dvf
* dvsys
* ggsys
* gsmadmin\_internal
* gsmcatuser
* gsmrootuser
* gsmuser
* lbacsys
* ops$oracle
* oracle\_ocm
* outln
* remote\_scheduler\_agent
* sys
* sys$umf
* sysbackup
* sysdg
* syskm
* sysrac
* system
* xdb
* xs$null

</details>

### Supported data types

The following table provides a summary of the current support and limitations for various data types when using connectors with Oracle. It indicates what's supported per generator type, and if any additional processing is performed by Syntho to successfully generate and write the data.

| Data Type                      | AI-powered Generation                                                                                | Mockers                                                                                    | Mask                                                                                    | Calculated Columns                                                                                         |
| ------------------------------ | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| INTEGER                        | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| SMALLINT                       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| NUMBER                         | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| FLOAT                          | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| BINARY\_FLOAT                  | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| BINARY\_DOUBLE                 | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| CHAR                           | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| VARCHAR2                       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| NCHAR                          | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| NVARCHAR2                      | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |
| CLOB                           | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      |
| DATE                           | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      |
| TIMESTAMP                      | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      |
| TIMESTAMP WITH TIME ZONE       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      |
| TIMESTAMP WITH LOCAL TIME ZONE | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      |
| INTERVAL YEAR TO MONTH         | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      |
| INTERVAL DAY TO SECOND         | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      |
| RAW                            | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      |
| BLOB                           | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      |
| NCLOB                          | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      |
| ROWID                          | True\*                                                                                               | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                     |
| UROWID                         | True\*                                                                                               | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                     |
| XMLTYPE                        | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) |

{% hint style="info" %}
**\***&#x53;ome data types are not actively supported; however, certain generators such as AI synthesize, mask, mockers, or calculated columns may still show 'True' for these fields. This means the generators can be applied, even though the types are not actively supported. Duplication is fully supported for these data types.
{% endhint %}


# PostgreSQL

<figure><img src="/files/nKMCEO5yVSPmOButk7dN" alt=""><figcaption><p>Source and Destination Databases</p></figcaption></figure>

## Before you begin

Before you begin, gather this connection information:

* Name of the server that hosts the database you want to connect to
* Database name
* User name and password
* Are you connecting to an SSL server?

## Connect and set up the workspace

Launch Syntho and select **Connect to a database**, or **Create workspace**. Then select **PostgreSQL** from **Type** under **The connection details**. For a complete list of data connections, click **Type** under **The connection details**. Then do the following:

1. Enter the name of the server that hosts the database that you want to connect to.
2. Enter the database name to connect to the contained database.
3. Enter the port number.
4. Enter user name and password.

   Select the **Require SSL** check box when connecting to an SSL server.
5. To connect to one schema only, enter its name in **Schema**.
6. Enable **Include views** to include views in the connection.
7. Select **Next**.\
   If Syntho can't make the connection, verify that your credentials are correct. If you still can't connect, your computer is having trouble locating the server. Contact your network administrator or database administrator.

{% hint style="warning" %}
When source and destination use the same database, select different schemas. Ensure no foreign keys span these schemas. Cross-schema foreign keys can cause data-generation issues.
{% endhint %}

## Considerations & limitations

* For JSON and XML columns, turn off validation in target and/or enable **ALTER** permission for user.
* When entering database or schema names, use lowercase letters. Names containing capital letters must be entered in lowercase to ensure a proper connection.

### Supported data types

This table outlines the support and limitations for various data types in PostgreSQL when using connectors. It indicates what's supported per generator type, and if any additional processing is performed by Syntho to successfully generate and write the data.

| Data type          | AI-powered generation                                                                                | Mockers                                                                                    | Mask                                                                                    | Calculated columns                                                                                        | Other                                                                                                 |
| ------------------ | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| **SMALLINT**       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#suported-data-types) | -                                                                                                     |
| **INTEGER**        | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#suported-data-types) | -                                                                                                     |
| **BIGINT**         | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#suported-data-types) | -                                                                                                     |
| **DECIMAL**        | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#suported-data-types) | -                                                                                                     |
| **NUMERIC**        | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#suported-data-types) | -                                                                                                     |
| **REAL**           | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#suported-data-types) | -                                                                                                     |
| **DOUBLE**         | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#suported-data-types) | -                                                                                                     |
| **CHAR**           | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#suported-data-types) | -                                                                                                     |
| **VARCHAR**        | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#suported-data-types) | -                                                                                                     |
| **TEXT**           | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#suported-data-types) | -                                                                                                     |
| **BYTEA**          | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                     | -                                                                                                     |
| **BOOLEAN**        | False                                                                                                | [True](/configure-a-data-generation-job/configure-column-settings/mockers/other)           | False                                                                                   | False                                                                                                     | -                                                                                                     |
| **DATE**           | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                     | -                                                                                                     |
| **TIME**           | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                     | -                                                                                                     |
| **TIMESTAMP**      | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                     | -                                                                                                     |
| **INTERVAL**       | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **JSON**           | True\*                                                                                               | [True](/configure-a-data-generation-job/configure-column-settings/mockers/other#json)      | False                                                                                   | False                                                                                                     | JSON / XML de-identification (Coming soon)                                                            |
| **JSONB**          | True\*                                                                                               | [True](/configure-a-data-generation-job/configure-column-settings/mockers/other#json)      | False                                                                                   | False                                                                                                     | JSON / XML de-identification (Coming soon)                                                            |
| **HSTORE**         | True\*                                                                                               | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **UUID**           | False                                                                                                | [True](/configure-a-data-generation-job/configure-column-settings/mockers/other)           | [True](/configure-a-data-generation-job/configure-column-settings/mask/uuid)            | False                                                                                                     | -                                                                                                     |
| **INET**           | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **CIDR**           | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **TSVECTOR**       | True\*                                                                                               | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **TSQUERY**        | True\*                                                                                               | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **POINT**          | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **LSEG**           | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **BOX**            | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **PATH**           | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **POLYGON**        | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **LINE**           | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **CIRCLE**         | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **MONEY**          | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#suported-data-types) | -                                                                                                     |
| **OID**            | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **BIT**            | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#suported-data-types) | -                                                                                                     |
| **BIT VARYING**    | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#suported-data-types) | -                                                                                                     |
| **ENUM**           | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                     | -                                                                                                     |
| **TSRANGE**        | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **TSTZRANGE**      | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **NUMRANGE**       | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **INT4RANGE**      | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **INT8RANGE**      | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **INT4MULTIRANGE** | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **INT8MULTIRANGE** | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **XML**            | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                     | JSON / XML de-identification (Coming soon)(Content may be `<blank>blank</blank>` if no rule applies.) |
| **ARRAY INT**      | True\*                                                                                               | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **ARRAY VARCHAR**  | True\*                                                                                               | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **FLOAT**          | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#suported-data-types) | -                                                                                                     |
| **MACADDR**        | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **MACADDR8**       | False                                                                                                | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#suported-data-types) | -                                                                                                     |
| **REGCONFIG**      | True\*                                                                                               | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **REGCLASS**       | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |
| **DATERANGE**      | False                                                                                                | True\*                                                                                     | True\*                                                                                  | True\*                                                                                                    | -                                                                                                     |

{% hint style="info" %}
**\***&#x53;ome data types are not actively supported; however, certain generators such as AI synthesize, mask, mockers, or calculated columns may still show 'True' for these fields. This means the generators can be applied, even though the types are not actively supported. Duplication is fully supported for these data types.
{% endhint %}


# SAP Sybase

{% hint style="warning" %}
**Warning**

The Sybase Connector/Python DBAPI is a currently no longer maintained by SAP and has no support for versions 16 below. The Sybase dialect is not tested as part of Syntho’s continuous integration, due to the unavailability of adequate test databases / environments.
{% endhint %}

<figure><img src="/files/xU95BLP4FonI4przpH4c" alt=""><figcaption><p>Source and Destination Databases</p></figcaption></figure>

## Before you begin <a href="#before-you-begin" id="before-you-begin"></a>

Before you begin, gather this connection information:

* Name of the server you want to connect to, port number, and if you use single sign-on, service name
* The database name
* Optionally, the name of the schema that you want to connect to
* User name and password
* Are you connecting to an SSL server?

## Connect and set up the workspace

Launch Syntho and select **Connect to a database**, or **Create workspace**. Then select **SAP Sybase** from **Type** under **The connection details**. For a complete list of data connections, click **Type** under **The connection details**. Then do the following:

1. Enter the name of the server that hosts the database that you want to connect to.
2. Enter the database name to connect to the contained database.
3. Enter the port number.
4. Enter your user name and password.

   Select the **Require SSL** check box when connecting to an SSL server.
5. Select **Create Workspace**.\
   If Syntho can't make the connection, verify that your credentials are correct. If you still can't connect, your computer is having trouble locating the server. Contact your network administrator or database administrator.

## Limitations

* When entering database or schema names, use lowercase letters. Names containing capital letters must be entered in lowercase to ensure a proper connection.

### Supported data types

The following table outlines the support and limitations for various data types in **Sybase ASE** when using connectors. It indicates what's supported per generator type, and if any additional processing is performed by Syntho to successfully generate and write the data.

| Data Type     | AI-powered Generation                                                                                | Mockers                                                                                    | Mask                                                                                    | Calculated Columns                                                                                         | Other                                               |
| ------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | --------------------------------------------------- |
| INT           | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                                   |
| BIGINT        | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                                   |
| SMALLINT      | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                                   |
| TINYINT       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                                   |
| DECIMAL       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                                   |
| NUMERIC       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                                   |
| FLOAT         | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                                   |
| REAL          | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                                   |
| MONEY         | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                                   |
| SMALLMONEY    | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)  | [True](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal) | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                                   |
| DATE          | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      | -                                                   |
| TIME          | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      | -                                                   |
| SMALLDATETIME | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      | -                                                   |
| DATETIME      | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      | -                                                   |
| BIGDATETIME   | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      | -                                                   |
| BIGTIME       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)    | [True](/configure-a-data-generation-job/configure-column-settings/mockers/datetime)        | [True](/configure-a-data-generation-job/configure-column-settings/mask/datetime)        | False                                                                                                      | -                                                   |
| CHAR          | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                                   |
| VARCHAR       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                                   |
| NCHAR         | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                                   |
| NVARCHAR      | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                                   |
| TEXT          | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | -                                                   |
| UNITEXT       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | Completely excluded from writing                    |
| BINARY        | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      | Writing BINARY might fail unless ODBC is configured |
| VARBINARY     | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      | -                                                   |
| IMAGE         | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      | -                                                   |
| BIT           | False                                                                                                | [True](/configure-a-data-generation-job/configure-column-settings/mockers/other)           | False                                                                                   | False                                                                                                      | Completely excluded from writing                    |
| UNICHAR       | [True](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | [True](/configure-a-data-generation-job/configure-column-settings/mockers/text)            | [True](/configure-a-data-generation-job/configure-column-settings/mask/text)            | [True](/configure-a-data-generation-job/configure-column-settings/calculated-columns#supported-data-types) | Completely excluded from writing                    |
| UNIVARCHAR    | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      | Completely excluded from writing                    |
| TIMESTAMP     | False                                                                                                | False                                                                                      | False                                                                                   | False                                                                                                      | Completely excluded from writing                    |

{% hint style="info" %}
**\***&#x53;ome data types are not actively supported; however, certain generators such as AI synthesize, mask, mockers, or calculated columns may still show 'True' for these fields. This means the generators can be applied, even though the types are not actively supported. Duplication is fully supported for these data types.
{% endhint %}


# Azure Data Lake Storage (ADLS)

<figure><img src="/files/3TDDL4DeXqiUDEwqvLnJ" alt=""><figcaption><p>Source and Destination Databases</p></figcaption></figure>

{% hint style="info" %}
**Destination only**

This connector can only be used as a destination for writing your generated data.

* Supported File Types: Parquet
* Supported Partitioning: Horizontal partitioning based on the batch size. Each batch is written to a separate file.
  {% endhint %}

## Before you begin <a href="#before-you-begin" id="before-you-begin"></a>

Before you begin, review the items in the following list:

* Get the URL for your **Azure** storage endpoint.
* Depending on how you want to connect, prepare either:
  * the storage account name and storage account key, *or*
  * the connection string.
* Provide read/write permissions to the storage container that is used (i.e., the storage container name).
* Get the remote path, which is the relative path to the storage container that is used.

## Output format

Syntho's ADLS output connector will write all generated data to **Parquet** files as follows:

* Each generated table will be written to a Parquet file in the following format:\
  `{schema-name}-{table_name}_part_{part_number}.parquet`
* The number of rows in a single Parquet file (part) is defined by the batch size. All Parquet parts of a single table are stored in their own directory.
* Each folder name will use the following format:

  `{schema_name}.{table_name}`

## Connect and set up the workspace

Launch Syntho and select **Connect to a database**, or under **Create workspace > Destination Database**, select **ADLS**. For a complete list of data connections, select **More** under **From database**. Then do the following:

1. Enter the remote path.
2. Enter the storage container name.
3. Either:
   * Enter the storage account name and the storage account key, *or:*
   * Enter the storage connection string.
4. Select **Create Workspace**.\
   If Syntho can't make the connection, verify that your credentials are correct. If you still can't connect, your computer is having trouble locating the server. Contact your network administrator or database administrator.

## Limitations & considerations

Contact your Syntho contact person to discuss possible limitations regarding this connector.


# Amazon Simple Storage Service (S3)

<figure><img src="/files/JjklyvPzMNlmn9i02ImE" alt=""><figcaption><p>Source and Destination Databases</p></figcaption></figure>

{% hint style="info" %}
**Destination only**

This connector can only be used as a destination for writing your generated data.

* Supported File Types: Parquet
* Supported Partitioning: Horizontal partitioning based on the batch size. Each batch is written to a separate file.
  {% endhint %}

## Before you begin <a href="#before-you-begin" id="before-you-begin"></a>

Before you begin, gather this connection information:

* Get the connection details to connect with your **S3 bucket**

## Output format

Syntho's S3 output connector writes all generated data to Parquet files as follows:

* Each generated table will be written to a Parquet file in the following format:\
  `{schema-name}-{table_name}_part_{part_number}.parquet`
* The number of rows in a single Parquet file (part) is defined by the batch size. All Parquet parts of a single table are stored in their own directory.
* Each folder name will use the following format:

  `{schema_name}.{table_name}`

## Connect and set up the workspace

Launch Syntho and select **Connect to a database**, or under **Create workspace > Destination Database**, select **S3**. For a complete list of data connections, select **More** under **From database**. Then do the following:

1. Enter the bucket name.
2. Enter the region name.
3. Enter the port number.
4. Enter the AWS access key id.
5. Enter the AWS secret access key.
6. Enter the prefix.\
   If Syntho can't make the connection, verify that your credentials are correct. If you still can't connect, your computer is having trouble locating the server. Contact your network administrator or database administrator.

## Limitations & considerations

Contact your Syntho contact person to discuss possible limitations regarding this connector.


# Local filesystem

{% hint style="info" %}
**Syntho beta feature**
{% endhint %}

The local filesystem connector enables data writing to a specified path in the host's local filesystem. To utilize this connector, specific configuration steps are required to ensure proper functionality.

<figure><img src="/files/2mj7M0z1okJ5hZyE93ro" alt=""><figcaption><p>Source and Destination Databases</p></figcaption></figure>

{% hint style="info" %}
**Destination Only**

This connector can only be used as a destination for writing your generated data.

Supported File Types: Parquet and ORC
{% endhint %}

## **Before you begin**

#### **Environment variable setup**:

Before running the Syntho application, declare the environment variable `SYNTHO_LOCALFS`. This variable should be set to the desired path in the host local filesystem where the data will be written.\
Example:

```bash
export SYNTHO_LOCALFS="/path/on/host"
```

Alternatively, you can declare the environment variable when running Docker:

```bash
SYNTHO_LOCALFS="/path/on/host" docker-compose up
```

**Note**: It is recommended to declare the environment variable using `export`.

## **Usage**

* **Destination Only**: The Local Filesystem connector can only be used as a target destination.
* **Parameter Requirement**: The only required parameter is `path`. The value of this parameter will be concatenated with the value of `SYNTHO_LOCALFS`, and the resulting path will be where the data is written.

#### **Example**

If `SYNTHO_LOCALFS` is set to `/mnt/data` and the `path` parameter is `output`, the data will be written to `/mnt/data/output`.

This configuration allows the Syntho application to write data to the local filesystem, ensuring compatibility with various deployment environments without affecting other users who do not require this functionality.

## Connect and set up the workspace

Launch Syntho and select Connect to a database, or under Create workspace, select Local Filesystem. Then do the following:

1. Enter the local target path where the data will be written.
2. Choose a file format: Parquet or ORC.
3. Select Create Workspace.

If Syntho can't make the connection, verify that the target path is correct. If you still can't connect, contact your database administrator.

## File formats

Supported file type formats include:

* Parquet
* ORC

## Limitations & considerations

* For ORC files, columns full of None values which are of type Char, String, or Varchar will be written as "None" (i.e., a string value) to the destination database instead of None.
* Local Filesystem connector is only supported for Docker Compose.

Contact your Syntho representative to discuss possible limitations regarding this connector.


# Workspace modes

With Syntho, you can initialize a workspace by automatically applying suggested generators to all columns based on your workspace mode, speeding up your configuration process.

When creating a workspace, select one of the following modes to control how data is generated.

{% embed url="<https://www.guidejar.com/guides/4a0b2d1e-b093-4910-b550-81c863921bba>" %}

### Initialization modes overview

| Mode             | When to use                                                                                                                                                                 | What happens                                                                                                     |
| ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| Synthesize all   | For one or more two related tables. To preserve maximum statistical validity with high privacy.                                                                             | PII columns receive mock generator suggestions, and non-PII columns use AI synthesis.                            |
| Mock all         | Theoretically the most secure, since all data is generated independently of the original data. This is also the best choice if there is no or limited input data available. | All PII and non-PII columns get one or more suggested mock generators.                                           |
| Mock or mask all | Like “Mock all”, except masking generators may be applied in certain scenarios to generate more realistic data.                                                             | All PII and non-PII columns get one or more suggested mock and/or mask generators.                               |
| De-identify      | When only replacement of PII is required, while non-PII columns should remain in-tact, making it suitable for internal test cases.                                          | Suggested generators are applied only to PII columns. Non-PII columns do not receive any generator suggestions.. |
| From scratch     | For custom configurations or when only a few tables need to be configured.                                                                                                  | All tables start excluded by default, and the application doesn't suggest generators.                            |

### PII scan

For all workspace modes (except **From Scratch**), a **PII scan** is automatically run during workspace creation. This scan detects and configures sensitive columns for de-identification, ensuring privacy compliance from the start.

This step is essential for modes that involve synthetization or de-identification to maintain data privacy and security. For more information, please refer to [Automatic PII discovery with PII scanner](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner).

### Key generator

Select a **default key generator** to handle primary key (PK) and foreign key (FK) generation, ensuring data integrity and privacy.

1. **Duplicate**: Keeps original key values intact, maintaining relationships between tables.
2. **Hash**: Converts original keys into hashed values, preserving table relationships while obfuscating data for enhanced privacy.

You can adjust this setting later in your workspace configuration. For more information, refer to [Key generators](/configure-a-data-generation-job/configure-column-settings/key-generators).

### **Locale settings**

Set the workspace locale for mockers and PII processors. The PII scanner supports English, Dutch, German, and Japanese.

Select the locale that matches the data you scan. Syntho uses English (US) when the selected locale is unsupported. You can update this setting later in the workspace settings.

### User roles

* **Before workspace creation**:\
  Any user can make full use of this feature.
* **After workspace creation**:\
  Only users with **Editor rights or higher** can make full use of this feature.


# Use SQL views as input tables

Shape your source data into a single, analysis-ready entity table (often via a SQL view) before configuring Syntho.

Use a SQL view when you need to reshape data *before* Syntho reads it.

This is most common for:

* **AI synthesize** workflows that work best on a single entity table.
* **Data sharing** where recipients prefer one clean table.
* **Analytics sandboxes** where BI dashboards query a flattened model.

### Why this matters

A well-designed input table makes configuration simpler and safer:

* Fewer joins means fewer linkage points.
* You can enforce “one row = one entity” rules.
* You can precompute derived fields once, then keep Syntho configs minimal.

### Practical pattern: create a single entity view

1. Pick the entity you care about (customer, patient, account).
2. Join only what you need for the use case.
3. Ensure every row has a stable, unique key.

{% hint style="info" %}
Syntho can read from a **table or view** in the source. See [Table view](/configure-a-data-generation-job/configure-table-settings).
{% endhint %}

#### Example (PostgreSQL): flatten customers + last order

```sql
CREATE OR REPLACE VIEW syntho_customer_entity_v AS
SELECT
  c.customer_id,
  c.country_code,
  c.segment,
  c.signup_date,
  o.order_date  AS last_order_date,
  o.order_total AS last_order_total
FROM customers c
LEFT JOIN LATERAL (
  SELECT order_date, order_total
  FROM orders o
  WHERE o.customer_id = c.customer_id
  ORDER BY order_date DESC
  LIMIT 1
) o ON true;
```

### Destination considerations

Syntho writes to the **destination**. The destination object must usually be a **table**.

If your source is a view:

* Create a destination table with the same columns.
* Or write to a filesystem destination (e.g., Parquet), if that fits your workflow.

### Edge cases and gotchas

* **Missing primary keys**: add a stable surrogate key in the view, or model a different entity.
* **Exploding joins** (1-to-many): aggregate first (e.g., counts, sums, “last event”), then join.
* **Materialized views**: use them when the view becomes slow to query during training.
* **Security**: grant **read-only** access to the view, not the full underlying schema.

### Where to use this in Syntho

* Use the view as the source table when configuring [AI synthesize](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation).
* Keep the generator config small. Prefer calculated columns for only the rules you must enforce.


# Edit a workspace

Edit an existing workspace using either option:

1. If the workspace you want to edit is already selected, click the **Edit** button (![](/files/2kcZ6EO1AS2GTspHIVFU)) in the top toolbar.
2. If you’re inside the workspace you want to edit, open the **Actions** menu in the top toolbar and choose Edit (![](/files/CTEr1hZdx7dS0o9q05ZM)).

{% embed url="<https://www.guidejar.com/guides/4f692004-285d-46e0-818b-3d5fa434b7de>" %}

After launching the Edit workspace panel, you can edit the fields you want to.

For example, for MySQL, you can edit the following fields:

* Host (**destination** database only)
* Database (**destination** database only)
* Port number (**destination** database only)
* Protocol (**destination** database only)
* Username
* Password
* SSL settings
* SSHTunnel settings

Finally, select **Save changes** to confirm editing the workspace.


# Duplicate a workspace

Duplicate an existing workspace using either option:

1. If the workspace you want to duplicate is already selected, click the **Duplicate** button in the top toolbar.
2. If you’re inside the workspace you want to duplicate, open the **Actions** menu in the top toolbar and choose **Duplicate** (![](/files/CTEr1hZdx7dS0o9q05ZM)).

{% embed url="<https://www.guidejar.com/guides/2936e5b7-d3f8-4bd6-9b8e-c6e0e1fcf57b>" %}

Note that, the Duplicate workspace feature also allows you to optionally set:

* The **source** and **destination** databases for the duplicated workspace.
* **Workspace modes** to configure specific processing modes for tables in the new workspace.

Finally, select **Yes** to confirm duplication the workspace or select **No** to cancel the workspace duplication.


# Transfer workspace ownership

Each workspace has exactly one owner. You can become a workspace owner, either by:

* When you create a workspace, you automatically become the owner of that workspace.
* You can transfer ownership of a workspace that you own to another user. When you transfer ownership, you can set a new role for yourself for the workspace. For example, you can transfer ownership to another user, but grant yourself **Editor** access.

Admin users can transfer ownership of any workspace. Transferring ownership does not affect the existing access of other non-owner users.

To transfer ownership of a workspace:

1. If the workspace you want to transfer is already selected, click **Transfer Ownership** button in the top toolbar.
2. If you’re inside the workspace, open the **Actions** menu in the top toolbar and choose **Transfer Ownership** (![](/files/CTEr1hZdx7dS0o9q05ZM)).
3. In the **Select the user** dropdown, choose the new owner.
4. To keep your own access, switch on **Keep access to workspace** and select a role for yourself from the **Role** dropdown.
5. Click **Confirm**.

{% embed url="<https://www.guidejar.com/guides/599b28cb-fe17-420d-adcb-7a52bbc8a512>" %}


# Share a workspace

## Assign workspace role to other users

As a workspace owner or editor, you can utilize the workspace sharing feature to delegate roles within your workspaces.

If you're an admin user, you have the ability to assign roles for any workspace. Share an existing workspace using either option:

1. If the workspace you want to share is already selected, click the **Share** button in the top toolbar.
2. If you’re inside the workspace you want to share, open the **Actions** menu in the top toolbar and choose **Share** (![](/files/CTEr1hZdx7dS0o9q05ZM)).
3. In the **Share workspace** panel you’ll see everyone who already has access.
4. To add a collaborator, open the **Select the user** dropdown and select the user.

{% embed url="<https://www.guidejar.com/guides/5bf1d1f4-c366-4457-95d9-04e72d3c2f2d>" %}

Assigning a role to a user works as follows:

4. Select the **Role** dropdown and choose the desired role.
5. Use the **Data access** toggle to control the user's data visibility.
6. Select **Confirm**.

## Available workspace roles

You can select the following roles when sharing a workspace with a user.

<table><thead><tr><th width="300">Action</th><th>Reader</th><th>Editor</th><th>Owner</th></tr></thead><tbody><tr><td>View Job Settings</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>View Subset Configuration</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>View Job History / Summary</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>View Data</td><td>t.b.d. by Owner</td><td>t.b.d. by Owner</td><td>✓</td></tr><tr><td>Edit Job Settings</td><td></td><td>✓</td><td>✓</td></tr><tr><td>Edit Subsetting Configuration</td><td></td><td>✓</td><td>✓</td></tr><tr><td>View Database Validation Results</td><td></td><td>✓</td><td>✓</td></tr><tr><td>Synchronize Workspace</td><td></td><td>✓</td><td>✓</td></tr><tr><td>Run PII Scan</td><td></td><td>✓</td><td>✓</td></tr><tr><td>Add/Remove PII columns</td><td></td><td>✓</td><td>✓</td></tr><tr><td>Run FK Scan</td><td></td><td>✓</td><td>✓</td></tr><tr><td>Add/Remove Foreign Keys</td><td></td><td>✓</td><td>✓</td></tr><tr><td>Generate Synthetic Data</td><td></td><td>✓</td><td>✓</td></tr><tr><td>Cancel Job</td><td></td><td>✓</td><td>✓</td></tr><tr><td>Delete Job</td><td></td><td>✓</td><td>✓</td></tr><tr><td>View Connection Details</td><td></td><td>✓</td><td>✓</td></tr><tr><td>Edit Connection Details of Destination database</td><td></td><td>✓</td><td>✓</td></tr><tr><td>Duplicate Workspace</td><td></td><td>✓</td><td>✓</td></tr><tr><td>Edit Connection Details of Source database</td><td></td><td></td><td>✓</td></tr><tr><td>Share Workspace</td><td></td><td></td><td>✓</td></tr><tr><td>Rename Workspace</td><td></td><td></td><td>✓</td></tr><tr><td>Delete Workspace</td><td></td><td></td><td>✓</td></tr></tbody></table>


# Delete a workspace

Delete an existing workspace using either option:

1. If the workspace you want to delete is already selected, click the **Delete** button in the top toolbar (![](/files/lPrXcwf7esx7rgZ2EwYd)).
2. If you’re inside the workspace, open the **Actions** menu in the top toolbar and choose **Delete**  (![](/files/CTEr1hZdx7dS0o9q05ZM)).

{% embed url="<https://www.guidejar.com/guides/e139b5b3-6adb-4838-bbef-8b29c752ce3b>" %}

Finally, select **Yes** to confirm deleting the workspace or select **No** to cancel the workspace deletion.


# Workspace default settings

The **Workspace default settings** menu allows to configure default parameters for workspaces. These settings ensure consistent behavior for data processing, privacy protection, and synthetic data generation. Below is a detailed explanation of the available options.

{% hint style="info" %}
These defaults may vary by deployment/version; your admin may expose a subset.
{% endhint %}

## Access the workspace default settings

Note that user should be an **Owner** or **Editor** to access Workspace Default Settings.

1. Go to **Edit workspace** via **Saved workspaces** panel or from **Configuration panel**.
2. Select **Default settings**.

## **How to modify settings**

1. Access the **Workspace default settings** menu.
2. Modify the required values directly.
3. **Confirm** changes to apply them to the workspace.

## **Configuration options**

Below is an overview of the default settings and their functionalities.

{% hint style="info" %}
`pii_model_settings` controls models used for PII scanning/mocking. `default_text_processor_model_settings` uses the same structure (engine/models/gpu), but applies to text processing defaults.
{% endhint %}

### Consistency (seed) settings

For more information please see [Configure to use other NLP models (limited support)](/configure-a-data-generation-job/configure-column-settings/duplicate/automatic-pii-discovery-and-de-identification-in-free-text-columns#configure-to-use-other-nlp-models-limited-support).

### AI synthesis (performance)

| **Parameter**                    | **Default**  | **Possible Values**                   | **Description**                                                                                                                                 |
| -------------------------------- | ------------ | ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `n_parallel_pipeline_processes`  | 1            | Any integer ≥ 1                       | Number of column-level processing tasks that run in parallel. Higher values can speed up runs, but increase CPU/memory usage.                   |
| `default_n_training_rows`        | 100000       | Any integer ≥ 1                       | Default [number of rows](/configure-a-data-generation-job/configure-table-settings#adjust-the-number-of-rows-to-generate) used to train models. |
| `default_read_random_subset`     | `false`      | `true` \| `false` (Boolean)           | If `true`, components that support it may read a random subset of rows by default.                                                              |
| `default_max_n_feat_per_model`   | -1           | Any integer (use `-1` for “no limit”) | Default cap on number of features per AI synthesis model.                                                                                       |
| `default_feat_model_train_order` | `"as_given"` | Currently supported: `"as_given"`     | Default training order for AI synthesis models.                                                                                                 |

### AI synthesis (privacy)

| **Parameter**                       | **Default** | **Possible Values**         | **Description**                                                                                                                                                                                                                                                                                                                                                                                                     |
| ----------------------------------- | ----------- | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `default_sample_noise_ratio`        | 0.0001      | Any number                  | <p>Default noise ratio used by components that support noise injection.</p><p>The noise factor will be relative by having a decimal value lower than 1, where a value of 0.001 will lead to better privacy than 0.0001. The noise factor will be absolute, for integer values above 1, where higher values add more noise, so more privacy.</p>                                                                     |
| `default_noise_factor`              | 0.0001      | Any number                  | Default factor used by components that scale noise based on data.                                                                                                                                                                                                                                                                                                                                                   |
| `default_min_sample_size`           | 5           | Any integer ≥ 1             | Default minimum sample size used by components during training.                                                                                                                                                                                                                                                                                                                                                     |
| `default_cardinality_threshold`     | 10          | Any integer ≥ 1             | Categories with occurrences below this threshold are treated as rare (see [rare category protection](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#rare-category-protection)). NOTE: This works per batch as defined under: [Step 3. Generate](/configure-a-data-generation-job/generation-and-validation/view-and-adjust-generation-settings#tune-generation-settings-optional) |
| `default_rare_category_replacement` | `"*"`       | Any string                  | Replacement value for rare categories.                                                                                                                                                                                                                                                                                                                                                                              |
| `default_clip_threshold`            | 0           | Any number                  | Default clip threshold used by components that support clipping/extreme-value limiting.                                                                                                                                                                                                                                                                                                                             |
| `default_pii_mock_replace`          | `false`     | `true` \| `false` (Boolean) | Default for replacing detected PII with mock data where applicable. See [Free text PII obfuscation](/configure-a-data-generation-job/configure-column-settings/duplicate/automatic-pii-discovery-and-de-identification-in-free-text-columns).                                                                                                                                                                       |

### Text processing

| **Parameter**                           | **Default**                                                                                                                                                                                                                                                 | **Possible Values**                                                             | **Description**                                                                                                                                                                                                                                                                                         |
| --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `default_text_processor_model_settings` | `{"models":[{"lang_code":"en","model_name":"en_core_web_md"},{"lang_code":"nl","model_name":"nl_core_news_md"},{"lang_code":"de","model_name":"de_core_news_md"},{"lang_code":"ja","model_name":"ja_core_news_md"}],"nlp_engine_name":"spacy","gpu":false}` | Object with keys: `models` (array), `nlp_engine_name` (string), `gpu` (boolean) | Models/engine used for text processing defaults. Used by features like [Free text PII obfuscation](/overview/get-started/syntho-bootcamp/5.-generators/free-text-de-identification).                                                                                                                    |
| `default_textpii_parallel_jobs`         | 2                                                                                                                                                                                                                                                           | Any integer ≥ 1                                                                 | Parallel jobs used by PII text processing (see [Free text PII obfuscation](/configure-a-data-generation-job/configure-column-settings/duplicate/automatic-pii-discovery-and-de-identification-in-free-text-columns)). Higher values can be faster, but use more CPU/memory and can increase contention. |
| `default_textpii_scan_batch_size`       | 1000                                                                                                                                                                                                                                                        | Any integer ≥ 1                                                                 | Batch size used by PII text processing (see [Free text PII obfuscation](/configure-a-data-generation-job/configure-column-settings/duplicate/automatic-pii-discovery-and-de-identification-in-free-text-columns)). Larger batches can be faster, but use more memory.                                   |
| `default_cutoff_length`                 | 1000                                                                                                                                                                                                                                                        | Any integer ≥ 1                                                                 | Default cutoff length used by components that truncate long text/sequences during processing. See [Free text PII obfuscation](/overview/get-started/syntho-bootcamp/5.-generators/free-text-de-identification).                                                                                         |

### Throughput optimization

| **Parameter**                     | **Default** | **Possible Values**         | **Description**                                                                                                                                                             |
| --------------------------------- | ----------- | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `default_max_pending_tasks`       | 1           | Any integer ≥ 1             | Max number of tables/tasks queued for concurrent processing. Higher values can improve throughput, but increase memory usage and can increase database connection pressure. |
| `default_data_batch_size`         | 1000000     | Any integer ≥ 1             | The maximum number of datapoints processed at once (N columns x N rows).                                                                                                    |
| `default_n_connections`           | 1           | Any integer ≥ 1             | The maximum number of connections that can be made to the database during the writing. A higher number can speed up the process as it allows for more parallel writing.     |
| `default_simultaneous_read_write` | true        | `true` \| `false` (Boolean) | When enabled, the next batch is generated while the previous batch is still being written. Disabling this will make generation slower but use less memory.                  |

### Other

| **Parameter**                                  | **Default** | **Possible Values**         | **Description**                                                                                                                               |
| ---------------------------------------------- | ----------- | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| `default_consistent_integer_shuffle_threshold` | 0           | Any integer ≥ 0             | Threshold value used by consistent integer shuffle logic. `0` disables the threshold behavior.                                                |
| `default_order_by_nr_columns`                  | `[0, 0]`    | Array of two integers       | Default order-by configuration used by [order-by](/configure-a-data-generation-job/configure-table-settings#order-by) logic where applicable. |
| `default_exclude_tables`                       | `false`     | `true` \| `false` (Boolean) | If `true`, tables start excluded by default. Related: [Workspace modes](/setup-workspaces/create-a-workspace/workspace-modes).                |


# Import and export workspaces

### Overview

Import/export lets you move a workspace configuration between environments. Use it for backups, restores, and standardizing setups across teams.

You export a workspace configuration as a JSON file. You import that JSON into an **existing** workspace.

Import triggers a **workspace import job**. Track it from the **Saved Workspaces** panel.

{% hint style="info" %}
Import/export moves configuration only. It does not move data, generated outputs, or database credentials.
{% endhint %}

### Permissions and prerequisites

You need **Editor rights or higher** on the workspace.

Before importing:

* Create the target workspace first. See [Create a workspace](/setup-workspaces/create-a-workspace).
* Make sure the workspace mode matches. See [Workspace modes](/setup-workspaces/create-a-workspace/workspace-modes).
* Make sure the same the target workspace has the same db type for the source database.
* Make sure the same tables/columns exist in the target workspace.
* Make sure the Syntho version matches.

### When to use this

Use import/export to:

* Reuse configurations across environments
* Keep masking and synthesis settings consistent
* Back up a workspace before risky changes
* Restore previous configurations
* Review changes in Git (JSON is readable)
* Share a configuration with other teams

### Export a workspace

Export downloads a zipped JSON file with your workspace configuration.

{% stepper %}
{% step %}

#### Open the workspace

Open the workspace you want to export.
{% endstep %}

{% step %}

#### Start export

Click the **actions menu** next to the workspace name. Select **Export workspace**.
{% endstep %}

{% step %}

#### Save the JSON

Your browser downloads a `.json` file. Store it somewhere versioned if possible.
{% endstep %}
{% endstepper %}

You can also export from the **Workspaces view** by selecting **Export** in the workspace actions menu.

### Import a workspace

Workspace configurations can only be imported into an **existing workspace**.

{% hint style="warning" %}
Import **overwrites** configuration in the target workspace. Export the current state first if you may need to roll back.
{% endhint %}

{% stepper %}
{% step %}

#### Open the target workspace

Open the workspace that should receive the configuration.
{% endstep %}

{% step %}

#### Start import

Click the **actions menu** next to the workspace name. Select **Import workspace**.
{% endstep %}

{% step %}

#### Select the zipped JSON file

Browse and select the exported zipped JSON file. Click **Import**.
{% endstep %}

{% step %}

#### Confirm overwrite

Confirm the overwrite when prompted. This starts a **workspace import job**.
{% endstep %}
{% endstepper %}

Track the job status from [View workspaces](/setup-workspaces/view-workspaces).

### What’s included

The exported file contains all relevant workspace settings, including:

* PII status for each column
* Suggested generators
* Assigned generators
* Consistent mapping settings
* Table inclusion or exclusion
* Table and column-level configurations
* Workspace mode and default settings

The file is human-readable. You can review and edit it before importing.

### What’s not included

Import/export does **not** move:

* Source/destination connection credentials or secrets
* Actual data (source or generated)
* Job run history, logs, or reports

### How import applies changes

* Importing **overrides** the current workspace configuration.
* Only matching tables and columns are updated.
* Missing or unmatched tables/columns are ignored.
* If the workspace mode differs, import can fail or produce unexpected results.
* Import between different Syntho versions is not recommended.

### Best practices

* After upgrading Syntho to latest version, export a workspace before making major changes.
* Keep exports in Git to review diffs and audit changes.
* Import into a staging workspace first.
* Re-run the PII scanner if needed after importing. See [Automatic PII discovery with PII scanner](/configure-a-data-generation-job/privacy-dashboard/automatic-pii-discovery-with-pii-scanner).

### Troubleshooting

* **Nothing changed after import**: the target workspace may not contain matching tables/columns.
* **Mode mismatch**: align modes before importing. See [Workspace modes](/setup-workspaces/create-a-workspace/workspace-modes).
* **Settings behave differently than expected**: compare workspace defaults. See [Workspace default settings](/setup-workspaces/workspace-default-settings).
* **Version mismatch:** upgrade Syntho to latest before exporting. Then do the same before importing, so Syntho versions are the same.


# Main hub

The Main Hub offers a centralized view and configuration board of your database’s PII protection status. It available under the **Main Hub** tab and is designed to help you effortlessly monitor and manage sensitive data across all tables in your workspace.

## Table selection menu

On the left panel, the **Table selection menu** lists all tables and views in your database. You can select all table or views for which the columns will be displayed and configurable from the screen center.

Click **Edit** to choose which tables to include or exclude.

* **Include:** data of included tables will be transferred to the destination database.
* **Exclude:** data of excluded tables will not be transferred.

{% hint style="info" %}
**Hint:** To include or exclude several tables at once, use the `Bulk` option.
{% endhint %}

## Viewing the protection status for each column <a href="#privacy-hub-view-protection-status" id="privacy-hub-view-protection-status"></a>

<figure><img src="/files/YNeEobcV4cq4VP4su3Qj" alt=""><figcaption><p>Overview of protection statuses of columns</p></figcaption></figure>

### Unprotected PII columns

These columns have been identified as containing sensitive PII data but have not yet been protected. They require further action to protect the column, which typically includes assigning a generator.

<figure><img src="/files/0O4WryfZ3DXxSBSSkDUW" alt=""><figcaption><p>Unprotected PII columns</p></figcaption></figure>

#### Confidence scores

Here you also can review PII scan confidence scores to help evaluate the likelihood that a column contains sensitive information. These scores are displayed next to the mockers thus showing the confidence of a mocker for that column.

#### Confidence threshold slider

The confidence threshold slider enables you to fine-tune the sensitivity of PII detection to align with your privacy policies. When you set a specific threshold value, the dashboard will display only the columns whose PII confidence scores meet or exceed this threshold. This makes it easier to quickly apply a suitable generator to protect high-confidence PII columns directly from the interface.

This is especially useful when working with large databases. By setting a threshold value, only columns with a score equal to or greater than that value will be displayed, thus helping you prioritize which columns require protection.

If you click on **Show columns below PII scan acceptance threshold,** you will also see columns that fall below your defined confidence level, allowing for manual review of lower-confidence results as needed.

<figure><img src="/files/E0Bja3qP5wGiehyIX1Ht" alt=""><figcaption><p>PII scan acceptance threshold</p></figcaption></figure>

### Protected PII columns

These columns are already protected using Syntho’s generators. Examples include columns with mock data for birthplace, driving license and maiden names, or an ID column that has Hash applied.

<figure><img src="/files/LEvqclwpxyyfDpDjpFUl" alt=""><figcaption><p>Protected PII columns</p></figcaption></figure>

### Non-PII columns

These are columns the system has classified as not containing PII. While no action is required for these, it’s good practice to periodically review them, especially when the data or schema evolves.

<figure><img src="/files/DUXJoK2J14BTTslDy0nk" alt=""><figcaption><p>Non-PII columns</p></figcaption></figure>

## Marking and umarking columns as PII

On the Main hub **tab**, you can add new columns to the list of **PII** columns, either manually or using Syntho's **PII** scanner. You can manually mark or unmarks columns containing PII via either of below options:

* Select the PII toggle next to the column

<figure><img src="/files/6w9sllYp2E9bgfpLzE9E" alt=""><figcaption><p>Selecting column</p></figcaption></figure>

* Via the Column Parameters modal, select the **Mark as PII** toggle in the top right. Then, click **Confirm** to mark the column as PII and confirm the applied generator selection.

<figure><img src="/files/6OrFtZuVC9BKD5WS7kyr" alt=""><figcaption><p>Select generation method</p></figcaption></figure>

After adding a PII column, it will be listed under the **Main Hub** tab under the respective tab.

On the **Table view** tab, all columns marked as PII will also have a PII icon in the column header.

{% hint style="info" %}
**Reminder**: The PII scanner can help identify many PII columns, but users should manually review and identify any other sensitive columns that might not be detected automatically.
{% endhint %}

## View and configure columns

There are several options to apply and configure generators from the Main hub tab:

1. Open the dropdown in the **Generator** column to choose a generator. This will apply the generator with its default settings.
2. Click the gear icon next to the **Generator** column to open the column settings modal, where you can adjust all generator parameters.
3. Use the **Apply** or **Reject** buttons in the **Actions** menu to apply generator suggestions as based on your selected workspace mode.

### Bulk apply generators

Bulk options allow you to quickly manage multiple schemas, tables, or columns at once. This helps streamline configuration without having to edit each item individually.

#### Schemas and Tables

* **Select All / Unselect All**: Choose or clear all tables for which to display the generators in the center screen of the Main hub.
* **Include All / Exclude All**: Select **Edit** next to the left Tables menu panel to bulk include or excluded schemas or tables.

#### Columns

* **Select All / Unselect All**: Choose or clear all columns within one or more selected tables.

#### Actions on Selected Columns

* **Apply generator**: Assign a generator to all selected columns.
* **Unmark/Mark PII**: Mark selected columns as containing personally identifiable information (PII), or remove the label if not applicable.

#### Generator and Parameter Availability

* Bulk apply generator only exposes generators and parameters that are applicable to **all** selected columns.
  * The more diverse the column types, the fewer generators and parameters will remain available.
  * Example: selecting a mix of text and numeric columns will only display options valid for both.

#### Performance Considerations

* When working with a large number of schemas, tables, or columns, bulk edits may affect performance. At most 50 non-key columns can be selected at once, and 10 non-key columns for bulk applying generators. There is no bulk apply column limit for bulk rejecting or applying suggested generators.


# Automatic PII discovery with PII scanner

On the **Main Hub tab** on the **Job Configuration** panel, launch a **personally identifiable information (PII)** scan that scans all columns in your database for PII.

All positive scan results will show up in the list of PII entities on the **Main hub tab**.

On the **Job Settings** tab, PII entities listed on the **Main hub** tab are also labeled **PII** on the column header.

You can launch a **metadata** PII scan or a **data** PII scan. The metadata scan runs faster, since it is based on the name of an individual column on which regular expression rules are applied. On the other hand, the data scan is likely to be more accurate in detecting PII columns, since it analyses the data inside the column using state of the art natural language processing models.

{% hint style="info" %}
**Hint**: When using the PII scanner, always validate the resulting columns that are marked as PII, because the scanner might mistakenly flag some columns as PII that are not. It might also miss certain PII elements.
{% endhint %}

## Shallow scan (uses metadata)

On the **Main hub** tab, select the dropdown icon right of the **Start scan** button and select **Shallow scan**.

The shallow scan evaluates all columns available in the database and uses regular expression rules to deduce the type of PII each column might contain. This process is optimized for speed and runs in parallel; therefore, prediction accuracy might sometimes be less accurate.

Due to the nature of the metadata scan, results generally have a high confidence score. This is because they rely on rules established by Syntho. It is possible to add new rules to detect custom-defined PII entities. For more details, please contact your Syntho representative.

<figure><img src="/files/i3HnkyKaHR5s5otEdp2M" alt=""><figcaption><p>Shallow scan in Scan mode dropdown</p></figcaption></figure>

## Deep scan (uses metadata + data)

On the **Main hub** tab, select the dropdown icon right of the **Start scan** button and select **Deep scan**.

In some cases, Syntho might not detect PII entities with a shallow scan, especially if the column names aren't descriptive of their content. Creating an exhaustive list of rules is also not always practical. Therefore, Syntho offers an option to scan not just the metadata but also the data within the columns to pinpoint potential PII entities.

Initiating a PII scan first launches a metadata scan. Columns not identified as PII and of type "**string**" or "**text**" are then considered for the deep scan. We restrict the scan to these types because our **natural language processing (NLP)** models are trained to identify and extract PII from textual data, relying on word context for predictions.

{% hint style="warning" %}
**Caution**: The data PII scanner examines the content in each column, meaning the scan duration increases with the size of the database. To cut down on scanning time, you can limit the number of rows read per column. However, this might adversely affect the scan results.
{% endhint %}

In comparison with the metadata scan, data scan results may have a lower confidence level. If a column contains multiple PII types, our software calculates the confidence of the column being of a specific PII type based on how frequently that PII type is detected relative to the total number of rows scanned for that column.

### Limitations (Deep scan only)

* The deep scan examines each column of data using natural language processing (NLP) models, which rely on surrounding context to produce accurate results. However, columns containing Personally Identifiable Information (PII), like a `First_Name` column, typically lack this context. For instance, a `First_Name` column contains only first names, making it challenging for NLP models to accurately identify them as such without additional context.

## Supported PII entities

For more information about the PII entities that Syntho supports, see [Supported PII entities](/configure-a-data-generation-job/manage-personally-identifiable-information-pii/supported-pii-entities).

## PII scanner parameters

### Locale

When running a PII scan, select a locale for the scan. The PII scanner supports English, Dutch, German, and Japanese.

Select the locale that best matches the data being scanned. Use the same locale for mockers suggested by the PII scan.

### Overwrite PII scan results

Enable **Overwrite PII scan results** to rescan columns identified during a previous PII scan.

When disabled, the scanner does not rescan columns previously identified as PII.

* **Cardinality** toggle, if turned on, helps user to check whether there are as many unique values in a column as there are rows. In that case the column most likely contains PII.
* When selecting **Add**, the generation method / column modal will first appear and the user has to adjust/confirm the settings. Afterwards, the **Add** button will disappear and the wheel icon will appear.
* The **Allowlist** enables users to define a list of tokens that should not be marked as PII even if we want to identify other tokens of that entity type.
* The **Add new PII** entity will launch a modal that allows the user to create a new PII entity, by filling in three fields with
  1. a name for the user-defined entity,
  2. a **RegEx** (or list of words),
  3. a confidence percentage.
* The **PII entities** to look for, is a multi-select dropdown (same as schema dropdown) showing all the available PII entities (including entities created by the user).
* The **PII scan acceptance threshold** slider can be used to control the PII entities that are shown to the user.
* The **Learn more about PII** button will forward the user to the PII section in Syntho's User Documentation

Moreover:

Additionally, take into consideration below points:

* If column headers are red for PII columns, it means that there is no Mocker, Mask, Calculated Column or Exclude applied.

<figure><img src="/files/Zyx6nY9w94ELveyJO9uR" alt=""><figcaption><p>Columns "FIRSTNAME" and "MAIDENNAME" is detected as PII but no Mocker, Mask, Calculated Column or Exclude is applied</p></figcaption></figure>

* An exclamation mark (!) next to the table in the left, under table overview panel appears if that table has columns with PII labels on Duplicate (with no Mocker or Exclude applied). The (!) mark informs user that this table has columns that are labelled as Personally Identifiable Information (PII) and if user proceed, this PII will be duplicated, which could lead to unintentional sharing of sensitive data. To avoid this, user have two options:
  1. Apply a Mocker.
  2. Exclude the PII column(s).

Exclamation mark (!) helps user to understand that tables that are marked as de-identify must be de-identified. Please note that **de-identification** is equal to **excluding or mocking PII columns**. When there are PII columns that are not handled by applying a mocker or excluding it, it is at risk. Hence, the PII label is red and the table has an exclamation mark symbol next to it.

## Limitations

* **Scanner Accuracy**: The accuracy of the PII scanner depends on the metadata, data, and PII type. For more accurate PII detection, it is most effective to use descriptive column names like `FirstName` instead of generic names like `Col1`.
* **Multiple PII Detections**: The PII scanner can identify several possible PII entries in a single column. Be aware that the top-scored entry might not always be correct and this could lead to either misidentifying a non-PII item or wrongly categorizing the PII type.
* **Selecting PII Types:** Currently, users cannot specify which types of PII entities to scan for. For a comprehensive list of the types of PII entities that Syntho scans for, please refer to the section [Supported PII entities](/configure-a-data-generation-job/manage-personally-identifiable-information-pii/supported-pii-entities).

Understanding these points will help you better utilize the PII scanner and be aware of its limitations.

{% hint style="info" %}
Caution: The PII scanner is an excellent tool for initial PII detection, but it may not catch all sensitive data. Users are advised to conduct a thorough review to ensure all PII is properly identified.
{% endhint %}


# Table view

The **Table View** displays source data for a single table or view.

To display **Table View**, on the configuration panel, click **Table view**. From **Table View**, you can view and update the table and column configuration.

## Table selection menu

On the left panel, the **Table selection menu** section lists all tables in your database. You can select any table for which the source data will be displayed in the screen center, for which you can then configure the table and column settings.

Click **Edit** to choose which tables to include or exclude.

* **Include:** data of included tables will be transferred to the destination database.
* **Exclude:** data of excluded tables will not be transferred.

{% hint style="info" %}
**Hint:** To include or exclude several tables at once, use the `Bulk` option.
{% endhint %}

## Selecting and configuring a generator for a column

Select the icon next to a column in the table to open the Column Parameters modal, where you can assign and configure the column generator.

When you change the column generator, Syntho updates the protection status panels.

For more information on configuring generators, see [Generators](/configure-a-data-generation-job/configure-column-settings).

## Configure table settings

### Adjust the number of rows to generate

By default, Syntho generates the same number of rows in the destination table as in your source table.

To change the number of rows to generate for a table:

1. Go to **Rows to generate** field in the **Table settings** menu right on the **Table view** panel.
2. Update the field value to the desired number of destination rows.

The behaviour when adjusting the destination table row count is the following:

* For tables that are included:
  * If an **AI synthesize** or **mockers** are applied, Syntho will generate the exact number of rows you specify.
  * If **Duplicate** is applied to any column, it generates the specified number of rows (*`n`*) by duplicating from the original table (*`n_original`*).
    * If *`n`*` ``≤`` `*`n_original`*, the original rows are copied as they are.
    * If *`n`*` ``>`` `*`n_original`*, the original *`n_original`* rows are copied, and any additional rows are randomly sampled (with replacement) from the original rows
* For tables that are excluded, Syntho does not generate any rows (since the table is excluded).

### Considerations for adjusting the number of rows to generate

* The **Rows to generate** field will be disabled if the table doesn't support oversampling, which can be due to the following:
  * The table has another method than **Generate** as the applied key generator method.
* If its number was previously changed and the table doesn't support oversampling anymore, the value will be reverted to the original one.
* Adjusting **Rows to generate** could cause conflicts with foreign key constraints in your destination database.

### Advanced table settings

Unfold **Advanced settings** under the **Table settings** to view and adjust settings on the table-level. Note that these settings will only be relevant for any columns that use AI synthesize.

You can adjust the following advanced table settings:

1. **Maximum rows used for training**: The maximum number of rows to be used for training. Using fewer rows can speed up the process, but may come at the cost of lower synthetic data utility.
2. **Take random sample:**

   * **On**: takes a random sample of rows used for training. Note that choosing this option can cause a data generation job to run significantly longer, depending on the database.
   * **Off (default)**: takes the top rows as defined in the database.

   <figure><img src="/files/vVM6TylMgIm2MxXZ1K2z" alt="" width="398"><figcaption><p>Advanced settings in Table settings view</p></figcaption></figure>
3. **Choose Table Model**: The generative AI model that will be applied to all columns using AI synthesize. This feature allows users to flexibly manage multiple table models by selecting between the following options:

   * **Single table model**
   * **Sequence table model**

   Please note that you can create multiple sequence models as long as the foreign key (FK) relationship limit between the tables is present.

<figure><img src="/files/F7Lln0OFLIKJCk6FTiz5" alt="" width="388"><figcaption><p>Choosing table mode</p></figcaption></figure>

### ORDER BY

{% hint style="info" %}
Hive only
{% endhint %}

In the Table Settings panel, a dropdown field allows users to specify which columns should be used in the "ORDER BY" clause. This feature enables users to define a set of columns that ensure the uniqueness of the returned results for a given table. By selecting the appropriate columns, users can achieve deterministic ordering even in the absence of primary keys or indexes.

* **Order By dropdown**: Located in the Table Settings panel on the right side of the **Table view** tab, this dropdown lets users choose the columns for the "ORDER BY" clause.

**Steps to configure**:

1. Open the **Table settings** panel in the **Table view** tab.
2. Scroll to find the "ORDER BY" dropdown.
3. Select the desired columns from the dropdown to define the order.

**Example scenario**:

* If a table does not have a primary key or index, and the first column contains duplicates, the application may not order the data consistently. By using the new "ORDER BY" dropdown, users can select a combination of columns (e.g., `ColumnA, ColumnB, ColumnC`) that together provide a unique ordering for the table.

To improve the user experience when loading application screens and panels, Syntho has efficient data loading mechanisms. These aim to ensure smoother interaction, especially when the source database contains a significant amount of data.


# Generators

There are two locations from which to change the column generator settings:

1. From the **Main hub** tab.
2. On the **Table view** tab, click on data type icon above a column to open the column settings.

<figure><img src="/files/L4iJuikVRzYdJFjZ6rt2" alt=""><figcaption><p>Data types and their respective icons</p></figcaption></figure>

Here are the articles in this section:

<table data-card-size="large" data-view="cards"><thead><tr><th></th><th data-hidden></th><th data-hidden></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>AI synthesize</strong></td><td></td><td></td><td><a href="/pages/QuqiZ2pZ7fvksb26ZanG">/pages/QuqiZ2pZ7fvksb26ZanG</a></td></tr><tr><td><strong>Mockers</strong></td><td></td><td></td><td><a href="/pages/5zxQAQDDY05BVgAvv4wT">/pages/5zxQAQDDY05BVgAvv4wT</a></td></tr><tr><td><strong>Mask</strong></td><td></td><td></td><td></td></tr><tr><td><strong>Duplicate</strong></td><td></td><td></td><td><a href="/pages/KEGxi2pAoxBWY4mhIlSC">/pages/KEGxi2pAoxBWY4mhIlSC</a></td></tr><tr><td><strong>Exclude</strong></td><td></td><td></td><td><a href="/pages/Kn2zPu6bxZ9drncZgjri">/pages/Kn2zPu6bxZ9drncZgjri</a></td></tr><tr><td><strong>Hash</strong></td><td></td><td></td><td></td></tr><tr><td><strong>Calculated columns</strong></td><td></td><td></td><td><a href="/pages/IGE7PdfKclq1J0KY2yYN">/pages/IGE7PdfKclq1J0KY2yYN</a></td></tr><tr><td><strong>Key generators</strong></td><td></td><td></td><td></td></tr><tr><td><strong>JSON de-identification</strong></td><td></td><td></td><td></td></tr></tbody></table>

To obfuscate PII entities in columns containing free text information, refer to:

<table data-card-size="large" data-view="cards"><thead><tr><th></th><th data-hidden></th><th data-hidden></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Free text PII obfuscation</strong></td><td></td><td></td><td><a href="/pages/SUpdAhf5FosWttzYTvr5">/pages/SUpdAhf5FosWttzYTvr5</a></td></tr></tbody></table>


# AI synthesize

AI synthesize is built for one job: learning patterns from a single entity table.

### When to use AI synthesize

Use AI synthesize when all of these are true:

1. Your input is a single entity table.
2. One row describes one entity.
3. Rows are independent, and row order does not matter.
4. You need realistic statistical patterns, not exact record recovery.
5. You want a synthetic feature dataset for ML or analytics.
6. You want more rows that follow the source distribution.

### When not to use AI synthesize

Do not use AI synthesize for these cases:

1. You need multi-table logic, joins, or cross-table consistency.
2. You need sequence or time-series behavior where order carries meaning.
3. You must preserve rare events, edge cases, or low-frequency patterns.
4. You need hard business rules to hold with 100% certainty.
5. You need reconciliation, regression assertions, or 1:1 traceability.
6. You expect extra synthetic rows to create new real-world information.

{% hint style="warning" %}
AI synthesize is often misunderstood as a general-purpose data generator.

It is not designed to solve every data generation case.

It learns dominant patterns from one entity table.

It does not guarantee full relational logic, exact edge cases, or new signal that is absent from the source.

If you need strict rules or multi-table behavior, use [Mock](/configure-a-data-generation-job/configure-column-settings/mockers), [Mask](/configure-a-data-generation-job/configure-column-settings/mask), and [Calculated columns](/configure-a-data-generation-job/configure-column-settings/calculated-columns), or reshape the source into a single entity table first.
{% endhint %}

## Apply AI synthesize

1. Open your **workspace**.
2. From the **Main hub** or **Table view** tab, select the column where you want to apply a generator.
3. Under **Column parameters** > **Generator,** select AI synthesize to enable Syntho's machine learning (ML) models to automatically synthesize the data in your tables.
4. Set the relevant AI synthesize parameters.
5. Select **Confirm**.

<figure><img src="/files/UlV98TNQHvJzuGcnxr2b" alt="" width="563"><figcaption><p>Selecting generators in column parameters</p></figcaption></figure>

## Preparing your data

When using AI synthesize, it is important that your data is fit to synthesize.

### Entity tables

Syntho expects your data to be stored in **entity tables** that satisfy the following:

* To minimize privacy risks and improve the training algorithm's generalization ability, as a rule of thumb, a minimum **column-to-row ratio of 1:500** is recommended. For example, if your source table has 6 columns, it should contain a minimum of 3000 rows.
* Each entity is described in one row.
* Each row can be treated **independently.**\
  The order of the rows does not convey any information. The contents of one row also do not affect other rows.
* Avoid column names with **privacy-sensitive information**, like`patient_a_medications`, `patient_b_medications`, etc.. Instead, have a patient column with the names. This prevents patient names from being exposed in metadata or bypass rare category protection (e.g., there’s a `patient_a` column, but this patient only appeared five times in the whole dataset).
* Remove columns that are **derived directly from other columns**. For example, you may have a `net_amount` column that is derived from the `gross_amount` and `taxes` columns. For categorical columns, there could be hierarchical relationships, such as a redundant `Treatment category` column referring to a `Treatment` column. Removing such redundant columns will simplify the modeling process and will lead to higher quality synthetic data.

<figure><img src="/files/1F5E2DgaFCqm3tORDCiG" alt=""><figcaption><p>Example of an entity table (each row describes an individual patient, and be treated independently)</p></figcaption></figure>

The Syntho platform supports a wide variety of data types. Under the hood, Syntho uses an encoding scheme where each data type is mapped to one of the following encoding types.

| Data type                                                                                                   | Description                                  |
| ----------------------------------------------------------------------------------------------------------- | -------------------------------------------- |
| [Discrete](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#discrete)       | Numerical counts (e.g. number of visits)     |
| [Continuous](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#continuous)   | Continuous values (e.g. weight, temperature) |
| [Categorical](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#categorical) | Predefined values (e.g. blood type, country) |
| [Datetime](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#datetime)       | Timestamps and dates (e.g. created at)       |

### Discrete

Syntho uses a discrete encoding type to synthesize numerical values that have a countable number of values between any two values. For example, the number of customer complaints or the number of flaws or defects.

### Continuous

To synthesize numerical values that have an infinite number of values between any two values, such as weight and height, Syntho uses a continuous encoding type.

### Categorical

A categorical column has one of a fixed number of possible values. These variables, like the blood type of a person (i.e., `A, B, AB or O`), have a fixed set of categories. Categorical encoding prevents random values (for instance, `M, X or Z`) from appearing in your synthetic dataset.

Under the **Encoding type >** [**Advanced settings**](#advanced-column-settings), the [**Rare category protection** **settings**](#rare-category-protection) will appear, which can be used to protect rare categories. These categories could potentially re-identify outliers within the synthetic data.

{% hint style="info" %}
**Note**: The categorical encoding type is the **default fallback encoding type** used by Syntho. This means that any database types that are unknown by Syntho will automatically be encoded as categorical.
{% endhint %}

### Datetime

The encoding type known as **Datetime** is used to describe values that incorporate either one of, or both a date component and a time component.

By using this encoding type, Syntho is able to synthesize these values and generate dates and times that are statistically valid and representative.

Syntho supports all date and datetime data types for the [**Syntho connectors**](/setup-workspaces/create-a-workspace/connect-to-a-database).

#### Limitations

* Datetime columns support precision up to milliseconds. Nanosecond precision is not supported.

## Rare category protection

Following the privacy-by-design principle, Syntho automatically replaces all rare categorical observations with a user-defined value in a column encoded as a categorical column.

Replacing those rare categories helps to prevent that those sensitive values leak through into the synthetic data.

* **Rare category protection threshold**: All column values that occur as frequently or less than the rare category protection threshold are automatically replaced.
* **Rare category replacement value**: All column values that occur as frequently or less than the rare category protection threshold are automatically replaced by this replacement value.

Under **Column parameters > Encoding type,** select **Advanced settings** to adjust the **Rare category protection threshold** and **Rare category replacement value**.

By default, the **rare category protection threshold** value is set at 10. This means that all column values that occur 10 times or less are automatically replaced by the user-defined value.

Under **Column settings > Encoding type,** select **Advanced settings** to adjust the **Rare category replacement value**.

By default, the **rare category replacement value** is an asterisk (**\***). This means that all values that occur equal or fewer times than the **rare category protection threshold** value will be replaced with the replacement value.

<figure><img src="/files/mJHZOk9Pe2XYkl36S7nv" alt="" width="563"><figcaption><p>Advanced settings for a rare category</p></figcaption></figure>

## Advanced settings

### Advanced generator settings

Go to **Table settings** on the right panel, scroll down to see **Advanced settings** to view and adjust settings on the generator-level. Depending on the job configuration, a generator is applied to one or more columns.

You can adjust the following advanced generator settings:

1. **Maximum rows used for training**: The maximum number of rows to be used for training. Using fewer rows can speed up the process. Leave this value at None to use all rows for training.
2. **Take random sample:**
   * **On**: takes a random sample of rows used for training.
   * **Off**: takes the top rows as defined in the database.

### Advanced column settings

Select **Advanced settings** under **Encoding type** to view and adjust settings on the column-level.

You can adjust the following advanced column settings, depending on the selected encoding type:

#### Discrete | Continuous | Datetime

1. **Clipping threshold:** The floor and ceiling of a column as the *`Nth`* lowest and highest value, where *`N`* is the clipping threshold. The threshold value will process the values as not to exceed the ceiling and floor.

#### Categorical | Text containing PII

1. **Rare category protection threshold**: All column values that occur as frequently or less than the rare category protection threshold are automatically replaced.
2. **Rare category replacement value**: All column values that occur as frequently or less than the rare category protection threshold are automatically replaced by this replacement value.
3. **Locale**: The locale used by the text processing models for columns with text containing PII.


# Additional privacy controls

AI-generated synthetic data offers very high privacy levels. To maximize privacy levels with AI synthesis, Syntho provides a further set of privacy controls:

**1. Overfitting Prevention**

Prevents the model from memorizing specific patterns or properties of the original data, thus enhancing data confidentiality. During the training phase, Syntho minimizes overfitting by applying a so-called [sample noise noise ratio](/setup-workspaces/workspace-default-settings#privacy-control-defaults) that ensures synthetic data reflects general patterns rather than specific entries. The [privacy evaluation](/overview/get-started/ai-generated-synthetic-data#evaluation) metrics can also be used to demonstrate there is no overfitting.

[**2. Rare Category Protection**](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#rare-category-protection)

Protects the uniqueness of categorical data by substituting rare values. Rare categories, defined by a user-set threshold, are replaced with a placeholder (default: "\*"). This prevents overfitting on unique, infrequent categories and protects against potential identification based on rare data points.

[**3. Extreme Value Protection**](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#advanced-column-settings)

Removes outliers in numerical and date-time data to prevent re-identification based on extreme values. Outliers are detected and removed during the preprocessing phase, ensuring that potentially sensitive or identifiable extreme values do not appear in the synthetic data.

[**4. Random Noise Injection**](/setup-workspaces/workspace-default-settings)

Adds random noise to synthetic values to further enhance privacy. Random noise can be injected into generated synthetic data, introducing slight variations that enhance privacy while maintaining data utility. This noise factor (either absolute or relative) is configurable within the [**Workspace default settings**](/setup-workspaces/workspace-default-settings)**.**

[**5. Privacy evaluation**](/overview/get-started/ai-generated-synthetic-data#evaluation)

Syntho leverages open-source synthetic data evaluation libraries like SDMetrics to provide a transparent assessment of synthetic data quality and privacy. The platform includes an evaluation notebook that contains data quality and privacy metrics, allowing you to see how your synthetic data performs against industry standards for confidentiality and utility.


# Limitations

When using AI synthesis, it is always required to prepare your data into a [**single entity table**](/configure-a-data-generation-job/configure-column-settings/ai-powered-generation#entity-tables). This gives the highest utility. You can consider preparing the data into an SQL view.

When working with multiple related tables, AI synthesis is no solution, and it's advisable to use **mock** and **mask generators,** and **calculated columns.** If cross-table logic is important, first consider reshaping the source into a single entity table or SQL view. If that is not feasible, use de-identification instead of AI synthesis.

<table><thead><tr><th width="189">Approach</th><th>Table limit</th><th data-type="checkbox">Cross-table correlations</th><th width="166" data-type="checkbox">Consistency</th><th data-type="checkbox">Capture business rules</th></tr></thead><tbody><tr><td>AI synthesize</td><td>1 feature table</td><td>false</td><td>false</td><td>false</td></tr><tr><td>Mock</td><td>Unlimited</td><td>true</td><td>true</td><td>true</td></tr><tr><td>Mask</td><td>Unlimited</td><td>true</td><td>true</td><td>true</td></tr><tr><td>Calculated columns</td><td>Unlimited</td><td>true</td><td>true</td><td>true</td></tr></tbody></table>


# Mock

**Mock data generators** can be especially useful in the following situations:

* To fill columns that contain directly identifiable information, such as **Personally Identifiable Information (PII)**.
* To fill columns that do not contain any data yet. See [related FAQ question](https://docs.syntho.ai/configure-a-data-generation-job/configure-column-settings/pages/xdw54HbcWXZ4YqB5v30n#i-do-not-have-any-data-yet.-can-i-also-generate-data-from-scratch).

## Apply mock generator

You can apply mock generators in two different manners, via the **Job Configuration** tab, or via the **PII tab**.

### Apply mock generator via Job Configuration

You can apply a mock generator on a column via the **Job Configuration** tab as follows:

1. Open your **workspace**.
2. From the **Main hub** or **Table view** tab, select the column where you want to apply a generator.
3. Under **Generator,** select **Mock** to view the list of available mock generators.
4. Select the mock generator that you wish to apply from the dropdown list of available mock generators.
5. Set the relevant mock generator parameters.
6. Select **Confirm**.

<figure><img src="/files/DmznkxvcTt6PEPzZHA6p" alt="" width="563"><figcaption><p>Selecting data type</p></figcaption></figure>

### Apply mock generator via PII tab

You can apply a mock generator on a column via the **PII tab** as follows: [Broken mention](broken://pages/SKhbH7PpPL9uneS6pW0T).

### Edit mock settings

To edit any mock data settings you have applied previously:

1. Open your **Workspace**.
2. Now you can either:
   1. On the **Job Configuration** tab, select the column icon on the top left of the column where you want to edit a mock generator.
   2. On the **Job Configuration** tab, under **Applied steps**, select the **Edit** icon next to the column name where you want to edit a mock generator.
   3. On the **PII** tab, select the **Edit** icon behind the column where you want to edit a mock generator.
3. Under **Generation Method**, define the parameters that you want to change.
4. Select **Confirm**.

<figure><img src="/files/YgUqymfNFPRiaLqm1DeY" alt="" width="371"><figcaption><p>Applied steps</p></figcaption></figure>

## Available mock generators

There are various mock generators available, each designed to generate mock generator data based on different data types. You can explore them based on their categories, including [Text](/configure-a-data-generation-job/configure-column-settings/mockers/text), [Numeric (integer)](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-integer), [Numeric (decimal)](/configure-a-data-generation-job/configure-column-settings/mockers/numeric-decimal), [Custom Sampler](/configure-a-data-generation-job/configure-column-settings), and [Other](/configure-a-data-generation-job/configure-column-settings/mockers/other).

## Supported languages

Syntho supports each mock generator in multiple different languages. For the complete list of supported languages, see the following section:

{% content-ref url="/pages/J0SUxopaSfx19ZCyH9Ta" %}
[Supported languages](/configure-a-data-generation-job/configure-column-settings/mockers/text/supported-languages)
{% endcontent-ref %}

{% hint style="info" %}
The default language used by each mock generator is English (United States). In case a language is not available for a particular mock generator, the mock generator will revert back to this language.
{% endhint %}

* **Constraints on Key Columns:** Mock generators cannot be applied to primary key or foreign key columns.
* **Column-by-Column Operation:** Mock generatorsfunction on individual columns. At this point, they can't be used to preserve logical relationships across multiple columns.
* **Automatically Cutoff Values:** The Syntho platform automatically cuts off generated mock text values based on the data type's supported length. For example, a mock generator applied on an NVARCHAR(5) column, will cutoff all values beyond the first 5 characters of the text.
* **Automatically Clip Values:** The Syntho platform automatically clips numerical values that exceed the maximum or minimum size, to the largest or smallest value supported by the data type, respectively.
* **No Link with Original Records:** Mock generators do not link back to the original data records, enhancing privacy but potentially reducing the usefulness of the data. If you want to retain the link with the original values, you can enable the [**Consistent Mapping**](#mocker-parameters) feature.
* **Database Type Compatibility:** The return type of a mock generator indicates its compatibility with specific database data types. For example, a mock generator with a text return type is compatible with database types like **(N)VARCHAR** or **TEXT**, but not with a database type **NUMERIC (INTEGER)**. It is important to this into account when applying mock generators on your columns to prevent your data generation job to fail.

Being aware of these limitations and considerations will help you effectively use mock generators while understanding their constraints.


# Text

Below is a list of available text mock data generators.

## Standard mock data generators

| Mocker                        | Description                                                                                  | Localization |
| ----------------------------- | -------------------------------------------------------------------------------------------- | ------------ |
| Aba                           | A 9-digit American Bankers Association routing number used for electronic transactions.      | Yes          |
| Address                       | A street address, including street name and number, city, state/province, and postal code.   | Yes          |
| AM or PM                      | Either 'AM' or 'PM'.                                                                         | Yes          |
| Android platform token        | A 10-character Android platform token.                                                       | No           |
| Ascii company email           | An ASCII company email address.                                                              | Yes          |
| ASCII email                   | An ASCII email address.                                                                      | Yes          |
| ASCII free email              | An ASCII free email address.                                                                 | Yes          |
| ASCII safe email              | An ASCII safe email address.                                                                 | Yes          |
| Bank branch code              | A numeric identifier for a specific bank branch.                                             | No           |
| Bank code                     | A numeric identifier for a bank institution.                                                 | No           |
| Bank country                  | A country that uses the International Bank Account Number (IBAN) system.                     | Yes          |
| Bank location code            | A code representing the geographical location of a bank.                                     | No           |
| BBAN                          | A Basic Bank Account Number (BBAN) used in countries that do not use the IBAN system.        | Yes          |
| BSN                           | A Dutch citizen service number (Burgerservicenummer).                                        | No           |
| Bs                            | A business slogan or catchphrase.                                                            | Yes          |
| Catch phrase                  | A catchphrase or slogan used by a company.                                                   | Yes          |
| City                          | A city name.                                                                                 | Yes          |
| City suffix                   | A common suffix for city names, such as 'town' or 'ville'.                                   | Yes          |
| Color                         | A random color in hexadecimal format.                                                        | Yes          |
| Color (HSL)                   | A random color using the HSL (Hue, Saturation, Lightness) color model.                       | Yes          |
| Color (HSV)                   | A random color using the HSV (Hue, Saturation, Value) color model.                           | Yes          |
| Color (RGB)                   | A random color using the RGB (Red, Green, Blue) color model.                                 | Yes          |
| Color name                    | A color name, such as 'red' or 'green'.                                                      | Yes          |
| Company                       | A company name.                                                                              | Yes          |
| Company description           | A brief summary describing a company's operations or services.                               | Yes          |
| Company email                 | A company email address.                                                                     | Yes          |
| Company suffix                | A common suffix for company names, such as 'Inc.' or 'Ltd.'.                                 | Yes          |
| Country                       | A country name.                                                                              | Yes          |
| Country code                  | A two-letter country code, such as 'US' or 'GB'.                                             | Yes          |
| Credit card number            | A valid credit card number.                                                                  | Yes          |
| Credit card provider          | A credit card provider name, such as 'Visa' or 'MasterCard'.                                 | Yes          |
| Credit card security code     | A security code for a credit card.                                                           | Yes          |
| Cryptocurrency                | A cryptocurrency name.                                                                       | Yes          |
| Cryptocurrency code           | A cryptocurrency code, such as 'BTC' or 'ETH'.                                               | Yes          |
| Cryptocurrency name           | A cryptocurrency name.                                                                       | Yes          |
| Currency                      | A currency name.                                                                             | Yes          |
| Currency code                 | A three-letter currency code, such as 'USD' or 'EUR'.                                        | Yes          |
| Currency name                 | A currency name.                                                                             | Yes          |
| Currency symbol               | A random currency symbol.                                                                    | Yes          |
| Day of week                   | A day of the week, such as 'Monday' or 'Friday'.                                             | Yes          |
| Degree                        | A formal educational degree, such as Bachelor’s or Master’s degree.                          | Yes          |
| DE TIN                        | A German Tax Identification Number (Steuerliche Identifikationsnummer).                      | No           |
| Domain name                   | A domain name, such as 'example.com'.                                                        | Yes          |
| Domain word                   | A single word that could be used in a domain name.                                           | Yes          |
| Driving license               | A government-issued driving license number.                                                  | Yes          |
| ES TIN                        | A Spanish Tax Identification Number (Número de Identificación Fiscal).                       | No           |
| Ethnicity                     | A descriptor for a person’s ethnic background.                                               | Yes          |
| File extension                | A random file extension from a set of common file types.                                     | No           |
| File name                     | A random file name including an extension.                                                   | No           |
| File path                     | A random file path, optionally including a file name at the end.                             | No           |
| Firefox                       | A user agent string for the Firefox browser.                                                 | No           |
| First name                    | A first name.                                                                                | Yes          |
| First name female             | A female first name.                                                                         | Yes          |
| First name male               | A male first name.                                                                           | Yes          |
| First name nonbinary          | A non-binary first name.                                                                     | Yes          |
| Free email                    | A free email address.                                                                        | Yes          |
| Free email domain             | A domain name used for free email addresses.                                                 | Yes          |
| Health plan name              | The name of a health insurance plan.                                                         | Yes          |
| Hex color                     | A hexadecimal color code, such as '#FF0000'.                                                 | Yes          |
| Honorific                     | A formal title used before a person's name, such as Mr. or Dr.                               | Yes          |
| Host name                     | A random host name for a website or a networked device.                                      | Yes          |
| HTTP method                   | An HTTP method, such as 'GET' or 'POST'.                                                     | Yes          |
| IBAN                          | An International Bank Account Number (IBAN).                                                 | Yes          |
| ICD9                          | A code from the International Classification of Diseases, 9th Revision.                      | No           |
| ICD10                         | A code from the International Classification of Diseases, 10th Revision.                     | No           |
| iOS platform token            | An Apple iOS platform token.                                                                 | No           |
| Insurance policy number       | A unique identifier for an insurance policy.                                                 | No           |
| Internet explorer             | A user agent string for the Internet Explorer browser.                                       | No           |
| IPv4                          | An IPv4 address.                                                                             | Yes          |
| IPv4 network class            | The network class of an IPv4 address.                                                        | Yes          |
| IPv4 private                  | A private IPv4 address.                                                                      | Yes          |
| IPv4 public                   | A public IPv4 address.                                                                       | Yes          |
| IPv6                          | An IPv6 address.                                                                             | Yes          |
| ISBN-10                       | An ISBN-10 book identifier.                                                                  | Yes          |
| ISBN-13                       | An ISBN-13 book identifier.                                                                  | Yes          |
| IT TIN                        | An Italian Tax Identification Number (Codice Fiscale).                                       | No           |
| ITIN                          | A U.S. Individual Taxpayer Identification Number.                                            | No           |
| Job                           | A job title, such as "Marketing Manager" or "Software Engineer".                             | Yes          |
| JP TIN                        | A Japanese Tax Identification Number.                                                        | No           |
| Language code                 | A language code, such as "en" for English or "fr" for French, based on ISO 639-1.            | Yes          |
| Language name                 | The name of a language, such as "English" or "French", based on ISO 639-1.                   | Yes          |
| Last name                     | A last name, such as "Smith" or "Garcia".                                                    | Yes          |
| Last name female              | A last name suitable for a female person, such as "Johnson" or "Gonzalez".                   | Yes          |
| Last name male                | A last name suitable for a male person, such as "Smith" or "Rodriguez".                      | Yes          |
| Last name nonbinary           | A last name suitable for a nonbinary person, such as "Taylor" or "Lee".                      | Yes          |
| License plate                 | A randomly generated license plate number.                                                   | Yes          |
| Linux platform token          | A randomly generated Linux platform token.                                                   | No           |
| Linux processor               | A randomly generated Linux processor name.                                                   | No           |
| Locale                        | A randomly generated locale string.                                                          | No           |
| Localized ean13               | A randomly generated localized EAN-13 barcode number.                                        | Yes          |
| Localized ean8                | A randomly generated localized EAN-8 barcode number.                                         | Yes          |
| Mac address                   | A randomly generated MAC address.                                                            | Yes          |
| Mac platform token            | A randomly generated macOS platform token.                                                   | No           |
| Mac processor                 | A randomly generated macOS processor name.                                                   | No           |
| Major                         | An academic major or field of study.                                                         | Yes          |
| Marital status                | A descriptor of a person's marital condition.                                                | Yes          |
| Medical history               | A summary of a patient's past medical events and conditions.                                 | Yes          |
| Medical procedure             | A description of a medical procedure performed.                                              | Yes          |
| Month                         | A randomly generated month number in the range of 1 to 12.                                   | Yes          |
| Month name                    | A randomly generated month name.                                                             | Yes          |
| Name                          | A randomly generated full name.                                                              | Yes          |
| Name female                   | A randomly generated female full name.                                                       | Yes          |
| Name male                     | A randomly generated male full name.                                                         | Yes          |
| Name nonbinary                | A randomly generated non-binary full name.                                                   | Yes          |
| NIF                           | A Spanish Tax Identification Number (Número de Identificación Fiscal).                       | Yes          |
| NINO                          | A UK National Insurance Number.                                                              | Yes          |
| Opera                         | A randomly generated Opera browser user agent string.                                        | No           |
| Passport number               | An identifier found on a passport document.                                                  | No           |
| Phone number                  | A randomly generated phone number.                                                           | Yes          |
| Postcode                      | A randomly generated postal code.                                                            | No           |
| Prefix                        | A randomly generated name prefix.                                                            | Yes          |
| Prefix female                 | A randomly generated female name prefix.                                                     | Yes          |
| Prefix male                   | A randomly generated male name prefix.                                                       | Yes          |
| Prefix nonbinary              | A randomly generated non-binary name prefix.                                                 | Yes          |
| Pricetag                      | A randomly generated price tag in a format like $12.34.                                      | Yes          |
| Pronoun                       | A word used to refer to a person without using their name.                                   | Yes          |
| Random letter                 | A randomly generated letter character.                                                       | Yes          |
| Random lowercase letter       | A randomly generated lowercase letter character.                                             | Yes          |
| Random uppercase letter       | A randomly generated uppercase letter character.                                             | Yes          |
| Religion                      | A descriptor for a person's religious belief or affiliation.                                 | Yes          |
| Rgb color                     | A randomly generated RGB color in the format (r, g, b).                                      | Yes          |
| Rgb css color                 | A randomly generated RGB color in the format "rgb(r, g, b)".                                 | Yes          |
| Ripe id                       | A randomly generated RIPE network identifier.                                                | Yes          |
| Safari                        | A randomly generated Safari browser user agent string.                                       | No           |
| Safe color name               | A randomly generated safe color name.                                                        | Yes          |
| Safe domain name              | A randomly generated safe domain name.                                                       | Yes          |
| Safe email                    | A randomly generated safe email address.                                                     | Yes          |
| Safe hex color                | A randomly generated safe hexadecimal color.                                                 | Yes          |
| Sex type                      | A descriptor indicating a person's biological sex.                                           | Yes          |
| Sexual orientation            | A descriptor for a person's sexual orientation.                                              | Yes          |
| SIN                           | A Canadian Social Insurance Number.                                                          | No           |
| SHA-1                         | A randomly generated SHA-1 hash.                                                             | No           |
| SHA-256                       | A randomly generated SHA-256 hash.                                                           | No           |
| SSN                           | A randomly generated Social Security Number.                                                 | Yes          |
| State                         | A country state                                                                              | Yes          |
| Street address                | A randomly generated street address.                                                         | Yes          |
| Street name                   | A randomly generated street name.                                                            | Yes          |
| Street suffix                 | A randomly generated street suffix.                                                          | Yes          |
| Suffix                        | A randomly generated name suffix.                                                            | Yes          |
| Suffix female                 | A randomly generated female name suffix.                                                     | Yes          |
| Suffix male                   | A randomly generated male name suffix.                                                       | Yes          |
| Suffix nonbinary              | A randomly generated non-binary name suffix.                                                 | Yes          |
| Swift                         | A randomly generated SWIFT/BIC code.                                                         | Yes          |
| Swift11                       | A randomly generated 11-character SWIFT/BIC code.                                            | Yes          |
| Swift8                        | A randomly generated 8-character SWIFT/BIC code.                                             | Yes          |
| Time                          | A random time of day.                                                                        | Yes          |
| Timezone                      | A string representing a timezone, such as "America/New\_York" or "UTC".                      | Yes          |
| Tld                           | A top-level domain name, such as "com", "org", or "net".                                     | Yes          |
| Uri                           | A complete URI, including scheme, hostname, path, query parameters.                          | Yes          |
| Uri extension                 | A file extension, such as ".html" or ".pdf", suitable for use in a URI.                      | Yes          |
| Uri page                      | A URI path component suitable for use as a page name or ID, such as "/about".                | Yes          |
| URL                           | A complete URL, including scheme, hostname, path, query parameters, and fragment identifier. | Yes          |
| User agent                    | A string representing a web browser user agent, such as "Mozilla/5.0".                       | Yes          |
| User name                     | A string representing a username, typically formatted as "firstname.lastname".               | Yes          |
| VAT ID                        | A randomly generated VAT ID, supported for specific locales.                                 | Yes          |
| Vehicle identification number | A unique identifier for a motor vehicle.                                                     | No           |
| Vehicle name                  | The name or model of a vehicle.                                                              | Yes          |
| Windows platform token        | A string representing a version of the Windows operating system, like "Windows NT 10.0".     | No           |
| Year                          | A randomly generated year between 1 AD and the current year.                                 | Yes          |

## Standard mocker parameters

Standard mockers support parameters below:

* **Consistent Mapping**: Ensures that the same original input value always maps to the same mock value when applied repeatedly. This is useful for maintaining relationships between datasets. For more information please check [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).
* **Unique**: Guarantees that all generated values in a column are distinct, which is useful for fields like IDs, usernames, or other uniquely identifying data.

## Custom integer

Generates string values with each placeholder in `Custom integer` field.

#### Parameters

* `Custom integer` : Each placeholder is replaced according to the following rules:
  * Number signs (‘#’) are replaced with a random digit (0 to 9).
  * Percent signs (‘%’) are replaced with a random non-zero digit (1 to 9).
  * Dollar signs (‘$’) are replaced with a random digit above two (2 to 9).
* `Consistent mapping`: Custom integer supports [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).

**Note**: Additionally, any digits between 0 and 9 can be used in the pattern, apart from 0 as the first character. For example, '%#$#' will create a 4 digit number following the rules above.

#### Example <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Custom integer as: %#$#
```

The results will be:

```
1188,
6894,
3870,
...
```

## Custom text

Generates a text with each placeholder in `Custom text` field.

#### Parameters

* `Custom text`: Each placeholder is replaced according to the following rules:
  * Number signs (#) are replaced with a random digit (0 to 9).
  * Question marks (?) are replaced with a random character from `Letters`.
* `Letters` : By default, `Letters` contains all ASCII letters, uppercase and lowercase.
* `Consistent mapping`: Custom text supports [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).

**Note:** When adding a Custom text mocker as part of an initialization mode (when creating a workspace), there is a limit of 100 `?` placeholders in the custom text input.

#### Example <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Custom text as: Product Number: ??-##
Letters as: abc
```

The results will be:

```
Product Number: aa-59,
Product Number: cb-96,
Product Number: ac-96,
...
```

## Date/Time

Generates formatted string representation of a random datetime object between two specified datetime objects, from year until seconds.

#### **Parameters**

* `Start date`: The start date for the generation range. (Format: `%d/%m/%Y %H:%M:%S`)
* `End date`: The end date for the generation range.
* `Blank`: Percentage of missing values (`0 to 100%`).
* `Date format`: Optional parameter to return the result in a custom text format (e.g., `"%d/%m/%Y %H:%M:%S"`).
* `Consistent mapping`: Date/Time supports [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).

#### **Example**

If you configure:

```
03/11/2024 06:00:00 to 05/11/2024 22:00:00

Date Format as: %d/%m/%Y %H:%M:%S
```

The results will be:

```
03/11/2024 15:11:09,
05/11/2024 12:42:19,
04/11/2024 17:56:51
...
```

## Date/Time series

Generates a series of formatted datetime strings at regular intervals based on a specified time unit and step size.

#### **Parameters**

* `Start date`: The starting date or datetime for the series.
* `Blank`: Percentage of missing values (`0 to 100%`).
* `Time unit`: The unit of time for the step increment (`"day"`, `"week"`, `"month"`, `"year"`).
* `Number`: The number of units to step forward per iteration.
* `Date format`: Optional parameter for formatted text output.

#### **Example**

If you configure:

```
Start date: 03/11/2024 06:00:00
Time unit: Day
Time number: 2
Date format: %d/%m/%Y %H:%M:%S
```

The results will be:

```
03/11/2024 06:00:00,
05/11/2024 06:00:00,
07/11/2024 06:00:00,
...
```


# Supported languages

Syntho supports various languages for the Syntho mockers. Please note the following:

{% hint style="info" %}
The fallback language for mockers is English (United States). This means that each mocker will attempt to generate data in the configured language of the workspace mode and if a specific mocker does not support the requested language, it will automatically fall back to English (United States).
{% endhint %}

## Supported locales by mockers

This table lists each mocker alongside the number of supported locales it can generate data for. The locale column shows which language-region combinations are available for each mocker.

| Mocker name            | Supported locales                                                       |
| ---------------------- | ----------------------------------------------------------------------- |
| Aba                    | English, Dutch, German, French, Spanish, and 19 other locales           |
| Address                | English, Dutch, German, French, Spanish, Japanese, and 32 other locales |
| AmPm                   | English, Dutch, German, French, Spanish, Japanese, and 27 other locales |
| AsciiCompanyEmail      | English, German, French, Spanish, Japanese, and 28 other locales        |
| AsciiEmail             | English, German, French, Spanish, Japanese, and 28 other locales        |
| AsciiFreeEmail         | English, German, French, Spanish, Japanese, and 28 other locales        |
| AsciiSafeEmail         | English, German, French, Spanish, Japanese, and 28 other locales        |
| BankCountry            | English, Dutch, German, French, Spanish, and 19 other locales           |
| Bban                   | English, Dutch, German, French, Spanish, and 19 other locales           |
| Bs                     | English, Dutch, German, French, Spanish, Japanese, and 27 other locales |
| CatchPhrase            | English, Dutch, German, French, Spanish, Japanese, and 27 other locales |
| City                   | English, Dutch, German, French, Spanish, Japanese, and 32 other locales |
| CitySuffix             | English, Dutch, German, French, Spanish, Japanese, and 32 other locales |
| Color                  | English, German, French, Spanish, and 18 other locales                  |
| ColorHsl               | English, German, French, Spanish, and 18 other locales                  |
| ColorHsv               | English, German, French, Spanish, and 18 other locales                  |
| ColorName              | English, German, French, Spanish, and 18 other locales                  |
| ColorRgb               | English, German, French, Spanish, and 18 other locales                  |
| Company                | English, Dutch, German, French, Spanish, Japanese, and 27 other locales |
| Company description    | English                                                                 |
| CompanyEmail           | English, German, French, Spanish, Japanese, and 28 other locales        |
| CompanySuffix          | English, Dutch, German, French, Spanish, Japanese, and 27 other locales |
| Coordinate             | English, German, and 4 other locales                                    |
| Country                | English, Dutch, German, French, Spanish, Japanese, and 32 other locales |
| CountryCallingCode     | English, Dutch, German, French, Spanish, Japanese, and 39 other locales |
| CountryCode            | English, Dutch, German, French, Spanish, and 19 other locales           |
| CreditCardNumber       | English, German, and 4 other locales                                    |
| CreditCardProvider     | English, German, and 4 other locales                                    |
| CreditCardSecurityCode | English, German, and 4 other locales                                    |
| Cryptocurrency         | English, Dutch, German, French, Spanish, and 15 other locales           |
| CryptocurrencyCode     | English, Dutch, German, French, Spanish, and 15 other locales           |
| CryptocurrencyName     | English, Dutch, German, French, Spanish, and 15 other locales           |
| Currency               | English, Dutch, German, French, Spanish, and 15 other locales           |
| CurrencyCode           | English, Dutch, German, French, Spanish, and 15 other locales           |
| CurrencyName           | English, Dutch, German, French, Spanish, and 15 other locales           |
| CurrencySymbol         | English, Dutch, German, French, Spanish, and 15 other locales           |
| DayOfWeek              | English, Dutch, German, French, Spanish, Japanese, and 27 other locales |
| DomainName             | English, German, French, Spanish, Japanese, and 28 other locales        |
| DomainWord             | English, German, French, Spanish, Japanese, and 28 other locales        |
| Driving license        | English (US)                                                            |
| Email                  | English, German, French, Spanish, Japanese, and 28 other locales        |
| FirstName              | English, Dutch, German, French, Spanish, Japanese, and 47 other locales |
| FirstNameFemale        | English, Dutch, German, French, Spanish, Japanese, and 47 other locales |
| FirstNameMale          | English, Dutch, German, French, Spanish, Japanese, and 47 other locales |
| FirstNameNonbinary     | English, Dutch, German, French, Spanish, Japanese, and 47 other locales |
| FreeEmail              | English, German, French, Spanish, Japanese, and 28 other locales        |
| FreeEmailDomain        | English, German, French, Spanish, Japanese, and 28 other locales        |
| HexColor               | English, German, French, Spanish, and 18 other locales                  |
| Hostname               | English, German, French, Spanish, Japanese, and 28 other locales        |
| Iban                   | English, Dutch, German, French, Spanish, and 19 other locales           |
| Ipv4                   | English, German, French, Spanish, Japanese, and 28 other locales        |
| Ipv4NetworkClass       | English, German, French, Spanish, Japanese, and 28 other locales        |
| Ipv4Private            | English, German, French, Spanish, Japanese, and 28 other locales        |
| Ipv4Public             | English, German, French, Spanish, Japanese, and 28 other locales        |
| Ipv6                   | English, German, French, Spanish, Japanese, and 28 other locales        |
| JobName                | English, German, French, Spanish, Japanese, and 23 other locales        |
| LanguageName           | English, Dutch, German, French, Spanish, Japanese, and 47 other locales |
| LastName               | English, Dutch, German, French, Spanish, Japanese, and 47 other locales |
| LastNameFemale         | English, Dutch, German, French, Spanish, Japanese, and 47 other locales |
| LastNameMale           | English, Dutch, German, French, Spanish, Japanese, and 47 other locales |
| LastNameNonbinary      | English, Dutch, German, French, Spanish, Japanese, and 47 other locales |
| Latitude               | English, German, and 4 other locales                                    |
| LicensePlate           | English, Dutch, German, French, Spanish, and 30 other locales           |
| LocalizedEan13         | English, German, French, Spanish, Japanese                              |
| LocalizedEan8          | English, German, French, Spanish, Japanese                              |
| Longitude              | English, German, and 4 other locales                                    |
| MacAddress             | English, German, French, Spanish, Japanese, and 28 other locales        |
| Month                  | English, Dutch, German, French, Spanish, Japanese, and 27 other locales |
| MonthName              | English, Dutch, German, French, Spanish, Japanese, and 27 other locales |
| Name                   | English, Dutch, German, French, Spanish, Japanese, and 47 other locales |
| NameFemale             | English, Dutch, German, French, Spanish, Japanese, and 47 other locales |
| NameMale               | English, Dutch, German, French, Spanish, Japanese, and 47 other locales |
| NameNonbinary          | English, Dutch, German, French, Spanish, Japanese, and 47 other locales |
| NINO                   | English (UK)                                                            |
| Passport               | English (US)                                                            |
| Password               | English, German and English (US)                                        |
| PhoneNumber            | English, Dutch, German, French, Spanish, Japanese, and 39 other locales |
| Postcode               | English, Dutch, German, French, Spanish, Japanese, and 32 other locales |
| Prefix                 | English, Dutch, German, French, Spanish, Japanese, and 47 other locales |
| PrefixFemale           | English, Dutch, German, French, Spanish, Japanese, and 47 other locales |
| PrefixMale             | English, Dutch, German, French, Spanish, Japanese, and 47 other locales |
| PrefixNonbinary        | English, Dutch, German, French, Spanish, Japanese, and 47 other locales |
| Pricetag               | English, Dutch, German, French, Spanish, and 15 other locales           |
| RgbColor               | English, German, French, Spanish, and 18 other locales                  |
| RgbCssColor            | English, German, French, Spanish, and 18 other locales                  |
| RipeId                 | English, German, French, Spanish, Japanese, and 28 other locales        |
| SafeColorName          | English, German, French, Spanish, and 18 other locales                  |
| SafeDomainName         | English, German, French, Spanish, Japanese, and 28 other locales        |
| SafeEmail              | English, German, French, Spanish, Japanese, and 28 other locales        |
| SafeHexColor           | English, German, French, Spanish, and 18 other locales                  |
| Ssn                    | English, Dutch, German, French, Spanish, and 48 other locales           |
| State                  | English (US)                                                            |
| StreetAddress          | English, Dutch, German, French, Spanish, Japanese, and 39 other locales |
| StreetName             | English, Dutch, German, French, Spanish, Japanese, and 39 other locales |
| StreetSuffix           | English, Dutch, German, French, Spanish, Japanese, and 39 other locales |
| Suffix                 | English, German, and 10 other locales                                   |
| SuffixFemale           | English, German, and 10 other locales                                   |
| SuffixMale             | English, German, and 10 other locales                                   |
| SuffixNonbinary        | English, German, and 10 other locales                                   |
| Swift                  | English, Dutch, German, French, Spanish, and 19 other locales           |
| Swift11                | English, Dutch, German, French, Spanish, and 19 other locales           |
| Swift8                 | English, Dutch, German, French, Spanish, and 19 other locales           |
| Time                   | English, Dutch, German, French, Spanish, Japanese, and 30 other locales |
| Timezone               | English, Dutch, German, French, Spanish, Japanese, and 30 other locales |
| Tld                    | English, Dutch, German, French, Spanish, Japanese, and 34 other locales |
| UnixTime               | English, Dutch, German, French, Spanish, Japanese, and 30 other locales |
| Uri                    | English, Dutch, German, French, Spanish, Japanese, and 28 other locales |
| UriExtension           | English, Dutch, German, French, Spanish, Japanese, and 28 other locales |
| UriPage                | English, Dutch, German, French, Spanish, Japanese, and 28 other locales |
| Url                    | English, Dutch, German, French, Spanish, Japanese, and 28 other locales |
| UserName               | English, Dutch, German, French, Spanish, Japanese, and 28 other locales |
| VAT ID                 | English, Dutch, German, French, Spanish, Japanese, and 28 other locales |
| Vehicle name           | English                                                                 |
| Year                   | English, Dutch, German, French, Spanish, Japanese, and 30 other locales |

## Language coverage among mockers

A breakdown of each language and the number of mockers that support it.

| Language name                    | Number of mockers |
| -------------------------------- | ----------------: |
| English - (Generic)              |               125 |
| German - (Generic)               |               125 |
| Greek - Greece                   |               103 |
| Russian - Russia                 |               103 |
| Spanish - Traditional            |               102 |
| Spanish - Spain                  |               102 |
| Czech - Czech Republic           |               101 |
| English - United States          |               101 |
| Azerbaijani - Azerbaijan         |               100 |
| French - France                  |               100 |
| Slovak - Slovakia                |               100 |
| Thai (Generic)                   |               100 |
| Thai - Thailand                  |               100 |
| Persian - Iran                   |                91 |
| Spanish - Chile                  |                92 |
| Portuguese - Brazil              |                92 |
| German - Germany                 |                90 |
| Polish - Poland                  |                90 |
| Romanian - Romania               |                90 |
| Italian - Italy                  |                89 |
| Portuguese - Portugal            |                88 |
| German - Austria                 |                85 |
| Turkish - Turkey                 |                85 |
| Hungarian - Hungary              |                84 |
| Croatian - Croatia               |                83 |
| Chinese - China (Simplified)     |                82 |
| Indonesian - Indonesia           |                82 |
| Norwegian - Norway               |                81 |
| Ukrainian - Ukraine              |                81 |
| Finnish - Finland                |                72 |
| Japanese - Japan                 |                74 |
| Chinese - Taiwan (Traditional)   |                74 |
| Danish - Denmark                 |                73 |
| Korean - South Korea             |                73 |
| French - Switzerland             |                71 |
| Swedish - Sweden                 |                71 |
| Slovenian - Slovenia             |                72 |
| English - United Kingdom         |                67 |
| Bulgarian - Bulgaria             |                64 |
| Dutch - Netherlands              |                63 |
| English - New Zealand            |                58 |
| Arabic - Generic                 |                58 |
| Armenian - Armenia               |                56 |
| French - Canada                  |                49 |
| Spanish - Mexico                 |                45 |
| Dutch - Belgium                  |                45 |
| English - Australia              |                44 |
| Hebrew - Israel                  |                43 |
| English - Ireland                |                41 |
| Hindi - India                    |                41 |
| Tamil - India                    |                41 |
| English - India                  |                40 |
| German - Switzerland             |                39 |
| Spanish - Colombia               |                34 |
| Arabic - Palestine               |                34 |
| Nepali - Nepal                   |                31 |
| Georgian - Georgia               |                29 |
| Bosnian - Bosnia and Herzegovina |                29 |
| Lithuanian - Lithuania           |                25 |
| Latvian - Latvia                 |                24 |
| Twi - Ghana                      |                23 |
| Estonian - Estonia               |                23 |
| English - Canada                 |                22 |
| Arabic - Saudi Arabia            |                22 |
| Catalan - Spain                  |                22 |
| English - Thailand               |                21 |
| French - Belgium                 |                21 |
| French - Quebec                  |                21 |
| Irish - Ireland                  |                21 |
| Odia (Oriya) - India             |                21 |
| Zulu - South Africa              |                21 |
| Japanese (Generic)               |                16 |
| Dutch (Generic)                  |                16 |
| Italian - Switzerland            |                 8 |
| English - Bangladesh             |                 8 |
| Arabic - Egypt                   |                10 |
| Arabic - Jordan                  |                 3 |
| Arabic - United Arab Emirates    |                 2 |
| Arabic - Bahrain                 |                 1 |
| Albanian - Albania               |                 1 |
| Vietnamese - Vietnam             |                 1 |
| Danish - Denmark                 |                 1 |
| Greek - Cyprus                   |                 1 |
| Luxembourgish - Luxembourg       |                 1 |
| Maltese - Malta                  |                 1 |


# Numeric (integer)

Below is a list of available numeric (integer) mock data generators.

## Standard mock data generators

| Mocker                | Description                                                   |
| --------------------- | ------------------------------------------------------------- |
| Age                   | A numerical representation of a person's age.                 |
| HTTP status code      | A random HTTP status code from a list of commonly used codes. |
| Port number           | A random network port number used for computer networking.    |
| Random digit          | A randomly generated digit character.                         |
| Random digit not null | A randomly generated non-zero digit character.                |

## Custom integer

Generates integer values with each placeholder in `Custom integer` field.

#### Parameters

* `Custom integer`: Each placeholder is replaced according to the following rules:
  * Number signs (‘#’) are replaced with a random digit (0 to 9).
  * Percent signs (‘%’) are replaced with a random non-zero digit (1 to 9).
  * Dollar signs (‘$’) are replaced with a random digit above two (2 to 9).
* `Consistent mapping`: Custom integer supports [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).

**Note**: Additionally, any digits between 0 and 9 can be used in the pattern, apart from 0 as the first character. For example, '%#$#' will create a 4 digit number following the rules above.

#### Example <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Custom integer as: %#$#
```

The results will be:

```
1188,
6894,
3870,
...
```

## Uniform distribution

Generates a random integer between `Minimum` and `Maximum`.

#### Parameters

* `Minimum` : The minimum value of the integer (inclusive).
* `Maximum`: The maximum value of the integer (exclusive).
* `Consistent mapping`: Uniform distribution supports [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).

#### Example <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Minimum as: 5
Maximum as: 10
```

The results will be:

```
9,
5,
7,
...
```

## Numeric series

Generates numeric (integer) series starting at `Start` with `Steps`.

#### Parameters

* `Start` (int): The minimum value of the numeric (integer) (inclusive).
* `Step` (int): The step at a size numeric (integer) series will continue.
* `Consistent mapping`: Numeric series supports [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).

#### Example <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Start: 3
Step: 5
```

The results will be:

```
2,
7,
12,
...
```


# Numeric (decimal)

Below is a list of available numeric (decimal) mock data generators.

## Standard mock data generators

| Mocker     | Description                                                                |
| ---------- | -------------------------------------------------------------------------- |
| Coordinate | A random geographical coordinate (latitude and longitude).                 |
| Latitude   | A randomly generated latitude coordinate in the range of -90.0 to 90.0.    |
| Longitude  | A randomly generated longitude coordinate in the range of -180.0 to 180.0. |
| Unix time  | A random Unix timestamp, representing seconds since the Unix epoch.        |
| Salary     | A numerical representation of a person's salary.                           |

## Uniform distribution

Generates a random numeric (decimal) between `Minimum` and `Maximum`, with `Decimals` places.

#### Parameters

* `Minimum` : The minimum value of the decimal (inclusive).
* `Maximum` : The maximum value of the decimal (exclusive).
* `Decimals` : The number of decimal places to include in the numeric (decimal) (default is `10`, which generates a numeric (decimal) with up to 10 decimal places).
* `Consistent mapping`: Uniform distribution supports [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).

#### Example <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Minimum as: 1
Maximum as: 10
Decimal as: 3
```

The results will be:

```
9.052,
1.844,
7.416,
...
```


# Datetime

Below is a list of available datetime mock data generators.

## Date

Generates a random datetime object between two specified date objects, from year until days.

#### **Parameters**

* `Start date`: The start date for the generation range (Format: `%d/%m/%Y`).
* `End date`: The end date for the generation range (Format: `%d/%m/%Y`).
* `Consistent mapping`: Date supports [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).

#### Example <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
03/11/2024 to 05/11/2024
```

The results will be:

```
2024-11-04,
2024-11-03,
2024-11-03,
...
```

## Date of birth

Generates a date of birth which is a datetime object.

#### **Parameters**

* `Consistent mapping`: Date supports [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).

#### Example <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Locale: English (United States)
```

The results will be:

```
2025-03-17
2025-03-04
2025-03-13
2025-04-02
...
```

## Date/Time

Generates a random datetime object between two specified datetime objects, from year until seconds.

#### **Parameters**

* `Start date`: The start date for the generation range. (Format: `%d/%m/%Y %H:%M:%S`)
* `End date`: The end date for the generation range.
* `Consistent mapping`: Date/Time supports [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).

#### Example

If you configure:

```
03/11/2024 06:00:00 to 05/11/2024 22:00:00
```

The results will be:

```
2024-11-05 08:52:41.340181,
2024-11-04 11:44:02.440943,
2024-11-03 21:43:52.323659,
...
```

## Date/Time series

Generates a series of datetime values at regular intervals based on a specified time unit and step size.

#### **Parameters**

* `Start date`: The starting date or datetime for the series.
* `Time unit`: The unit of time for the step increment (`"day"`, `"week"`, `"month"`, `"year"`).
* `Number`: The number of units to step forward per iteration.

#### Example <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Start date: 03/11/2024 06:00:00
Time unit: Month
Number: 3
```

The results will be:

```
2024-11-03 06:00:00,
2025-02-03 06:00:00,
2025-05-03 06:00:00,
...
```

## Future date

Generates a future datetime object, always after the current day.

#### **Parameters**

* `Consistent mapping`: Future date supports [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).

#### Example <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Locale: English (United States)
```

The results will be:

```
2025-03-17
2025-03-04
2025-03-13
2025-04-02
...
```


# Other

Below is a list of available other types of mock data generators.

## Standard mock data generators

| Mocker             | Return Type | Description                                                                |
| ------------------ | ----------- | -------------------------------------------------------------------------- |
| Boolean            | bool        | Either True or False.                                                      |
| Latitude longitude | geo         | A tuple of (latitude, longitude) coordinates.                              |
| Pybool             | bool        | A randomly generated Python boolean value.                                 |
| UUID               | uuid        | A string representing a UUID (universally unique identifier) in v4 format. |

## JSON

This mocker uses Faker to generate randomized, complex and a text representation of JSON data, simulating real-world scenarios with specified data types across fields.

#### Parameters

* `Data columns`: Specification for the data structure
* `Number of rows`: Number of rows the returned
* `cls`(json.JSONEncoder): Optional JSON encoder to use for non-standard objects such as datetimes
* `Consistent mapping`: JSON supports [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).

**Note**: For more information, refer to the [Faker documentation](https://faker.readthedocs.io/en/master/providers/faker.providers.misc.html#faker.providers.misc.Provider.json).

#### Example <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```json
{"Name":"name", "Address":"address"}
```

The results will be:

```json
{"Name": "Rebecca Crawford", "Address": "USCGC Harrell\nFPO AP 64614"},
{"Name": "Mark Ayala", "Address": "979 Clay Vista Apt. 789\nNew Zacharymouth, NC 40691"},
{"Name": "Stephanie Chaney", "Address": "852 Debbie Valley\nBrittanystad, FM 41302"}
```

## Custom sampler

Generates random text, numeric (integer or decimal) object depending on provided values, sampled from a provided list of input values.

#### Parameters

* `Values` (comma separated values): The list of values to sample from.
* `...` (File upload): Allows users to import predefined lists of values from `.csv` or `.txt` files. The uploaded file must be within 2MB and contain a maximum of 5000 characters.
* `Consistent mapping`: Custom sampler supports [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).

#### Example <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
values1, values2, values3

1, 25, 99

0.1, 2.5, 99
```

The results will be:

```
values3,
values1,
values2,
...

99,
25,
1,
...

99.0,
0.1,
2.5,
...
```


# Mask

Masking can be especially useful in the following situations:

1. To mask columns that contain directly identifiable information, such as Personally Identifiable Information (PII).
2. To mask columns that contain indirectly identifiable information, such as Birthdate columns.
3. When data needs to remain recognizable in format.

## Apply mask

1. Open your **workspace**.
2. From the **Main hub** or **Table view** tab, select the column where you want to apply a generator.
3. Under **Generator**, select **Mask** to view the list of available mask functions.
4. Select the **Mask** function that you wish to apply from the dropdown list of available mask functions.
5. Set the relevant mask parameters.
6. Select **Confirm**.

<figure><img src="/files/cSS3musOYNM9nFa7nvFb" alt=""><figcaption></figcaption></figure>

### Edit mask settings

To edit any mask settings you have applied previously:

1. Open your **Workspace**.
2. Now you can either:
   1. On the **Job Configuration** tab, select the column icon on the top left of the column where you want to edit a mask function.
   2. On the **Job Configuration** tab, under **Applied steps**, select the **Edit** icon next to the column name where you want to edit a mask function.
   3. On the **PII** tab, select the **Edit** icon behind the column where you want to edit a mask function.
3. Under the **Generation Method**, define the parameters that you want to change.
4. Select **Confirm**.

## Mask functions

Syntho offers various masking functions. Each function is designed to handle different types of sensitive data. You can explore them based on their categories, including [Text](/configure-a-data-generation-job/configure-column-settings/mask/text), [Numeric (integer)](/configure-a-data-generation-job/configure-column-settings/mask/numeric-integer), [Numeric (decimal)](/configure-a-data-generation-job/configure-column-settings/mask/numeric-decimal), [Datetime](/configure-a-data-generation-job/configure-column-settings/mask/datetime) and [UUID](/configure-a-data-generation-job/configure-column-settings/mask/uuid).


# Text

Below is a list of available text mask functions.

## **Format preserving encryption (FPE)**

The **Format preserving encryption (FPE)** function utilizes the FF3 algorithm to encrypt sensitive data while preserving its original format and length. This makes it ideal for fields where the data's structure must remain intact (e.g., credit card numbers or dates). Note that it preserves data format during encryption and supports unique and randomized subsets for varied datasets​. The values are generated in a consistent manner.

#### **Parameters**

* `Alphabet` : By default, it contains all ASCII letters, uppercase and lowercase and numbers.

#### **Example** <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Alphabet: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ 0123456789
```

The results will be:

```
0kVRnFWud,
uCgInrq6,
772edmb G,
...
```

<figure><img src="/files/t2HxriRtSVESLyz1ZCZC" alt=""><figcaption></figcaption></figure>

## **Numeric hasher**

The **Numeric hasher** function provides secure hashing for categorical values. This method replaces original values with a hashed representation, ideal for ensuring data privacy while maintaining referential integrity in numerical datasets. Note that It maintains the original structure of hashed fields. and the minimum number of characters depends on the size of the alphabet used, typically resulting in a minimum of 4 characters in a consistent manner. To ensure accurate ordering, please see [ordering and indexing considerations](/configure-a-data-generation-job/configure-column-settings/consistent-mapping#ordering-and-indexing-considerations).

#### Parameters

* No parameters.

#### Example <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Column names:
Anne,
John,
Chris,
...
```

The results will be:

```
Column names:
3842648596,
3397993411,
70461273,
...
```

<figure><img src="/files/aZLZ41Rf7njTa15Vo1OT" alt=""><figcaption></figcaption></figure>

## **Random character swap**

The **Random character swap** function replaces individual characters in categorical values while preserving the structure of punctuation, spaces, and symbols. Characters are swapped within their respective categories (letters with letters, digits with digits), ensuring that the field's overall format remains usable, or in other words, the original data type and structure of each field (letters, numbers, symbols) are preserved. Note that it preserves non-alphabetic characters (e.g., punctuation, spaces)​.

#### **Parameters**

* `Consistent mapping`: Random character swap supports [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping). When enabled, each input character is always mapped to the same output character. The mapping is case-preserving: if a lowercase character is mapped to another lowercase character, the corresponding uppercase character is mapped to the uppercase version of that same output character. For example, if `a` is mapped to `z`, then `A` is mapped to `Z`.

#### **Example** <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Column names:
Mavis612,
Frank378,
Tijuana228,
...
```

The results will be:

```
Column names:
Eiqxj928,
Wawak904,
Rqrsuzb283,
...
```

<figure><img src="/files/IUGHWVhcevezQ9frG38O" alt="" width="529"><figcaption><p>Consistent mapping enabled for Random character swap</p></figcaption></figure>

## **Shuffle**

The **Shuffle** function shuffles existing text values while preserving the overall frequency of values.

Shuffle works batch-wise. Each generated batch is shuffled independently, based on the configured **Batch size**.

`NULL` values are treated as a distinct value and are shuffled like any other value.

#### **Parameters**

* `Rare Category Protection`: Enables protection for infrequent values during shuffling. Configure its threshold and replacement value.

<figure><img src="/files/dVDarVPpTCOvdcVXBxWq" alt=""><figcaption></figcaption></figure>

## **Rare category protection**

The **Rare category protection** generator replaces infrequent text values with a user-defined replacement value.

Use this standalone Mask generator when you need to protect rare values without shuffling the column. You can also enable it as a parameter of **Shuffle**.

#### **Parameters**

* `Rare category protection threshold`: Values occurring at or below this frequency are replaced. The default is `10`.
* `Rare category replacement value`: The value that replaces rare values. The default is `*`.

{% hint style="warning" %}
Rare category protection works per generated batch. A smaller **Batch size** makes values more likely to meet the threshold.
{% endhint %}

<figure><img src="/files/CHNWfqFcYgBiJKfeQpde" alt=""><figcaption></figcaption></figure>

## **Free text PII obfuscation**

{% hint style="warning" %}
**Caution**: this feature can slow down your data generation jobs significantly.
{% endhint %}

The **Free text PII obfuscation** function detects and obfuscates PII in columns containing free text information.

Select the correct **Locale** to ensure Syntho uses the right language models for detection.

If **Replace PII with mock data** is enabled, detected entities are replaced with mock values. If it is disabled, detected entities are annotated with a PII label.

For setup details, see [Free text PII obfuscation](/configure-a-data-generation-job/configure-column-settings/duplicate/automatic-pii-discovery-and-de-identification-in-free-text-columns). For supported entities, see [Supported PII entities](/configure-a-data-generation-job/manage-personally-identifiable-information-pii/supported-pii-entities).

#### **Parameters**

* `Locale`
* `Replace PII with mock data`

<figure><img src="/files/5zHih4fKoWdmFMNxo6Z2" alt=""><figcaption></figcaption></figure>


# Numeric (integer)

Below is a list of available numeric (integer) mask functions.

## **Hasher**

The Hasher function uses the Hasty Pudding Cipher algorithm to create a one-to-one mapping between input and hashed values, ensuring consistent anonymization. It maintains the sign of numbers, always hashing negative values to negative outputs and positive values to positive outputs, using an internal encoding mechanism. This method ensures stable, deterministic, and repeatable transformations, making it ideal for anonymization while preserving numerical relationships. To ensure accurate ordering, please see [ordering and indexing considerations](/configure-a-data-generation-job/configure-column-settings/consistent-mapping#ordering-and-indexing-considerations).

#### **Parameters**

* No parameters.

**Note**: The default fallback range aligns with 32-bit integer limits (-2,147,486,647 to 2,147,486,647), though actual range depends on database support. Note that 0 is never hashed.

#### **Example** <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Column names:
2002,
1944,
2002,
...
```

The results will be:

```
Column names:
1962697134,
943111608,
1962697134,
...
```

<figure><img src="/files/dF86SLzxxbsBLIrLtfdX" alt="" width="529"><figcaption><p>Hasher</p></figcaption></figure>

## **Numeric Noise**

Adds noise to numeric data based on a uniform distribution, ensuring that the values are randomized while preserving the overall structure of the dataset. This is useful for anonymizing numerical fields where consistency and distribution must be maintained.

#### **Parameters**

* `Maximum negative noise`: The smallest amount the date can be adjusted, relative to the original date.
* `Maximum positive noise`: The largest amount by which the date can be adjusted from the original value.
* `Noise type`: The unit of time (Additive, Multiplicative, Absolute) that will define the granularity of the shift.
  * **Additive**: Syntho adds random noise sampled from a range between –x% and +x% of the absolute value. For example, with a 10% max negative and positive noise percentage, and value 20, noise is sampled from –2 (inclusive) to 2 (exclusive).
  * **Multiplicative**: Syntho multiplies the value by a random factor between the specified Min (inclusive) and Max (exclusive). For example, with values -5 and 5, values are multiplied by a factor between -5 and 5.
  * **Absolute**: Adds random noise directly to the value, within the specified fixed range (e.g., –5 to 5).
* `Consistent mapping`: Numeric Noise supports [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).

#### **Example** <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Maximum negative noise: 3
Maximum positive noise: 3
Noise type: Absolute

2173,
2090,
2227,
...
```

The results may be:

```
2170,
2088,
2227,
...
```

<figure><img src="/files/oOBurC8QCkDZUv5fR9Ts" alt="" width="525"><figcaption><p>Numeric Noise</p></figcaption></figure>

## **Random Character Swap**

The **Random Character Swap** function replaces individual characters in categorical values while preserving the structure of punctuation, spaces, and symbols. Characters are swapped within their respective categories (letters with letters, digits with digits), ensuring that the field's overall format remains usable, or in other words, the original data type and structure of each field (letters, numbers, symbols) are preserved. Note that it preserves non-alphabetic characters (e.g., punctuation, spaces)​.

#### **Parameters**

* `Consistent mapping`: Random Character Swap supports [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).

#### **Example** <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Column names:
Mavis612,
Frank378,
Tijuana228,
...
```

The results will be:

```
Column names:
Eiqxj928,
Wawak904,
Rqrsuzb283,
...
```

<figure><img src="/files/rL7thti661QutkvEjaEL" alt="" width="530"><figcaption><p>Consistent Mapping enabled for Random Character Swap</p></figcaption></figure>


# Numeric (decimal)

Below is a list of available numeric (decimal) mask functions.

## **Numeric Noise**

Adds noise to numeric data based on a uniform distribution, ensuring that the values are randomized while preserving the overall structure of the dataset. This is useful for anonymizing numerical fields where consistency and distribution must be maintained.

#### **Parameters**

* `Maximum negative noise`: The smallest amount the date can be adjusted, relative to the original date.
  * Use negative numbers to shift the date into the past.
    * Example: If the date part is set to "Day" and the minimum shift is set to `-5`, this ensures the date will not be shifted earlier than 5 days prior to the original date.
    * A positive number shifts the date forward.
    * Example: If the minimum shift is `5`, the date will not shift earlier than 5 days after the original date.
* `Maximum positive noise`: The largest amount by which the date can be adjusted from the original value.
  * Use positive numbers to shift the date into the future.
    * Example: If the date part is set to "Day" and the maximum shift is set to `5`, the date will not be shifted later than 5 days after the original date.
* `Noise type`: The unit of time (Additive, Multiplicative, Absolute) that will define the granularity of the shift.
  * Additive: If the data has a value of x, we will add random noise within the range -10% to +10% of x's absolute value.
  * Multiplicative: Multiply the value by a random factor within the range -5 to 5.
  * Absolute: Add direct random noise to the value within the range from -5 to 5.
  * The selected unit will be applied to both the minimum and maximum shift fields
* `Consistent mapping`: Numeric Noise supports [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).

#### **Example** <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Maximum negative noise: 3
Maximum positive noise: 3
Noise type: Absolute

2173,
2090,
2227,
...
```

The results will be:

```
2170,
2088,
2227,
...
```

<figure><img src="/files/xftilYTx48LJz9rWL2PA" alt=""><figcaption></figcaption></figure>


# Datetime

Below is a list of available datetime mask functions.

## **Hasher**

The Hasher anonymizes datetime values by applying a random offset while maintaining consistency. This ensures that the relative differences between timestamps remain intact while preventing direct identification of the original values. To ensure accurate ordering, please see [ordering and indexing considerations](/configure-a-data-generation-job/configure-column-settings/consistent-mapping#ordering-and-indexing-considerations).

#### **Parameters**

* No parameters.

#### **Example** <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Column names:
1951-12-07,
1966-12-26,
1971-09-23,
...
```

The results will be:

```
Column names:
1957-06-18,
1943-07-30,
2015-09-01,
...
```

<figure><img src="/files/79VdD28XqEkIkjqhuJF3" alt=""><figcaption></figcaption></figure>

## **Datetime Noise**

Adds a random shift to a datetime value based on specified minimum and maximum shift parameters.

#### **Parameters**

* `Maximum negative noise`: The smallest amount the date can be adjusted, relative to the original date.
  * Use negative numbers to shift the date into the past.
    * Example: If the date part is set to "Day" and the minimum shift is set to `-5`, this ensures the date will not be shifted earlier than 5 days prior to the original date.
    * A positive number shifts the date forward.
    * Example: If the minimum shift is `5`, the date will not shift earlier than 5 days after the original date.
* `Maximum positive noise`: The largest amount by which the date can be adjusted from the original value.
  * Use positive numbers to shift the date into the future.
    * Example: If the date part is set to "Day" and the maximum shift is set to `5`, the date will not be shifted later than 5 days after the original date.
* `Noise type`: The unit of time (e.g., second, minute, hour day, month, year) that will define the granularity of the shift.
  * The selected unit will be applied to both the minimum and maximum shift fields.
* `Consistent mapping`: Datetime Noise supports [consistent mapping](/configure-a-data-generation-job/configure-column-settings/consistent-mapping).

#### **Example** <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Maximum negative noise: 2
Maximum positive noise: 2
Noise type: Day

1951-12-07,
1966-12-26,
1971-09-23,
...
```

The results will be:

```
1951-12-05,
1966-12-26,
1971-09-23,
...
```

<figure><img src="/files/6QEijpnKkmmCd7CsAy1W" alt="" width="525"><figcaption><p>Datetime Noise</p></figcaption></figure>


# UUID

Below is a list of available UUID mask functions.

## **Hash**

The Hash ensures consistent anonymization by mapping each UUID to a new, uniquely generated UUID. This transformation preserves uniqueness and referential integrity while ensuring that the original values cannot be traced back. Identical input UUIDs will always produce the same hashed output. To ensure accurate ordering, please see [ordering and indexing considerations](/configure-a-data-generation-job/configure-column-settings/consistent-mapping#ordering-and-indexing-considerations).

#### **Parameters**

* No parameters.

#### **Example** <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

If you configure:

```
Column names:
1cb29fe0-cd08-4bc7-8c43-c5f132e8af3b,
5764c99f-e3e7-4a5d-95bb-f667c6b71de9,
5764c99f-e3e7-4a5d-95bb-f667c6b71de9,
...
```

The results will be:

```
Column names:
yWlRcHBNHm34kvegsQveOijBPYV6xO4l,
NjSEpGvyLeBC3x2UU8a OKq5VPYGvGmG,
NjSEpGvyLeBC3x2UU8a OKq5VPYGvGmG,
...
```

<figure><img src="/files/5DAxqhXGJKrxIwOUuk8X" alt="" width="510"><figcaption><p>Hash</p></figcaption></figure>


# Duplicate

Duplicate can be especially useful in the following situations:

1. When data does not contain personally identifiable information (PII) or sensitive elements, duplicating it allows for efficient replication without modification.

## Apply duplicate

1. Open your **Workspace**.
2. From the **Main hub** or **Table view** tab, select the column where you want to apply a generator.
3. Under **Generator,** select **Duplicate** to copy the column from the source table to the destination table *as-is.*
4. Set the relevant duplicate parameters.
5. Select **Confirm**.

<figure><img src="/files/JvTBKml1mLOpZwSu0BW3" alt="" width="563"><figcaption><p>Selecting Duplicate in Generation Method panel</p></figcaption></figure>

{% hint style="info" %}
**Note:** When you duplicate a column, the column is still used during the training process, as it can contain valuable information.

This means, however, that excluding columns *cannot* be used to to reduce hardware requirements or increase the speed of your synthetic data jobs.
{% endhint %}

## Text-specific masking

Text-specific masking options now live under **Mask** for text columns.

Use [Text](/configure-a-data-generation-job/configure-column-settings/mask/text) to configure:

* **Shuffle**
* [Rare Category Protection](/configure-a-data-generation-job/configure-column-settings/mask/text#rare-category-protection)
* [Free text PII obfuscation](/configure-a-data-generation-job/configure-column-settings/duplicate/automatic-pii-discovery-and-de-identification-in-free-text-columns)

## **Ordering and indexing considerations**

To ensure accurate ordering, please see [ordering and indexing considerations](/configure-a-data-generation-job/configure-column-settings/consistent-mapping#ordering-and-indexing-considerations).

## Supported data types

| Generator | Supported data types                                                                          |
| --------- | --------------------------------------------------------------------------------------------- |
| Duplicate | Categorical, Continuous, Discrete, Datetime, Bytes, Bool, UUID, JSON, XML, Geo, Sets, Unknown |


# Free text PII obfuscation

{% hint style="warning" %}
**Caution**: this feature will slow down your data generation jobs significantly. Consider using a GPU or reducing the number of input records to speed up your job.
{% endhint %}

For more information about the PII entities that Syntho supports, see [Supported PII entities](/configure-a-data-generation-job/manage-personally-identifiable-information-pii/supported-pii-entities).

## Use free text PII obfuscation

Use **Free text PII obfuscation** as a **Mask** generator for text columns.

To apply it:

1. Under **Column settings** > **Generator**, select **Mask**.
2. Under **Mask generator**, select **Free text PII obfuscation**.
3. Select the correct **Locale** for the language in your text column.
4. Optionally, enable **Replace PII with mock data**.
   * Enabled: detected PII is replaced with mock values.
   * Disabled: detected PII is annotated with a PII label.

When you apply the PII text scanner to specific columns, Syntho automatically scans for PII elements in those columns. Identified PII elements can then be replaced with mock data. Syntho employs a variety of algorithms and methods to improve the scanning process.

Here's an overview of the steps taken in the detection process, in chronological order:

1. **Regex**: for pattern recognition.
2. **Named Entity Recognition** **(NER)**: to recognize natural language PII entities.
3. **Checksums**: to validate detected patterns.
4. **Context words**: to increase detection certainty.
5. **Label**: to label detected PII entity with a descriptor of the entity.
6. **(Optional) Obfuscate**: to replace detected PII descriptors with mock data.

Under **Encoding type > Locale**, you can define the locale used by the text processing models for text columns containing PII.

Syntho supports detection and de-identification of PII fields for the languages **English** and **Dutch** in columns containing free text data.

Syntho allows adding **NLP (natural language processing)** models with limited support for different languages (see next section).

## Configure to use other NLP models (limited support)

{% hint style="info" %}
**Note**: using non-default NLP models requires having an active internet connection to retrieve those models.
{% endhint %}

Syntho uses NLP engines for two main tasks: NER-based PII identification, and feature extraction for custom rule based logic (such as leveraging context words for improved detection).

By default, with each deployment, Syntho ships the following open-source models from spaCy:

* `en_core_web_md` for English.
* `nl_core_news_md` for Dutch.
* `de_core_news_md` for German.

These models can be replaced by leveraging other NLP models, either public or proprietary. As its internal NLP engine, Syntho supports both [spaCy](https://spacy.io/usage/models) and [Stanza](https://github.com/stanfordnlp/stanza).

This feature can be enabled via the workspace default settings. Hold **CTRL + SHIFT + ALT + 0** to open the **Workspace Default Settings** and enable the model by setting the **model\_name** to any model name as defined in [spaCy](https://spacy.io/usage/models) or [Stanza](https://github.com/stanfordnlp/stanza). For example, to use the English transformer spaCy model:

```
"text_processor_model_settings": {
    "models": [
        {
            "lang_code": "en",
            "model_name": "en_core_web_trf"
        }, ...
    ],
    "nlp_engine_name": "spacy",
     "gpu": false
    }
```

Optionally, if you have configured a GPU in your deployment setup, the `"gpu"` parameter can be set to `true` for faster results.

### Other model requests

Other NLP models, such as [transformer models](https://github.com/huggingface/transformers), can be added on request with limited support from Syntho. It is important to remember that using other models will impact the PII detection flow and its performance.

* **PII Detection and Confidence Score:** The PII text scanner may identify multiple potential **Personally Identifiable Information (PII)** entities within a text column. When this occurs, the entity with the highest confidence score is presented to the user. However, it's important to understand that a high confidence score doesn't guarantee accuracy. This could result in mislabeling the type of PII detected.
* **Internet Requirement for Non-Default NLP Models:** If you opt to use specialized **Natural Language Processing (NLP)** models to accommodate different languages or regions, an active internet connection is necessary to download these models.
* **Detection Methods:** The scanner employs a multi-method approach for PII detection, including the use of **Regex** patterns, **Named Entity Recognition (NER)** models, checksum validation, and examination of context words. Note that the effectiveness of the NER models can vary in different context it's being used. For instance, a NER model trained on Wikipedia text may not perform well when applied to medical data.

By understanding these details, you can better navigate how free text PII obfuscation works and what its limitations may be.


# Exclude

Exclude generation can be especially useful in the following situations:

1. When data neither contain necessary nor useful data.

## Apply Exclude

1. Open your **Workspace**.
2. From the **Main hub** or **Table view** tab, select the column where you want to apply a generator.
3. Under **Generator,** select **Exclude** to exclude the column from the generation job.
4. Select **Confirm**.

<figure><img src="/files/6DuDYZULVyFF9ORhvqwS" alt=""><figcaption><p>Selecting Exclude in Generation Method</p></figcaption></figure>

{% hint style="info" %}
**Note:** When you exclude a column, data will not be read, so it can be used to reduce hardware requirements or increase the speed of your synthetic data jobs, e.g. by excluding large text or blob columns.
{% endhint %}


# Calculated columns

**Calculated Column** formulas can be used to perform a wide range of operations on data, from simple arithmetic to complex logical and statistical computations. Whether you're rounding numbers, extracting portions of dates, calculating averages, or transforming texts, these formulas provide the versatility to generate the data you need.

Here are some typical use cases for **Calculated Column** functions:

1. **Data Cleaning and Transformation**: Effortlessly clean and reformat data, such as trimming whitespace, changing text casing, or converting date formats.
2. **Statistical Analysis**: Perform statistical calculations like averages, variances, or standard deviations to derive insights from numerical data sets.
3. **Logical Operations**: Apply logical tests to data to create flags, indicators, or to filter and categorize data based on specific criteria.
4. **Mathematical Operations**: Execute a variety of mathematical operations, enabling complex calculations like financial modelling or engineering calculations.
5. **Text and Date Manipulation**: Extract or transform portions of text and date fields, which is particularly useful in data preparation for reporting or further analysis.

Note that when a calculated column is applied to generated data, it is processed last. This ensures that all generated data and applied mockers are considered before the calculation occurs, preserving the intended transformations and logic.

## Syntax

**Calculated Columns** are composed of a library of functions and operators used to build formulas and expressions in Syntho. When creating **calculated columns** in Syntho, you must follow specific syntax rules and best practices. Here are the fundamental syntax rules for **Calculated Column** expressions when creating **calculated columns**:

### **Formula Structure**

Formulas are expressions made from:

* **Functions** like `ROUND(...)` or `DATEADD(...)`
* **Column references** like `[AMOUNT]`
* **Constants** like `2`, `"US"`, or `DATE(2024, 12, 31)`
* **Operators** like `+`, `-`, `*`, `/`, `=`, `<>`
* **Mockers** like `MOCK_FIRST_NAME` or `MOCK_FREE_EMAIL_DOMAIN`

### **Column references**

When referring to a column within the same table, you can just use the column name in square brackets: `[ColumnName]`. Only references to columns within the same tables are supported.

### **Functions**

**Calculated Column** includes a wide range of functions that can be used in your expressions, such as time intelligence functions (`DATE`, `DATEADD`), logical functions (`IF`, `AND`, `OR`), and many more. Functions follow the syntax of `FUNCTION_NAME(arguments)`.

### **Mock generators**

If you would like to add a mocker to a calculated column formula, type "`MOCK_`" and you will see the names of mockers in the dropdown list. Either choose from the list or add the name of the mocker to the end of "`MOCK_`", with whitespaces replaced by underscores. Therefore, mockers like "**First name female**" and "**Company Email**" should be entered as "`MOCK_FIRST_NAME_FEMALE`" and "`MOCK_COMPANY_EMAIL`," respectively. A complete list of mockers can be found in the [Mockers ](/configure-a-data-generation-job/configure-column-settings/mockers)section.

When you add a mock generator inside a Calculated Column formula, the settings for that mock generator can be configured in the right-hand screen.

Each mock generator you insert into a formula is automatically given a unique identifier by appending an underscore followed by a number (e.g., `MOCK_FIRST_NAME_1`). This identifier allows you to **reuse** the same mock generator value multiple times within the same formula by referring to it via the assigned identifier.

<figure><img src="/files/kJ1nsshOJRQVOY6z5yN4" alt=""><figcaption></figcaption></figure>

### **Operators**

**Calculated Column** supports various operators for arithmetic calculations (`+`, `-`, `*`, `/`), and comparison (`<`, `>`, `<=`, `>=`, `=`, `<>`) operations.

### **Constants**

You can use constants like numbers (`123`), strings (`"text"`), and dates (`DATE(2020, 12, 31)`).

### **Whitespace and Line Breaks**

Whitespace and line breaks do not affect the interpretation of the formula but can make your formulas easier to read.

### Comments

Certain sections in a formula can be commented out by encapsulating them with `(*` and `*)`. These sections will be ignored when processing the formula. This can be useful when you want to add comments to a formula or when are creating larger complex formulas, for which you want to evaluate that certain sections result in the expected output. If you select a formula section and press `CTRL` + `/` the selected section will be commented out.

### Examples

#### 1) `ROUND` — round numbers to a fixed number of digits

Use `ROUND` when you need consistent decimal precision (for example, currency).

```dax
ROUND(
  [AMOUNT],
  2
)
```

* `number` (number | numeric column): value to round. Example: `[AMOUNT]`.
* `count` (integer): number of digits to keep.
  * `2` keeps two decimals.
  * `0` rounds to whole numbers.
  * `-1` rounds to tens, `-2` to hundreds, etc.

#### 2) `IF` — return one value when a condition is true, else another

Use `IF` to map rows into labels based on a boolean test.

```dax
IF(
  [COUNTRY] = "US",
  "domestic",
  "international"
)
```

* `test` (boolean expression): must evaluate to `TRUE` or `FALSE`. Example: `[COUNTRY] = "US"`.
* `then_value` (any type): returned when `test` is `TRUE`. Example: `"domestic"`.
* `otherwise_value` (any type, optional): returned when `test` is `FALSE`. Example: `"international"`. If omitted, `IF` returns `FALSE`.

#### 3) `DATEADD` — shift a date forward or backward

Use `DATEADD` to create derived dates like “due date” or “follow-up date”.

```dax
DATEADD(
  [ORDER_DATE],
  14,
  "day"
)
```

* `dates` (date/datetime column): base date to shift. Example: `[ORDER_DATE]`.
* `number_of_intervals` (integer): how many units to add or subtract. Example: `14`. Use negative values to subtract.
* `interval` (string): unit for the shift. Use `"year"`, `"quarter"`, `"month"`, or `"day"`.

#### 4) Invoice total (combined) — tax + discount with null-safe inputs

Compute an invoice total that still works when tax rate or discount is missing.

```dax
ROUND(
  ([AMOUNT] * (
    1 + IFNULL(
      [TAX_RATE],
      0
    )
  ))
    - IFNULL(
      [DISCOUNT],
      0
    ),
  2
)
```

* `IFNULL([TAX_RATE], 0)` falls back to `0` when `[TAX_RATE]` is `NULL`.
* `IFNULL([DISCOUNT], 0)` falls back to `0` when `[DISCOUNT]` is `NULL`.
* `ROUND(..., 2)` rounds to 2 decimals for currency-like output.

#### 5) Normalized email (combined) — conditional separators + normalization + generated domain

Build a stable email-like value from name columns, even with missing parts.

```dax
SUBSTITUTE(
  CONCATENATE(
    LOWER(
      TRIM(
        IFNULL(
          [FIRSTNAME],
          ""
        )
      )
    ),
    IF(
      ISNULL([LASTNAME]),
      "",
      "."
    ),
    LOWER(
      TRIM(
        IFNULL(
          [LASTNAME],
          ""
        )
      )
    ),
    "@",
    MOCK_FREE_EMAIL_DOMAIN
  ),
  " ",
  ""
)
```

* `IFNULL(..., "")` prevents `NULL` from nulling the entire result.
* `IF(ISNULL([LASTNAME]), "", ".")` only inserts `.` when a last name exists.
* `TRIM(...)` removes leading and trailing whitespace.
* `LOWER(...)` standardizes casing.
* `SUBSTITUTE(..., " ", "")` removes remaining spaces (for double names).

{% hint style="info" %}
If you want **one generated value reused multiple times** inside a single formula, insert a mocker once and reuse the auto-suffixed name (for example: `MOCK_FREE_EMAIL_DOMAIN_1`).
{% endhint %}

Examples use existing columns and mockers. Results depend on each input row.

<figure><img src="/files/trjTkOwn1ul5wAKgYdZr" alt="" width="353"><figcaption><p>Using calculated column feature</p></figcaption></figure>

### Limitations & considerations

* Column names that begin with an underscore (e.g., `_MyColumn`) will cause “Invalid formula” errors. Rename any columns that start with an underscore so that they begin with a letter or digit instead.
* You can only reference columns within the same table as the calculated column formula. Referencing columns in other tables is not supported.

| Generator          | Supported data types                    |
| ------------------ | --------------------------------------- |
| Calculated Columns | Categorical, Discrete, Continuous, Date |

## Supported calculated column functions

## ABS

Returns the absolute value of a number.

### Syntax

```dax
ABS(number)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>number</td><td>Any number, column or mock value.</td></tr></tbody></table>

### Return value

An absolute value of the number.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">ABS(
  [HEALTHCARE_COVERAGE]
)
</code></pre></td><td>Returns the absolute value of the <code>[HEALTHCARE_COVERAGE]</code> column for each row.</td><td>Varies by row (e.g., <code>[HEALTHCARE_COVERAGE]=-20</code> → <code>20</code>).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ABS(
  -5
)
</code></pre></td><td>Returns the absolute value of a literal number.</td><td><code>5</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ABS(
  MOCK_RANDOM_DIGIT
)
</code></pre></td><td>Returns the absolute value of a generated random digit.</td><td>Varies by row (e.g., <code>MOCK_RANDOM_DIGIT=7</code> → <code>7</code>).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ABS(
  [HEALTHCARE_COVERAGE] - [HEALTHCARE_EXPENSES]
)
</code></pre></td><td>Returns the absolute difference between <code>[HEALTHCARE_COVERAGE]</code> and <code>[HEALTHCARE_EXPENSES]</code> for each row.</td><td>Varies by row (e.g., <code>[HEALTHCARE_COVERAGE]=250</code> and <code>[HEALTHCARE_EXPENSES]=100</code> → <code>150</code>).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  ABS([HEALTHCARE_COVERAGE] - [HEALTHCARE_EXPENSES]) > 100,
  "outlier",
  "ok"
)
</code></pre></td><td>Flags rows where the absolute difference between coverage and expenses exceeds <code>100</code>.</td><td>Either <code>"outlier"</code> or <code>"ok"</code> per row.</td></tr></tbody></table>

## AND

The `AND` function returns `TRUE` if all provided arguments evaluate to `TRUE`, and `FALSE` if any argument evaluates to `FALSE`.

A common use case for the `AND` function is enhancing other functions that perform logical tests. For instance, the `IF` function tests a condition and returns one value if the condition is `TRUE` and another value if it is `FALSE`. By using `AND` as the `logical_test` argument in the `IF` function, you can test multiple conditions simultaneously rather than just one.

### Syntax

```dax
AND(
  Logicalvalue1,
  Logicalvalue2,
  ...Logicalvalue30
)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>Logicalvalue1</td><td>The first condition to evaluate, which must result in either <code>TRUE</code> or <code>FALSE</code>.</td></tr><tr><td>Logicalvalue2</td><td>Additional conditions to evaluate, up to a maximum of 30, each of which must result in either <code>TRUE</code> or <code>FALSE</code>.</td></tr></tbody></table>

### Return value

Returns TRUE if all arguments are TRUE.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">AND(
  [GENDER] = "M",
  [HEALTHCARE_COVERAGE] > 100
)
</code></pre></td><td>Checks if gender is male <strong>and</strong> healthcare coverage exceeds <code>100</code>.</td><td><code>TRUE</code> or <code>FALSE</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">AND(
  [STATE] = "Massachusetts",
  [HEALTHCARE_COVERAGE] > 100,
  [HEALTHCARE_EXPENSES] &#x3C; 500
)
</code></pre></td><td>Checks three conditions at once: state, minimum coverage, and maximum expenses.</td><td><code>TRUE</code> or <code>FALSE</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  AND(
    [AGE] >= 18,
    [AGE] &#x3C; 65
  ),
  "adult",
  "senior_or_minor"
)
</code></pre></td><td>Classifies rows as <code>"adult"</code> when age is between 18 and 64 inclusive.</td><td>Either <code>"adult"</code> or <code>"senior_or_minor"</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">AND(
  NOT([IS_ACTIVE] = 0),
  [STATUS] &#x3C;> "Deleted"
)
</code></pre></td><td>Checks that a row is active and not marked as deleted.</td><td><code>TRUE</code> or <code>FALSE</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  AND(
    ISNULL([ZIP]),
    [COUNTRY] = "US"
  ),
  "missing_us_zip",
  "ok"
)
</code></pre></td><td>Flags US rows with a missing ZIP code.</td><td>Either <code>"missing_us_zip"</code> or <code>"ok"</code> per row.</td></tr></tbody></table>

## BASE

Converts a number into its text representation in the specified radix (base).

### Syntax

```dataweave
BASE(number, Radix, MinimumLength)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>number</td><td>The number, column or mock value to be converted. It must be an integer greater than or equal to 0 and less than 2^53.</td></tr><tr><td>radix</td><td>The base (radix) to convert the number into. It must be an integer between 2 and 36, inclusive.</td></tr><tr><td>MinimumLength</td><td>The minimum length of the returned string. It must be an integer greater than or equal to 0.</td></tr></tbody></table>

### Return value

Converts a positive integer to a specified base into a text from the numbering system.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">BASE(
  5,
  2
)
</code></pre></td><td>Converts the decimal number <code>5</code> to base <code>2</code> (binary).</td><td><code>101</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">BASE(
  5,
  2,
  8
)
</code></pre></td><td>Converts <code>5</code> to base <code>2</code> and left-pads with zeros to length <code>8</code>.</td><td><code>00000101</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">BASE(
  42,
  16
)
</code></pre></td><td>Converts the decimal number <code>42</code> to base <code>16</code> (hexadecimal).</td><td><code>2A</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">BASE(
  [NUMBER],
  [RADIX]
)
</code></pre></td><td>Converts the <code>[NUMBER]</code> column to the base specified in <code>[RADIX]</code> for each row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">BASE(
  RANDBETWEEN(0, 9),
  RANDBETWEEN(2, 36)
)
</code></pre></td><td>Converts a random digit using a random radix. Radix must be between <code>2</code> and <code>36</code>.</td><td>Varies by row (or errors if radix is out of range).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">BASE(
  [NUMBER],
  36,
  6
)
</code></pre></td><td>Converts <code>[NUMBER]</code> to base <code>36</code> and pads the output to length <code>6</code>.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">CONCATENATE(
  "acct_",
  UPPER(
    BASE(
      [ACCOUNT_ID],
      16,
      8
    )
  )
)
</code></pre></td><td>Builds an account-like identifier by converting <code>[ACCOUNT_ID]</code> to padded hex and prefixing with <code>acct_</code>.</td><td>Varies by row.</td></tr></tbody></table>

## CEILING

Rounds a number up, away from zero, to the nearest multiple of a specified significance. The function always rounds a value up, away from zero, regardless of the sign of the number. If the number is an exact multiple of the significance, no rounding is performed. If both the number and significance are negative, the value is rounded down, away from zero. If the number is negative and the significance is positive, the value is rounded up toward zero.

### Syntax

```dax
CEILING(number, Significance)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>number</td><td>The column, number or mock value to be rounded.</td></tr><tr><td>significance</td><td>The multiple to which you want to round the number.</td></tr></tbody></table>

### Return value

A number rounded up to the nearest multiple of Significance.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">CEILING(
  3.1,
  1
)
</code></pre></td><td>Rounds <code>3.1</code> up to the nearest multiple of <code>1</code>.</td><td><code>4</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">CEILING(
  -4.5,
  -2
)
</code></pre></td><td>Rounds <code>-4.5</code> to the nearest multiple of <code>-2</code>, away from zero.</td><td><code>-6</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">CEILING(
  -2.5,
  2
)
</code></pre></td><td>Rounds <code>-2.5</code> to the nearest multiple of <code>2</code>, toward zero.</td><td><code>-2</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">CEILING(
  5.5,
  0.1
)
</code></pre></td><td>Rounds <code>5.5</code> up to the nearest multiple of <code>0.1</code>.</td><td><code>5.5</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">CEILING(
  0.9345,
  0.001
)
</code></pre></td><td>Rounds <code>0.9345</code> up to the nearest multiple of <code>0.001</code>.</td><td><code>0.935</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">CEILING(
  ABS([AMOUNT]),
  0.05
)
</code></pre></td><td>Rounds the absolute amount up to the nearest <code>0.05</code>.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  [AMOUNT] &#x3C; 0,
  -CEILING(
    ABS([AMOUNT]),
    1
  ),
  CEILING(
    [AMOUNT],
    1
  )
)
</code></pre></td><td>Rounds amounts to whole numbers while keeping the original sign.</td><td>Varies by row.</td></tr></tbody></table>

## CLEAN

Removes all nonprintable characters from a given text. The `CLEAN` function is useful for cleaning text imported from other applications that may contain characters not supported by your operating system. For example, it can remove low-level computer code that often appears at the beginning or end of data files and cannot be printed.

{% hint style="info" %}
**Important**

The `CLEAN` function is designed to remove the first 32 nonprinting characters in the 7-bit ASCII code (values 0 through 31). However, in the Unicode character set, there are additional nonprinting characters (values 127, 129, 141, 143, 144, and 157) that the `CLEAN` function does not remove.
{% endhint %}

### Syntax

```dart
CLEAN(text)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>text</td><td>The text, column or mock values from which you want to remove nonprintable characters.</td></tr></tbody></table>

### Return value

The text that has been "cleaned" of line breaks and other non-printable characters.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">CLEAN(
  CHAR(3)
    &#x26; "Patient Medications"
    &#x26; CHAR(17)
)
</code></pre></td><td>Removes non-printable characters from the string, leaving only readable text.</td><td><code>"Patient Medications"</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">CLEAN(
  CHAR(3)
    &#x26; [description]
    &#x26; CHAR(17)
)
</code></pre></td><td>Cleans non-printable characters around the <code>[description]</code> column value.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">CLEAN(
  CHAR(3)
    &#x26; MOCK_FIRST_NAME
    &#x26; CHAR(17)
)
</code></pre></td><td>Cleans non-printable characters around a generated first name.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">CLEAN(
  TRIM([DESCRIPTION])
)
</code></pre></td><td>Trims whitespace and removes non-printable characters from <code>[DESCRIPTION]</code>.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">CLEAN(
  CONCATENATE(
    MOCK_FIRST_NAME,
    " ",
    MOCK_LAST_NAME
  )
)
</code></pre></td><td>Joins a generated first and last name and removes any non-printable characters.</td><td>Varies by row.</td></tr></tbody></table>

## CONCATENATE

The `CONCATENATE` function joins two or more text strings into a single string.

### Syntax

```dax
CONCATENATE("Text1", ..., "Text30")
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>text1</td><td>The first item to join. This can be a text, column or mock values.</td></tr><tr><td>text2, ...text30</td><td>Additional text items to join. You can include up to 30 items, with a total character limit of 8,192.</td></tr></tbody></table>

### Return value

Several text strings combined into one string.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">CONCATENATE(
  [STATE],
  ", ",
  [COUNTY]
)
</code></pre></td><td>Joins state and county into a single string per row.</td><td>Varies by row (e.g., <code>"Massachusetts, Bristol County"</code>).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">CONCATENATE(
  MOCK_FIRST_NAME_FEMALE,
  " ",
  MOCK_LAST_NAME_FEMALE
)
</code></pre></td><td>Builds a full name using mock generators.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">CONCATENATE(
  "Brittany",
  " ",
  "Jenkins"
)
</code></pre></td><td>Joins literal strings into a full name.</td><td><code>"Brittany Jenkins"</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">CONCATENATE(
  UPPER([STATE]),
  "-",
  [COUNTY]
)
</code></pre></td><td>Uppercases the state and joins it with the county, separated by <code>-</code>.</td><td>Varies by row (e.g., <code>"MASSACHUSETTS-Bristol County"</code>).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">CONCATENATE(
  LOWER(
    TRIM([FIRSTNAME])
  ),
  ".",
  LOWER(
    TRIM([LASTNAME])
  ),
  "@",
  MOCK_FREE_EMAIL_DOMAIN
)
</code></pre></td><td>Builds a lowercase email-like string from name columns plus a generated domain.</td><td>Varies by row.</td></tr></tbody></table>

## DATE

The `DATE` function creates a date value by combining individual year, month, and day components. This function is useful for combining separate year, month, and day values into a single date.

### Syntax

```dax
DATE(year, month, day)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>year</td><td>A number, column or mock value representing the year.</td></tr><tr><td>month</td><td>A number, column or mock value column representing the month of the year.</td></tr><tr><td>day</td><td>A number, column or mock value column representing the day of the month.</td></tr></tbody></table>

### Return value

The specified date as the number of full days since nullDate.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

The following examples build date values from literal values, columns, and other functions.

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">DATE(
  2012,
  8,
  24
)
</code></pre></td><td>Creates the date for August 24, 2012.</td><td><code>2012-08-24</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATE(
  "2012",
  "8",
  "24"
)
</code></pre></td><td>Creates the same date, but from string inputs.</td><td><code>2012-08-24</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATE(
  [year],
  [month],
  [day]
)
</code></pre></td><td>Creates a date per row from the <code>[year]</code>, <code>[month]</code>, and <code>[day]</code> columns.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATE(
  YEAR(TODAY()) - 18,
  1,
  1
)
</code></pre></td><td>Creates January 1st of “current year - 18”.</td><td>Varies by run date.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATE(
  [year],
  MONTH(TODAY()),
  1
)
</code></pre></td><td>Creates the first day of the current month, using <code>[YEAR]</code> per row.</td><td>Varies by row and run date.</td></tr></tbody></table>

## DATEADD

Returns a column of dates, shifted forward or backward by the specified number of intervals.

### Syntax

```dax
DATEADD(dates, number_of_intervals, interval)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>dates</td><td>A column or mocker that contains dates.</td></tr><tr><td>number_of_intervals</td><td>A numeric value specifying the number of intervals to add or subtract from the dates. For calendar intervals (<code>year</code>, <code>quarter</code>, <code>month</code>), fractional values are truncated before applying the shift.</td></tr><tr><td>interval</td><td>The interval used to shift the dates. Options include: <code>year</code>, <code>quarter</code>, <code>month</code>, <code>week</code>, <code>day</code>, <code>hour</code>, <code>minute</code>, or <code>second</code>.</td></tr></tbody></table>

### Return value

A column of date values.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">DATEADD(
  [Birthdate],
  20,
  "year"
)
</code></pre></td><td>Shifts <code>[Birthdate]</code> 20 years forward.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATEADD(
  [Birthdate],
  -6,
  "month"
)
</code></pre></td><td>Shifts <code>[Birthdate]</code> 6 months backward.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATEADD(
  [OrderDate],
  14,
  "day"
)
</code></pre></td><td>Shifts <code>[OrderDate]</code> 14 days forward.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATEADD(
  [OrderDate],
  12,
  "hour"
)
</code></pre></td><td>Shifts <code>[OrderDate]</code> 12 hours forward.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATEADD(
  DATEVALUE([ORDER_DATE_TEXT]),
  7,
  "day"
)
</code></pre></td><td>Parses a text date first, then shifts it 7 days forward.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATEADD(
  [BIRTHDATE],
  IF(
    [IS_LEAP_YEAR] = 1,
    1,
    0
  ),
  "year"
)
</code></pre></td><td>Shifts <code>[BIRTHDATE]</code> by 0 or 1 year depending on <code>[IS_LEAP_YEAR]</code>.</td><td>Varies by row.</td></tr></tbody></table>

## DATEVALUE

#### Description

The `DATEVALUE` function converts a date stored as text into a date. Note that the results of the `DATEVALUE` function may vary depending on your computer's system date settings. The `DATEVALUE` function is useful when you have dates in a text format that you want to filter, sort, format as dates, or use in date calculations.

### Syntax

```dax
DATEVALUE(date_text)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>date_text</td><td>A text column, string or mock representing a date.</td></tr></tbody></table>

### Return value

A date that is converted from text to a date in datetime format.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">DATEVALUE(
  [BIRTHDATE]
)
</code></pre></td><td>Converts <code>[BIRTHDATE]</code> stored as text into a date per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATEVALUE("6/24/2012")
</code></pre></td><td>Converts a literal date string into a date value (system locale dependent).</td><td>A date value.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATEVALUE("2012/6/24")
</code></pre></td><td>Converts a literal date string into a date value (system locale dependent).</td><td>A date value.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATEVALUE("24-JUNE-2024")
</code></pre></td><td>Converts a literal date string into a date value.</td><td><code>2024-06-24</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">YEAR(
  DATEVALUE([BIRTHDATE_TEXT])
)
</code></pre></td><td>Extracts the year after converting a text date column.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATEADD(
  DATEVALUE([START_DATE_TEXT]),
  30,
  "day"
)
</code></pre></td><td>Converts <code>[START_DATE_TEXT]</code> to a date, then shifts it 30 days forward.</td><td>Varies by row.</td></tr></tbody></table>

## DAY

The `DAY` function returns the day of a date represented by a serial number, with the day provided as an integer ranging from 1 to 31.

### Syntax

```dax
DAY(number)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th width="535">Description</th></tr></thead><tbody><tr><td>number</td><td>The date, column or mock date for which you want to find the day. Dates should be entered using the <code>DATE</code> function or as results from other formulas or functions.</td></tr></tbody></table>

### Return value

The day of the given date value.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">DAY("24/6/2012")
</code></pre></td><td>Returns the day-of-month from a literal date string.</td><td><code>24</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DAY([BIRTHDATE])
</code></pre></td><td>Returns the day-of-month for <code>[BIRTHDATE]</code> per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DAY(DATE(2012, 8, 24))
</code></pre></td><td>Returns the day-of-month for a constructed date.</td><td><code>24</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DAY(
  DATEADD(
    [BIRTHDATE],
    1,
    "year"
  )
)
</code></pre></td><td>Returns the day-of-month after shifting <code>[BIRTHDATE]</code> 1 year forward.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  DAY([INVOICE_DATE]) = 1,
  "month_start",
  "other_day"
)
</code></pre></td><td>Flags whether <code>[INVOICE_DATE]</code> falls on the first day of a month.</td><td>Either <code>"month_start"</code> or <code>"other_day"</code> per row.</td></tr></tbody></table>

## DAYS

The `DAYS` function calculates the number of days between two dates.

### Syntax

```dataweave
DAYS(date1, date2)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>date1</td><td>The later of the two dates column between which you want to calculate the number of days.</td></tr><tr><td>date2</td><td>The earlier of the two dates column between which you want to calculate the number of days.</td></tr></tbody></table>

### Return value

The difference between two date values.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dart"><code class="lang-dart">DAYS(
  "24/6/2012",
  "14/6/2012"
)
</code></pre></td><td>Returns the number of days between two literal dates.</td><td><code>10</code></td></tr><tr><td><pre class="language-dart"><code class="lang-dart">DAYS(
  [END_DATE],
  [START_DATE]
)
</code></pre></td><td>Returns days between two date columns per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dart"><code class="lang-dart">DAYS(
  DATE(2024, 12, 31),
  DATE(2024, 1, 1)
)
</code></pre></td><td>Returns the days between Jan 1 and Dec 31, 2024.</td><td><code>365</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ABS(
  DAYS([END_DATE], [START_DATE])
)
</code></pre></td><td>Returns the absolute number of days between two date columns.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  DAYS([END_DATE], [START_DATE]) &#x3C;= 30,
  "short_stay",
  "long_stay"
)
</code></pre></td><td>Classifies stays as short (≤ 30 days) or long.</td><td>Either <code>"short_stay"</code> or <code>"long_stay"</code> per row.</td></tr></tbody></table>

## DATEDIFF

The `DateDiff` function returns the number of interval units between two dates.

### Syntax

```dax
DATEDIFF(date1, date2, interval)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>date1, date2</td><td>An integer specifying the number of intervals to add or subtract from the dates.</td></tr><tr><td>interval</td><td><p>A string expression that specifies the time interval you want to use to calculate the difference between <code>date1</code> and <code>date2</code>.</p><p>The value can be one of the following:</p><ul><li>"second"</li><li>"minute"</li><li>"hour"</li><li>"day"</li><li>"week"</li><li>"month"</li><li>"quarter"</li><li>"year"</li></ul></td></tr></tbody></table>

### Return value

The number of units between two dates as defined in the interval.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">DATEDIFF(
  [EXPIRY_DATE],
  [INITIAL_DATE],
  "day"
)
</code></pre></td><td>Returns the number of days between <code>[EXPIRY_DATE]</code> and <code>[INITIAL_DATE]</code> per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATEDIFF(
  [BIRTHDATE],
  TODAY(),
  "year"
)
</code></pre></td><td>Returns the age in years, based on <code>[BIRTHDATE]</code>.</td><td>Varies by row and run date.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATEDIFF(
  DATE(2024, 1, 1),
  DATE(2024, 2, 1),
  "month"
)
</code></pre></td><td>Returns the number of months between Jan 1 and Feb 1, 2024.</td><td><code>1</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATEDIFF(
  DATEVALUE([START_DATE_TEXT]),
  DATEVALUE([END_DATE_TEXT]),
  "day"
)
</code></pre></td><td>Parses two text dates first, then returns the days between them.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  DATEDIFF([SIGNUP_DATE], TODAY(), "day") &#x3C;= 30,
  "new",
  "existing"
)
</code></pre></td><td>Classifies customers as <code>"new"</code> if they signed up in the last 30 days.</td><td>Either <code>"new"</code> or <code>"existing"</code> per row.</td></tr></tbody></table>

## DELTA

The `DELTA` function checks whether two values are equal. It returns `1` if `number1` equals `number2`, and `0` otherwise.

### Syntax

```dataweave
DELTA(number_1, number_2)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>number_1</td><td>The first number, column or mocker.</td></tr><tr><td>number_2</td><td>The second number, column or mocker. If omitted, <code>column2</code> is assumed to be zero.</td></tr></tbody></table>

### Return value

Returns TRUE (1) if both numbers are equal, otherwise returns FALSE (0).

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dart"><code class="lang-dart">DELTA(
  [HEALTHCARE_COVERAGE],
  [HEALTHCARE_EXPENSES]
)
</code></pre></td><td>Returns <code>1</code> when <code>[HEALTHCARE_COVERAGE]</code> equals <code>[HEALTHCARE_EXPENSES]</code>, else <code>0</code>.</td><td><code>0</code> or <code>1</code> per row.</td></tr><tr><td><pre class="language-dart"><code class="lang-dart">DELTA(
  10,
  10
)
</code></pre></td><td>Checks equality of two literal numbers.</td><td><code>1</code></td></tr><tr><td><pre class="language-dart"><code class="lang-dart">DELTA(
  [COPAY_AMOUNT]
)
</code></pre></td><td>Checks whether <code>[COPAY_AMOUNT]</code> equals <code>0</code> (second argument defaults to <code>0</code>).</td><td><code>0</code> or <code>1</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  DELTA([STATUS_CODE], 0) = 1,
  "unknown",
  "known"
)
</code></pre></td><td>Maps rows to <code>"unknown"</code> when <code>[STATUS_CODE]</code> equals <code>0</code>.</td><td>Either <code>"unknown"</code> or <code>"known"</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DELTA(
  ROUND([AMOUNT], 0),
  ROUND([AMOUNT_APPROVED], 0)
)
</code></pre></td><td>Checks equality after rounding <code>[AMOUNT]</code> and <code>[AMOUNT_APPROVED]</code> to whole numbers.</td><td><code>0</code> or <code>1</code> per row.</td></tr></tbody></table>

## EVEN

The `EVEN` function rounds a number up to the nearest even integer. The function always rounds a value up and away from zero, regardless of its sign. If the `number column` is already an even integer, no rounding occurs.

### Syntax

```dax
EVEN(number)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="209">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>number</td><td>The number, column or mocker to be rounded up to the nearest even integer.</td></tr></tbody></table>

### Return value

A positive number rounded up to the next even integer and a negative number rounded down to the next even integer.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">EVEN(
  [HEALTHCARE_COVERAGE]
)
</code></pre></td><td>Rounds <code>[HEALTHCARE_COVERAGE]</code> up (away from zero) to the nearest even integer per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">EVEN(
  3.2
)
</code></pre></td><td>Rounds <code>3.2</code> up to the next even integer.</td><td><code>4</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">EVEN(
  -3.2
)
</code></pre></td><td>Rounds <code>-3.2</code> down (away from zero) to the next even integer.</td><td><code>-4</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">EVEN(
  ROUND([HEALTHCARE_COVERAGE], 0)
)
</code></pre></td><td>Rounds <code>[HEALTHCARE_COVERAGE]</code> to a whole number first, then to the nearest even integer.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  ISODD([SEAT_NUMBER]) = 1,
  EVEN([SEAT_NUMBER] + 1),
  [SEAT_NUMBER]
)
</code></pre></td><td>Makes odd <code>[SEAT_NUMBER]</code> values even by adding <code>1</code>; leaves even values unchanged.</td><td>Varies by row.</td></tr></tbody></table>

## FALSE

Returns the logical value FALSE.

### Syntax

```dax
FALSE()
```

### Return value

The logical value FALSE.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">FALSE()
</code></pre></td><td>Returns the logical value <code>FALSE</code>.</td><td><code>FALSE</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  [IS_ACTIVE] = 1,
  TRUE(),
  FALSE()
)
</code></pre></td><td>Returns <code>TRUE</code> when <code>[IS_ACTIVE]</code> equals <code>1</code>; otherwise returns <code>FALSE</code>.</td><td><code>TRUE</code> or <code>FALSE</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  FALSE(),
  "never",
  "always"
)
</code></pre></td><td>Demonstrates that a constant <code>FALSE()</code> condition always returns the <code>otherwise</code> value.</td><td><code>"always"</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">AND(
  FALSE(),
  [IS_ACTIVE] = 1
)
</code></pre></td><td>Shows that <code>AND</code> returns <code>FALSE</code> when any argument is <code>FALSE</code>.</td><td><code>FALSE</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  FALSE(),
  MOCK_FIRST_NAME,
  MOCK_LAST_NAME
)
</code></pre></td><td>Always returns the <code>otherwise</code> branch, so it generates last names.</td><td>Varies by row.</td></tr></tbody></table>

## FLOOR

The `FLOOR` function rounds a number down toward zero to the nearest multiple of a specified significance. When the `number` is positive, it is rounded down and adjusted toward zero. When the `number` is negative, it is rounded down and adjusted away from zero. If the `number` is already an exact multiple of the `significance`, no rounding occurs.

* If either argument is nonnumeric, the function returns an error message.
* If the `number` is positive and the `significance` is negative, the function returns an error message.

### Syntax

```dax
FLOOR(number, significance)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>number</td><td>The numeric number, column or mocker you want to round down.</td></tr><tr><td>significance</td><td>The multiple to which you want to round the number.</td></tr></tbody></table>

### Return value

Rounds a number down to the nearest multiple of Significance.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

The following formulas round numbers down to the nearest multiple of Significance.

<table><thead><tr><th width="291">Formula</th><th width="333">Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">FLOOR(
  7.7,
  3
)
</code></pre></td><td>Rounds <code>7.7</code> down to the nearest multiple of <code>3</code>.</td><td><code>6</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">FLOOR(
  -3.4,
  -3
)
</code></pre></td><td>Rounds <code>-3.4</code> down to the nearest multiple of <code>-3</code>.</td><td><code>-3</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">FLOOR(
  1.453,
  0.01
)
</code></pre></td><td>Rounds <code>1.453</code> down to the nearest multiple of <code>0.01</code>.</td><td><code>1.45</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">FLOOR(
  [AMOUNT],
  1
)
</code></pre></td><td>Rounds <code>[AMOUNT]</code> down to the nearest whole number per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">FLOOR(
  ABS([AMOUNT]),
  0.5
)
</code></pre></td><td>Rounds the absolute amount down to the nearest multiple of <code>0.5</code>.</td><td>Varies by row.</td></tr></tbody></table>

## HEX2DEC

The `HEX2DEC` function converts a hexadecimal number to its decimal equivalent. If the input is not a valid hexadecimal number, the function returns an error.

### Syntax

```dataweave
HEX2DEC(number)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>number</td><td>The hexadecimal number, column or mocker column to convert. The number can have up to 10 characters (40 bits). The most significant bit represents the sign, and the remaining 39 bits represent the magnitude. Negative numbers are represented using two's-complement notation.</td></tr></tbody></table>

### Return value

The decimal number for the hexadecimal number entered.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dart"><code class="lang-dart">HEX2DEC(
  "CAB007"
)
</code></pre></td><td>Converts a literal hex string to its decimal value.</td><td><code>13283335</code></td></tr><tr><td><pre class="language-dart"><code class="lang-dart">HEX2DEC(
  "FF"
)
</code></pre></td><td>Converts <code>"FF"</code> (hex) to decimal.</td><td><code>255</code></td></tr><tr><td><pre class="language-dart"><code class="lang-dart">HEX2DEC(
  [HEX_VALUE]
)
</code></pre></td><td>Converts the hex value in <code>[HEX_VALUE]</code> to decimal per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">HEX2DEC(
  UPPER([HEX_VALUE])
)
</code></pre></td><td>Uppercases <code>[HEX_VALUE]</code> first, then converts to decimal.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">HEX2DEC(
  RIGHT([HEX_PAYLOAD], 6)
)
</code></pre></td><td>Extracts the last 6 hex characters from <code>[HEX_PAYLOAD]</code>, then converts to decimal.</td><td>Varies by row.</td></tr></tbody></table>

## HOUR

The `HOUR` function returns the hour component of a time value as an integer, ranging from 0 (12:00 A.M.) to 23 (11:00 P.M.). Time values are part of a date value and are represented as decimal numbers (e.g., 12:00 PM is represented as `0.5` because it is half of a day).

### Syntax

```dax
HOUR(time)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>time</td><td>The time number, column or mocker from which you want to extract the hour. This can be entered as a text string within quotation marks (e.g., <code>"6:45 PM"</code>), as a decimal number (e.g., <code>0.78125</code> representing 6:45 PM), or as the result of other formulas or functions (e.g., <code>TIMEVALUE("6:45 PM")</code>).</td></tr></tbody></table>

### Return value

Hour component of given time.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">HOUR(
  "6/24/2012 7:30"
)
</code></pre></td><td>Extracts the hour from a literal datetime string.</td><td><code>7</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">HOUR(
  [START]
)
</code></pre></td><td>Extracts the hour from the <code>[START]</code> datetime column per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">HOUR(
  "6:45 PM"
)
</code></pre></td><td>Extracts the hour from a literal time string.</td><td><code>18</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  HOUR([START]) >= 9,
  "business_hours",
  "off_hours"
)
</code></pre></td><td>Classifies rows based on whether <code>[START]</code> occurs at 09:00 or later.</td><td>Either <code>"business_hours"</code> or <code>"off_hours"</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">HOUR(
  DATEADD([START], 1, "day")
)
</code></pre></td><td>Adds one day to <code>[START]</code> and extracts the hour (same hour, shifted date).</td><td>Varies by row.</td></tr></tbody></table>

## IF

The `IF` function performs a logical test and returns one value if the test evaluates to TRUE and another value if it evaluates to FALSE. This function is commonly used to create conditional formulas.

### Syntax

```dax
IF(test, then value, otherwisevalue)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>test</td><td>The condition you want to test. This can be any expression that returns either TRUE or FALSE.</td></tr><tr><td>then value</td><td>The column or mock value that is returned if the <code>logical_test</code> evaluates to TRUE.</td></tr><tr><td>otherwisevalue</td><td>The column or mock value that is returned if the <code>logical_test</code> evaluates to FALSE. If omitted, the function returns FALSE by default.</td></tr></tbody></table>

### Return value

A logical test result.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  AND(
    [STATE] = "Massachusetts",
    [HEALTHCARE_COVERAGE] > 30
  ),
  MOCK_FIRST_NAME_FEMALE,
  MOCK_FIRST_NAME_MALE
)
</code></pre></td><td>Generates a female first name for MA rows with coverage over <code>30</code>; otherwise generates a male first name.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  [HEALTHCARE_COVERAGE] > 0,
  [HEALTHCARE_COVERAGE]
)
</code></pre></td><td>Returns <code>[HEALTHCARE_COVERAGE]</code> when it’s positive; otherwise returns <code>FALSE</code> (default <code>otherwise</code>).</td><td>Either a number or <code>FALSE</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  [ZIP] = "0000XX",
  "unknown",
  "known"
)
</code></pre></td><td>Maps the sentinel ZIP value <code>"0000XX"</code> to <code>"unknown"</code>, otherwise <code>"known"</code>.</td><td>Either <code>"unknown"</code> or <code>"known"</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  DATEDIFF([BIRTHDATE], TODAY(), "year") >= 18,
  "adult",
  "minor"
)
</code></pre></td><td>Classifies rows as <code>"adult"</code> when age is 18+; otherwise <code>"minor"</code>.</td><td>Either <code>"adult"</code> or <code>"minor"</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  ISNULL([MIDDLE_NAME]),
  CONCATENATE(
    [FIRSTNAME],
    " ",
    [LASTNAME]
  ),
  CONCATENATE(
    [FIRSTNAME],
    " ",
    [MIDDLE_NAME],
    " ",
    [LASTNAME]
  )
)
</code></pre></td><td>Builds a full name, optionally inserting <code>[MIDDLE_NAME]</code> when present.</td><td>Varies by row.</td></tr></tbody></table>

## IFS

The `IFS` function checks multiple conditions and returns a value corresponding to the first condition that evaluates to TRUE. This function simplifies the use of multiple nested `IF` statements, making it easier to read and manage multiple conditions.

To specify a default result, enter `TRUE` as the final condition argument. If none of the other conditions are met, the corresponding value will be returned. For example, if all other conditions fail, a default value can be returned by using `TRUE` as the final condition. If a condition argument is provided without a corresponding value, the function will return an error message. If a condition argument evaluates to something other than TRUE or FALSE, the function returns an error. If none of the conditions evaluate to TRUE, the function returns an error.

### Syntax

```dataweave
IFS(condition1, number1, condition2, number2..., condition_n, number_N)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>condition1</td><td>The condition that is evaluated to determine if it is TRUE or FALSE.</td></tr><tr><td>number1</td><td>The column or mock value returned if <code>condition1</code> evaluates to TRUE. This can be left empty.</td></tr><tr><td>condition2…condition_N</td><td>Additional conditions that are evaluated to determine if they are TRUE or FALSE.</td></tr><tr><td>number2<br>…number_N</td><td>Th column or mock value returned if the corresponding <code>conditionN</code> evaluates to TRUE. Each <code>valueN</code> is associated with its corresponding <code>conditionN</code>. These can also be left empty.</td></tr></tbody></table>

### Return value

A value that corresponds to the first true condition.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dart"><code class="lang-dart">IFS(
  [GENDER] = "F",
  MOCK_NAME_FEMALE,
  [GENDER] = "M",
  MOCK_NAME_MALE
)
</code></pre></td><td>Generates a mock female name for <code>"F"</code> and a mock male name for <code>"M"</code>.</td><td>Varies by row (or errors if no condition matches).</td></tr><tr><td><pre class="language-dart"><code class="lang-dart">IFS(
  [STATE] = "Massachusetts",
  "MA",
  [STATE] = "California",
  "CA",
  TRUE,
  "OTHER"
)
</code></pre></td><td>Maps states to abbreviations, with <code>"OTHER"</code> as a default.</td><td><code>"MA"</code>, <code>"CA"</code>, or <code>"OTHER"</code> per row.</td></tr><tr><td><pre class="language-dart"><code class="lang-dart">IFS(
  [SCORE] >= 90,
  "A",
  [SCORE] >= 80,
  "B",
  [SCORE] >= 70,
  "C",
  TRUE,
  "D"
)
</code></pre></td><td>Assigns a grade based on score thresholds.</td><td><code>"A"</code>, <code>"B"</code>, <code>"C"</code>, or <code>"D"</code> per row.</td></tr><tr><td><pre class="language-dart"><code class="lang-dart">IFS(
  AND(
    [COUNTRY] = "US",
    ISNULL([STATE]) = FALSE
  ),
  "US_WITH_STATE",
  [COUNTRY] = "US",
  "US_MISSING_STATE",
  TRUE,
  "NON_US"
)
</code></pre></td><td>Classifies US/non-US rows and whether a US row has a state value.</td><td><code>"US_WITH_STATE"</code>, <code>"US_MISSING_STATE"</code>, or <code>"NON_US"</code> per row.</td></tr><tr><td><pre class="language-dart"><code class="lang-dart">IFS(
  LEN(TRIM([PHONE])) &#x3C; 10,
  "invalid",
  TRUE,
  "valid"
)
</code></pre></td><td>Validates phone numbers by trimmed length, with a default <code>"valid"</code>.</td><td><code>"invalid"</code> or <code>"valid"</code> per row.</td></tr></tbody></table>

## IFNULL

Returns replacement\_value when value is NULL; otherwise returns value.

### Syntax

```excel-formula
IFNULL(value, replacement_value)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>value</td><td>The value, column or generator you want to check for NULL.</td></tr><tr><td>replacement</td><td>The value, column or generator to replace a NULL value with.</td></tr></tbody></table>

### Return value

Returns the specified replacement value if the observed value is NULL.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">IFNULL(
  [ZIP],
  "0000XX"
)
</code></pre></td><td>Replaces missing ZIP codes with <code>"0000XX"</code>.</td><td><code>[ZIP]</code> or <code>"0000XX"</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IFNULL(
  [MIDDLE_NAME],
  ""
)
</code></pre></td><td>Replaces a missing middle name with an empty string.</td><td><code>[MIDDLE_NAME]</code> or <code>""</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IFNULL(
  MOCK_POSTCODE,
  "0000XX"
)
</code></pre></td><td>Replaces a missing generated postcode value with a fallback.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IFNULL(
  TRIM([CITY]),
  "Unknown"
)
</code></pre></td><td>Trims <code>[CITY]</code>, then replaces a missing value with <code>"Unknown"</code>.</td><td><code>[CITY]</code> (trimmed) or <code>"Unknown"</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  ISNULL([ZIP]),
  "0000XX",
  IFNULL([ZIP], "0000XX")
)
</code></pre></td><td>Uses <code>ISNULL</code> and <code>IFNULL</code> together to guarantee a ZIP fallback.</td><td><code>[ZIP]</code> or <code>"0000XX"</code> per row.</td></tr></tbody></table>

## ISNULL

The `ISNULL` function checks whether a value is NULL and returns `TRUE` if the cell is NULL. If the value contains any data, including a formula that returns an empty string (`""`), `ISNULL` returns `FALSE`.

### Syntax

```excel-formula
ISNULL(value)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>value</td><td>The value, column or mocker you want to check for NULL.</td></tr></tbody></table>

### Return value

`TRUE` if the value is NULL; `FALSE` if it contains any data.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">ISNULL([ZIP])
</code></pre></td><td>Checks whether <code>[ZIP]</code> is NULL.</td><td><code>TRUE</code> or <code>FALSE</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ISNULL([MIDDLE_NAME])
</code></pre></td><td>Checks whether <code>[MIDDLE_NAME]</code> is NULL.</td><td><code>TRUE</code> or <code>FALSE</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  ISNULL([ZIP]),
  "missing",
  "present"
)
</code></pre></td><td>Maps missing ZIP codes to <code>"missing"</code>, otherwise <code>"present"</code>.</td><td>Either <code>"missing"</code> or <code>"present"</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">NOT(ISNULL([EMAIL]))
</code></pre></td><td>Returns <code>TRUE</code> when <code>[EMAIL]</code> is present (not NULL).</td><td><code>TRUE</code> or <code>FALSE</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">AND(
  ISNULL([PHONE]),
  ISNULL([MOBILE_PHONE])
)
</code></pre></td><td>Checks whether both phone fields are NULL.</td><td><code>TRUE</code> or <code>FALSE</code> per row.</td></tr></tbody></table>

## ISODD

The `ISODD` function checks whether a number is odd. It returns `1` if the number is odd, and `0` if the number is even. This function is useful for determining the parity of a number within a table. If the number is nonnumeric, `ISODD` returns an error.

### Syntax

```dax
ISODD(value)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>value</td><td>The value or column to test. If the number is not an integer, it is truncated.</td></tr></tbody></table>

### Return value

`0` if the value is an even integer; `1` if the value is odd.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">ISODD([HEALTHCARE_EXPENSES])
</code></pre></td><td>Checks whether <code>[HEALTHCARE_EXPENSES]</code> is odd.</td><td><code>0</code> or <code>1</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ISODD(7)
</code></pre></td><td>Checks whether the literal number <code>7</code> is odd.</td><td><code>1</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ISODD([HEALTHCARE_EXPENSES] + 1)
</code></pre></td><td>Adds <code>1</code> to the expenses value, then checks if the result is odd.</td><td><code>0</code> or <code>1</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  ISODD([INVOICE_ID]) = 1,
  "odd_id",
  "even_id"
)
</code></pre></td><td>Labels invoice ids by parity.</td><td>Either <code>"odd_id"</code> or <code>"even_id"</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ISODD(
  QUOTIENT(
    [TOTAL_MINUTES],
    60
  )
)
</code></pre></td><td>Converts minutes to whole hours and checks if the hour count is odd.</td><td><code>0</code> or <code>1</code> per row.</td></tr></tbody></table>

## LEFT

The `LEFT` function returns a specified number of characters from the left side of a string. To determine the number of characters in a string, use the `LEN` function.

### Syntax

```dax
LEFT("text", number)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>"text"</td><td>The string, column or mocker from which the leftmost characters are returned. If the string contains <code>Null</code>, the function returns <code>Null</code>.</td></tr><tr><td>number</td><td>A numeric expression or column indicating how many characters to return. If <code>0</code>, a zero-length string (<code>""</code>) is returned. If the length is greater than or equal to the number of characters in the string, the entire string is returned.</td></tr></tbody></table>

### Return value

The input text capped by a specified character length.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">LEFT([STATE], MOCK_RANDOM_DIGIT)
</code></pre></td><td>Returns a random-length left substring from <code>[STATE]</code> per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LEFT("Massachusetts", 4)
</code></pre></td><td>Returns the first 4 characters of a literal string.</td><td><code>"Mass"</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LEFT([STATE], 0)
</code></pre></td><td>Returns the first 0 characters, which is an empty string.</td><td><code>""</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LEFT(TRIM([STATE]), 2)
</code></pre></td><td>Trims <code>[STATE]</code> first, then returns the first 2 characters.</td><td>Varies by row (e.g., <code>"Ma"</code>).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LEFT(CONCATENATE([CITY], ", ", [STATE]), 12)
</code></pre></td><td>Builds <code>"City, State"</code> then returns the first 12 characters.</td><td>Varies by row.</td></tr></tbody></table>

## LEN

The `LEN` function returns a `Long` value representing the number of characters in a string or the number of bytes required to store a variable.

### Syntax

```dax
LEN("text")
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>"text"</td><td>Any valid string expression, column or mocker.</td></tr></tbody></table>

### Return value

Length of a given text.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">LEN([BIRTHPLACE])
</code></pre></td><td>Returns the number of characters in <code>[BIRTHPLACE]</code> per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LEN("Syntho")
</code></pre></td><td>Returns the number of characters in a literal string.</td><td><code>6</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LEN(CONCATENATE([CITY], ", ", [STATE]))
</code></pre></td><td>Builds <code>"City, State"</code> then returns the character count.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LEN(TRIM([ADDRESS]))
</code></pre></td><td>Trims whitespace from <code>[ADDRESS]</code> then returns the character count.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(LEN([PHONE]) &#x3C; 10, "invalid", "valid")
</code></pre></td><td>Validates phone numbers based on their length.</td><td>Either <code>"invalid"</code> or <code>"valid"</code> per row.</td></tr></tbody></table>

## LN

The `LN` function returns the natural logarithm of a specified number. Natural logarithms are based on the mathematical constant `e` (approximately 2.71828182845904). The `LN` function is the inverse of the `EXP` function.

### Syntax

```dax
LN(number)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>number</td><td>The positive real number, column or mocker for which you want to calculate the natural logarithm.</td></tr></tbody></table>

### Return value

The natural logarithm of the number.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">LN([REVENUE])
</code></pre></td><td>Returns the natural log of <code>[HEALTHCARE_EXPENSES]</code> per row.</td><td>Varies by row (and errors for non-positive values).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LN(10)
</code></pre></td><td>Returns the natural log of a literal number.</td><td><code>~2.3026</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LN([HEALTHCARE_EXPENSES] + 1)
</code></pre></td><td>Adds <code>1</code> to avoid <code>LN(0)</code> for zero values, then logs the result.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LN(ABS([REVENUE]))
</code></pre></td><td>Takes the absolute value first, then computes the natural log.</td><td>Varies by row (and errors when the absolute value is <code>0</code>).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF([HEALTHCARE_EXPENSES] > 0, LN([REVENUE]), 0)
</code></pre></td><td>Computes <code>LN</code> only for positive values; otherwise returns <code>0</code>.</td><td>Either <code>0</code> or a log value per row.</td></tr></tbody></table>

## LOG

The `LOG` function returns the logarithm of a specified number to a base that you define.

### Syntax

```dax
LOG(number, base)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>number</td><td>The positive real number, column or mocker for which you want to calculate the logarithm.</td></tr><tr><td>base</td><td>The base of the logarithm. If omitted, the base is assumed to be 10.</td></tr></tbody></table>

### Return value

The logarithm of a number to the specified base.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">LOG([REVENUE], 2)
</code></pre></td><td>Returns the base-2 log of <code>[HEALTHCARE_EXPENSES]</code> per row.</td><td>Varies by row (and errors for non-positive values).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LOG(100)
</code></pre></td><td>Returns the base-10 log of <code>100</code> (default base).</td><td><code>2</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LOG([REVENUE], [LOG_BASE])
</code></pre></td><td>Returns the log using a per-row base from <code>[LOG_BASE]</code>.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LOG([REVENUE])
</code></pre></td><td>Returns the base-10 log of <code>[HEALTHCARE_EXPENSES]</code> per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(LOG([REVENUE], 2), 3)
</code></pre></td><td>Computes base-2 log and rounds it to 3 decimals.</td><td>Varies by row.</td></tr></tbody></table>

## LOG10

The `LOG10` function returns the base-10 logarithm of a specified number.

### Syntax

```dax
LOG10(number)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>number</td><td>The positive real number, column or mocker for which you want to calculate the base-10 logarithm.</td></tr></tbody></table>

### Return value

The base-10 logarithm of a number.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">LOG10([REVENUE])
</code></pre></td><td>Returns the base-10 log of <code>[HEALTHCARE_EXPENSES]</code> per row.</td><td>Varies by row (and errors for non-positive values).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LOG10(1000)
</code></pre></td><td>Returns the base-10 log of <code>1000</code>.</td><td><code>3</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LOG10([HEALTHCARE_EXPENSES] + 1)
</code></pre></td><td>Adds <code>1</code> first, then computes the base-10 log.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF([REVENUE] > 0, LOG10([REVENUE]), 0)
</code></pre></td><td>Computes <code>LOG10</code> only for positive values; otherwise returns <code>0</code>.</td><td>Either <code>0</code> or a log value per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(LOG10([REVENUE]), 2)
</code></pre></td><td>Computes base-10 log and rounds it to 2 decimals.</td><td>Varies by row.</td></tr></tbody></table>

## LOWER

The `LOWER` function converts all uppercase letters in a text string to lowercase.

### Syntax

```dax
LOWER("text")
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>"text"</td><td>The text, column or mocker you want to convert to lowercase. The <code>LOWER</code> function does not alter characters in the text that are not letters.</td></tr></tbody></table>

### Return value

Text converted to lowercase.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">LOWER([ADDRESS])
</code></pre></td><td>Converts <code>[ADDRESS]</code> to lowercase per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LOWER("John.Doe@Example.com")
</code></pre></td><td>Converts a literal string to lowercase.</td><td><code>"john.doe@example.com"</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LOWER(CONCATENATE([FIRSTNAME], ".", [LASTNAME]))
</code></pre></td><td>Builds <code>"First.Last"</code> then lowercases it.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LOWER(TRIM([EMAIL]))
</code></pre></td><td>Trims whitespace from <code>[EMAIL]</code>, then lowercases it.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">CONCATENATE(LOWER([FIRSTNAME]), ".", LOWER([LASTNAME]), "@", MOCK_FREE_EMAIL_DOMAIN)
</code></pre></td><td>Builds a lowercase email-like string using name columns plus a generated domain.</td><td>Varies by row.</td></tr></tbody></table>

## MINUTE

The `MINUTE` function returns the minute component of a time value as an integer, ranging from 0 to 59. Time values are a portion of a date value and are represented as a decimal number (e.g., 12:00 PM is represented as `0.5` since it is half of a day).

### Syntax

```dax
MINUTE(time)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>time</td><td>The time, column or mocker from which you want to extract the minute. This can be entered as a text string (e.g., <code>"4:30 PM"</code>), as a decimal number (e.g., <code>0.78125</code> representing 6:45 PM), or as the result of other formulas or functions.</td></tr></tbody></table>

### Return value

Minute component of given time.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">MINUTE("4:30:45 PM")
</code></pre></td><td>Extracts the minute component from a literal time string.</td><td><code>30</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">MINUTE([START])
</code></pre></td><td>Extracts the minute component from the <code>[START]</code> time/datetime per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">MINUTE("6/24/2012 07:05:00")
</code></pre></td><td>Extracts the minute component from a literal datetime string.</td><td><code>5</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(MINUTE([START]) = 0, "on_the_hour", "off_cycle")
</code></pre></td><td>Flags whether <code>[START]</code> happens exactly on the hour.</td><td>Either <code>"on_the_hour"</code> or <code>"off_cycle"</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">MINUTE([START]) + (HOUR([START]) * 60)
</code></pre></td><td>Converts a time to minutes since midnight.</td><td>Varies by row.</td></tr></tbody></table>

## MOD

Calculates the remainder of a division and always ensures it matches the sign of the divisor for consistency.

* If the divisor is 0, `MOD` returns the `#DIV/0!` error value.
* The `MOD` function can be expressed using the `INT` function as follows:\
  \&#xNAN;*`MOD(n, d) = n - d*INT(n/d)`*

### Syntax

```dax
MOD(dividend, divisor)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>dividend</td><td>The number, column or mocker for which you want to find the remainder.</td></tr><tr><td>divisor</td><td>The number, column or mocker by which you want to divide the number.</td></tr></tbody></table>

### Return value

The remainder when one integer is divided by another.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

The following formula returns the remainders when integers with different signs is divided by integers with different signs.

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">MOD(5, 2)
</code></pre></td><td>Returns the remainder of <code>5 / 2</code>.</td><td><code>1</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">MOD(-5, 2)
</code></pre></td><td>Returns the remainder of <code>-5 / 2</code> (same sign as the divisor).</td><td><code>1</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">MOD(5, -2)
</code></pre></td><td>Returns the remainder of <code>5 / -2</code> (same sign as the divisor).</td><td><code>-1</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">MOD(-5, -2)
</code></pre></td><td>Returns the remainder of <code>-5 / -2</code> (same sign as the divisor).</td><td><code>-1</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">MOD([INVOICE_ID], 10)
</code></pre></td><td>Returns the last digit (mod 10) of <code>[INVOICE_ID]</code> per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(MOD([INVOICE_ID], 2) = 0, "even", "odd")
</code></pre></td><td>Classifies invoice ids as even or odd using a mod 2 check.</td><td>Either <code>"even"</code> or <code>"odd"</code> per row.</td></tr></tbody></table>

## MONTH

The `MONTH` function returns the month of a date represented by a serial number. The month is provided as an integer ranging from 1 (January) to 12 (December).

Values returned by the `YEAR`, `MONTH`, and `DAY` functions are based on the Gregorian calendar, regardless of the display format of the provided date.

### Syntax

```dax
MONTH(number)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>number</td><td>The date, column or mocker from which you want to extract the month. Dates should be entered using the <code>DATE</code> function or as results from other formulas or functions (e.g., <code>DATE(2008, 5, 23)</code> for May 23, 2008). Entering dates as text can cause issues.</td></tr></tbody></table>

### Return value

The month for the given date value.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">MONTH([BIRTHDATE])
</code></pre></td><td>Returns the month number from <code>[BIRTHDATE]</code> per row.</td><td>Varies by row (1–12).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">MONTH(DATE(2024, 12, 31))
</code></pre></td><td>Returns the month number from a constructed date.</td><td><code>12</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">MONTH("24/8/2012")
</code></pre></td><td>Returns the month number from a literal date string.</td><td><code>8</code> (locale dependent).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">MONTH(DATEADD([BIRTHDATE], 6, "month"))
</code></pre></td><td>Adds 6 months to <code>[BIRTHDATE]</code> then extracts the month.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(MONTH([ORDER_DATE]) = 12, "holiday_season", "regular")
</code></pre></td><td>Flags December orders as holiday season.</td><td>Either <code>"holiday_season"</code> or <code>"regular"</code> per row.</td></tr></tbody></table>

## NOT

The `NOT` function reverses the logical value of its argument. If the argument is `TRUE`, `NOT` returns `FALSE`, and if the argument is `FALSE`, `NOT` returns `TRUE`. One common use of the `NOT` function is to enhance the functionality of other logical functions, such as `IF`. By using `NOT` within the logical test of an `IF` function, you can test multiple conditions and control the flow of logic based on reversed outcomes.

### Syntax

```dax
NOT(Logicalvalue)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>Logicalvalue</td><td>A column or expression that can be evaluated to <code>TRUE</code> or <code>FALSE</code>.</td></tr></tbody></table>

### Return value

Complements (inverts) a logical value.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">IF(
  AND(
    NOT([GENDER] = "M"),
    NOT([HEALTHCARE_EXPENSES] &#x3C; 100)
  ),
  MOCK_NAME_FEMALE,
  MOCK_NAME_MALE
)
</code></pre></td><td>Generates a female name when gender is not <code>"M"</code> and expenses are not &#x3C; 100; otherwise generates a male name.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">NOT([IS_ACTIVE] = 1)
</code></pre></td><td>Inverts the active flag check.</td><td><code>TRUE</code> or <code>FALSE</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(NOT([ZIP] = "0000XX"), "valid", "fallback")
</code></pre></td><td>Returns <code>"valid"</code> unless <code>[ZIP]</code> equals the sentinel value <code>"0000XX"</code>.</td><td>Either <code>"valid"</code> or <code>"fallback"</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">NOT(ISNULL([ZIP]))
</code></pre></td><td>Returns <code>TRUE</code> when <code>[ZIP]</code> is present (not NULL).</td><td><code>TRUE</code> or <code>FALSE</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(NOT(AND([COUNTRY] = "US", LEN([ZIP]) = 5)), "invalid_zip", "ok")
</code></pre></td><td>Validates US ZIPs: must be length 5 when country is <code>"US"</code>.</td><td>Either <code>"invalid_zip"</code> or <code>"ok"</code> per row.</td></tr></tbody></table>

## ODD

The `ODD` function rounds a number up to the nearest odd integer.

* If the `number` is nonnumeric, the function returns an error.
* The function always rounds a value up and away from zero, regardless of its sign. If the `number` is already an odd integer, no rounding occurs.

### Syntax

```dax
ODD(number)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>number</td><td>The number, column or mocker to be rounded up to the nearest odd integer.</td></tr></tbody></table>

### Return value

A positive number rounded up to the nearest odd integer and a negative number rounded down to the nearest odd integer.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

The following formulas return positive numbers rounded up to the nearest odd integers and negative numbers rounded down to the nearest odd integers.

<table><thead><tr><th>DAX Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">ODD(3.4)
</code></pre></td><td>Rounds 3.4 up to the nearest odd integer.</td><td>5</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ODD(3)
</code></pre></td><td>Rounds 3 up to the nearest odd integer.</td><td>3</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ODD(6)
</code></pre></td><td>Rounds 6 up to the nearest odd integer.</td><td>7</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ODD(-3)
</code></pre></td><td>Rounds -3 up to the nearest odd integer.</td><td>-3</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ODD(-4)
</code></pre></td><td>Rounds -4 up (away from 0) to the nearest odd integer.</td><td>-5</td></tr></tbody></table>

## OR

The `OR` function returns `TRUE` if any of its arguments evaluate to `TRUE`, and returns `FALSE` if all arguments evaluate to `FALSE`.

* The arguments must evaluate to logical values, such as `TRUE` or `FALSE`, or be arrays that contain logical values.
* If an array or range argument contains text or empty values, those are ignored.
* If no logical values are found in the specified range, `OR` returns an error.
* You can use an `OR` array formula to check if a value occurs within an array.

### Syntax

```dax
OR(Logicalvalue1, Logicalvalue2 ...Logicalvalue30)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>Logicalvalue1</td><td>The first condition to test, which can evaluate to either <code>TRUE</code> or <code>FALSE</code>.</td></tr><tr><td>Logicalvalue2 ...Logicalvalue30</td><td>Additional conditions to test, up to a maximum of 30 conditions, each of which can evaluate to either <code>TRUE</code> or <code>FALSE</code>.</td></tr></tbody></table>

### Return value

`TRUE` if at least one argument is `TRUE`.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">OR([HEALTHCARE_COVERAGE] &#x3C; 100, [HEALTHCARE_EXPENSES] > 250)
</code></pre></td><td>Returns <code>TRUE</code> when coverage is &#x3C; 100 <strong>or</strong> expenses are > 250.</td><td><code>TRUE</code> or <code>FALSE</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">OR([GENDER] = "F", [GENDER] = "M", [GENDER] = "X")
</code></pre></td><td>Checks whether gender is one of <code>"F"</code>, <code>"M"</code>, or <code>"X"</code>.</td><td><code>TRUE</code> or <code>FALSE</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(OR([STATE] = "Massachusetts", [STATE] = "California"), "US", "non-US")
</code></pre></td><td>Labels rows as <code>"US"</code> when state is MA or CA.</td><td>Either <code>"US"</code> or <code>"non-US"</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">OR(ISNULL([EMAIL]), ISNULL([PHONE]))
</code></pre></td><td>Returns <code>TRUE</code> when either email or phone is missing.</td><td><code>TRUE</code> or <code>FALSE</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(OR([COUNTRY] = "US", [COUNTRY] = "CA"), "NA", "OTHER")
</code></pre></td><td>Classifies countries into North America vs other.</td><td>Either <code>"NA"</code> or <code>"OTHER"</code> per row.</td></tr></tbody></table>

## PI

The `PI` function returns the mathematical constant pi (π), accurate to 15 digits, which is approximately 3.14159265358979.

### Syntax

```dax
PI()
```

#### Parameters <a href="#parameters" id="parameters"></a>

The `PI` function does not require any arguments. It simply returns the value of π.

### Return value

Returns 3.14159265358979, the value of the mathematical constant PI to 14 decimal places.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">PI() * (5 ^ 2)
</code></pre></td><td>Computes the area of a circle with radius <code>5</code> (πr²).</td><td><code>~78.5398</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">2 * PI() * 5
</code></pre></td><td>Computes the circumference of a circle with radius <code>5</code> (2πr).</td><td><code>~31.4159</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">PI() * POWER([RADIUS], 2)
</code></pre></td><td>Computes πr² using <code>[RADIUS]</code> per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(PI(), 4)
</code></pre></td><td>Rounds π to 4 decimals.</td><td><code>3.1416</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(PI() / 180, 8)
</code></pre></td><td>Converts degrees to radians factor (π/180), rounded.</td><td><code>0.01745329</code></td></tr></tbody></table>

## POWER

The `POWER` function returns the result of a base number raised to a specified power. You can also use the `^` operator as an alternative to the `POWER` function, such as `4^3`, to indicate that the base number (4) is raised to the power of 3.

### Syntax

```dax
POWER(base, exponent)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>base</td><td>The base number, column or mocker, which can be any real number.</td></tr><tr><td>exponent</td><td>The exponent number, column or mocker to which the base number is raised.</td></tr></tbody></table>

### Return value

A number raised to another number.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">POWER(1.1, 3)
</code></pre></td><td>Raises <code>1.1</code> to the power of <code>3</code>.</td><td><code>1.331</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">POWER([AMOUNT], [EXPONENT])
</code></pre></td><td>Raises <code>[AMOUNT]</code> to <code>[EXPONENT]</code> per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">4 ^ 3
</code></pre></td><td>Raises <code>4</code> to the power of <code>3</code> using the <code>^</code> operator.</td><td><code>64</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">SQRT(POWER([X], 2) + POWER([Y], 2))
</code></pre></td><td>Computes the Euclidean norm (√(x² + y²)) per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(POWER(1 + [RATE], [YEARS]), 6)
</code></pre></td><td>Computes compound growth then rounds to 6 decimals.</td><td>Varies by row.</td></tr></tbody></table>

## PROPER

The `PROPER` function capitalizes the first letter of each word in a text string and any other letters that follow a non-letter character. It converts all other letters to lowercase.

### Syntax

```dataweave
PROPER("text")
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>"text"</td><td>The text, column or mocker you want to format. This can be a text string enclosed in quotation marks, a formula that returns text, or a reference to a value containing the text you want to capitalize.</td></tr></tbody></table>

### Return value

Capitalized words or texts.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dart"><code class="lang-dart">PROPER("5-star 99hoTels AMSTERDAM")
</code></pre></td><td>Capitalizes the first letter of each word and lowercases the rest.</td><td><code>"5-Star 99hotels Amsterdam"</code></td></tr><tr><td><pre class="language-dart"><code class="lang-dart">PROPER([ADDRESS])
</code></pre></td><td>Applies proper casing to <code>[ADDRESS]</code> per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dart"><code class="lang-dart">PROPER(MOCK_ADDRESS)
</code></pre></td><td>Applies proper casing to a generated address.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dart"><code class="lang-dart">PROPER(TRIM([ADDRESS]))
</code></pre></td><td>Trims whitespace first, then applies proper casing.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dart"><code class="lang-dart">PROPER(LOWER([FULL_NAME]))
</code></pre></td><td>Lowercases <code>[FULL_NAME]</code> first, then applies proper casing.</td><td>Varies by row.</td></tr></tbody></table>

## PV

The `PV` function returns the present value of an annuity based on periodic, fixed payments to be made in the future and a fixed interest rate. The present value is the total amount that a series of future payments is worth now.

* An annuity is a series of fixed payments made over time, such as a mortgage or a savings plan.
* The `rate` and `nper` arguments must be based on the same units of time. For example, if `rate` is calculated using months, `nper` should also be calculated using months.
* In financial functions, cash outflows (such as deposits or payments) are represented by negative numbers, while cash inflows (such as dividends) are represented by positive numbers.

### Syntax

```dataweave
PV(Rate, Nper, Pmt, Fv, Type)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>Rate</td><td>The interest rate per period. For example, if you have a car loan with an annual percentage rate (APR) of 10% and make monthly payments, the rate per period is <code>0.1/12</code>, or <code>0.0083</code>.</td></tr><tr><td>Nper</td><td>The total number of payment periods in the annuity. For example, a two-year car loan with monthly payments has <code>2 * 12 = 24</code> payment periods.</td></tr><tr><td>Pmt</td><td>The payment made each period, which typically includes both principal and interest and remains constant over the life of the annuity.</td></tr><tr><td>Fv</td><td>The future value or cash balance you want after the final payment. For example, the future value of a loan is $0 after the final payment. If omitted, <code>0</code> is assumed.</td></tr><tr><td>Type</td><td>Specifies when payments are due—<code>0</code> for payments at the end of the period, or <code>1</code> for payments at the beginning. If omitted, <code>0</code> is assumed.</td></tr></tbody></table>

### Return value

The present value of an investment.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dart"><code class="lang-dart">PV(0.06/12, 4*12, -250.99)
</code></pre></td><td>Calculates the present value of a 4-year loan at 6% APR with monthly payments.</td><td><code>~10687</code></td></tr><tr><td><pre class="language-dart"><code class="lang-dart">PV([RATE], [NPER], [PMT], [FV], [TYPE])
</code></pre></td><td>Calculates present value from per-row rate/period/payment inputs.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dart"><code class="lang-dart">PV(0.06/12, 4*12, -250.99, 0, 0)
</code></pre></td><td>Same as above, with explicit <code>FV=0</code> and end-of-period payments (<code>Type=0</code>).</td><td><code>~10687</code></td></tr><tr><td><pre class="language-dart"><code class="lang-dart">PV(0.06/12, 4*12, -250.99, 1000, 1)
</code></pre></td><td>Computes present value with a future value of <code>1000</code> and beginning-of-period payments.</td><td><code>~11778</code></td></tr><tr><td><pre class="language-dart"><code class="lang-dart">PV(0.06/12, 4*12, -250.99, 0, 1)
</code></pre></td><td>Computes present value with beginning-of-period payments (<code>Type=1</code>).</td><td><code>~10740</code></td></tr><tr><td><pre class="language-dart"><code class="lang-dart">ROUND(PV([RATE], [NPER], [PMT], [FV], [TYPE]), 2)
</code></pre></td><td>Rounds the computed present value to 2 decimals.</td><td>Varies by row.</td></tr></tbody></table>

## QUARTER

The `QUARTER` function returns the quarter of the year for a given date, represented as a number from 1 to 4, where:

* 1 corresponds to January – March,
* 2 corresponds to April – June,
* 3 corresponds to July – September,
* 4 corresponds to October – December.

### Syntax

```dax
QUARTER(date)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>date</td><td>The date, column or mocker for which you want to determine the quarter.</td></tr></tbody></table>

### Return value

An integer from 1 to 4, representing the quarter of the year.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">QUARTER(DATE(2012, 8, 24))
</code></pre></td><td>Returns the quarter for Aug 24, 2012.</td><td><code>3</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">QUARTER([BIRTHDATE])
</code></pre></td><td>Returns the quarter for <code>[BIRTHDATE]</code> per row.</td><td>Varies by row (1–4).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">QUARTER(DATEVALUE("2024/05/01"))
</code></pre></td><td>Parses the date first, then returns its quarter.</td><td><code>2</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">QUARTER(DATEADD([BIRTHDATE], 3, "month"))
</code></pre></td><td>Adds 3 months to <code>[BIRTHDATE]</code> then returns the quarter.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(QUARTER([ORDER_DATE]) = 4, "peak", "normal")
</code></pre></td><td>Flags Q4 orders as peak season.</td><td>Either <code>"peak"</code> or <code>"normal"</code> per row.</td></tr></tbody></table>

## QUOTIENT

The `QUOTIENT` function returns the integer portion of a division, discarding the remainder. This function is useful when you only need the whole number result of a division.

### Syntax

```dax
QUOTIENT(dividend, divisor)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>dividend</td><td>The dividend column, mocker or the number to be divided.</td></tr><tr><td>divisor</td><td>The divisor column, mocker or the number by which the numerator is divided.</td></tr></tbody></table>

### Return value

Integer part of a division.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">QUOTIENT(-12.5, 5.3)
</code></pre></td><td>Returns the integer part of the division.</td><td><code>-2</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">QUOTIENT([REVENUE], [NUMBER_OF_SALES])
</code></pre></td><td>Computes revenue per sale and returns the integer portion.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">QUOTIENT(17, 3)
</code></pre></td><td>Returns the integer part of <code>17/3</code>.</td><td><code>5</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">QUOTIENT([TOTAL_SECONDS], 60)
</code></pre></td><td>Converts seconds to whole minutes per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(QUOTIENT([ORDER_TOTAL], 1000) >= 1, "high_value", "standard")
</code></pre></td><td>Flags orders of at least 1000 as high value.</td><td>Either <code>"high_value"</code> or <code>"standard"</code> per row.</td></tr></tbody></table>

## RAND

The `RAND` function returns a random real number that is evenly distributed between 0 (inclusive) and 1 (exclusive). Each time the worksheet recalculates, a new random number is generated.

### Syntax

```dax
RAND()
```

#### Parameters <a href="#parameters" id="parameters"></a>

The `RAND` function does not require any arguments. To generate a random real number between two specified values `a` and `b`, use the following formula:

```dax
a + (b - a) * RAND()
```

### Return value

A random number between 0 and 1.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">5 + (3 - 5) * RAND()
</code></pre></td><td>Generates a random number in the range <code>[3, 5)</code>.</td><td>Random number per recalculation.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">RAND()
</code></pre></td><td>Generates a random number in the range <code>[0, 1)</code>.</td><td>Random number per recalculation.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">100 * RAND()
</code></pre></td><td>Scales a random number to the range <code>[0, 100)</code>.</td><td>Random number per recalculation.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(RAND(), 3)
</code></pre></td><td>Rounds a random number to 3 decimals.</td><td>Random number per recalculation.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(RAND() &#x3C; 0.05, "rare", "common")
</code></pre></td><td>Returns <code>"rare"</code> about 5% of the time.</td><td>Either <code>"rare"</code> or <code>"common"</code> per recalculation.</td></tr></tbody></table>

## RANDBETWEEN

The `RANDBETWEEN` function returns a random integer between the specified lower and upper bounds. Each time the worksheet recalculates, a new random integer is generated.

### Syntax

```dax
RANDBETWEEN(Lowerbound, Upperbound)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>lowerbound</td><td>The smallest integer, column or mocker that can be returned by the function.</td></tr><tr><td>upperbound</td><td>The largest integer, column or mocker that can be returned by the function.</td></tr></tbody></table>

### Return value

A random integer between two numbers.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">RANDBETWEEN(3, 5)
</code></pre></td><td>Returns a random integer between 3 and 5 (inclusive).</td><td><code>3</code>, <code>4</code>, or <code>5</code> per recalculation.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">RANDBETWEEN(-10, 10)
</code></pre></td><td>Returns a random integer between -10 and 10 (inclusive).</td><td>An integer in <code>[-10, 10]</code> per recalculation.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">RANDBETWEEN([MIN_AGE], [MAX_AGE])
</code></pre></td><td>Returns a random integer between the per-row bounds.</td><td>Varies by row and recalculation.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATEADD(TODAY(), RANDBETWEEN(-30, 30), "day")
</code></pre></td><td>Returns a random date within ±30 days of today.</td><td>Varies by run date and recalculation.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">CHAR(RANDBETWEEN(48, 57))
</code></pre></td><td>Generates a random digit character (<code>0</code>–<code>9</code>).</td><td>A character <code>"0"</code>…<code>"9"</code> per recalculation.</td></tr></tbody></table>

## REPLACE

The `REPLACE` function replaces part of a text string with a different text string, based on the number of characters you specify.

### Syntax

```dax
REPLACE(old_text, start_num, num_chars, new_text)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>old_text</td><td>The original text, column or mocker in which you want to replace some characters.</td></tr><tr><td>start_num</td><td>The position of the character in <code>old_text</code> that you want to start replacing with <code>new_text</code>.</td></tr><tr><td>num_chars</td><td>The number of characters in <code>old_text</code> that you want <code>REPLACE</code> to replace with <code>new_text</code>.</td></tr><tr><td>new_text</td><td>The text, column or mocker that will replace the specified characters or bytes in <code>old_text</code>.</td></tr></tbody></table>

### Return value

Part of a string with a new string.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">REPLACE("Hello Jupyter!", 6, 7, "Moon")
</code></pre></td><td>Replaces 7 characters starting at position 6 with <code>"Moon"</code>.</td><td><code>"Hello Moon!"</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">REPLACE([PASSPORT], 6, 7, "004A")
</code></pre></td><td>Replaces 7 characters in <code>[PASSPORT]</code> starting at position 6.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">REPLACE([PASSPORT], 1, 2, "NL")
</code></pre></td><td>Overwrites the first two characters of <code>[PASSPORT]</code> with <code>"NL"</code>.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">REPLACE(UPPER([COUNTRY_CODE]), 1, 2, "NL")
</code></pre></td><td>Uppercases <code>[COUNTRY_CODE]</code> then replaces the first two characters with <code>"NL"</code>.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">REPLACE([PASSPORT], LEN([PASSPORT]) - 1, 2, "XX")
</code></pre></td><td>Replaces the last 2 characters of <code>[PASSPORT]</code> with <code>"XX"</code>.</td><td>Varies by row.</td></tr></tbody></table>

## RIGHT

The `RIGHT` function returns a specified number of characters from the right side of a string.

### Syntax

```dax
RIGHT("text", number)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>"text"</td><td>The text, column or mocker from which the rightmost characters are returned. If the string contains <code>Null</code>, the function returns <code>Null</code>.</td></tr><tr><td>number</td><td>A numeric expression indicating how many characters to return. If <code>0</code>, a zero-length string (<code>""</code>) is returned. If the length is greater than or equal to the number of characters in the string, the entire string is returned.</td></tr></tbody></table>

### Return value

The input text starting from a specified character index.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">RIGHT("Hello Earth!", 6)
</code></pre></td><td>Returns the last 6 characters of a literal string.</td><td><code>"Earth!"</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">RIGHT([STATE], MOCK_RANDOM_DIGIT)
</code></pre></td><td>Returns a random-length suffix of <code>[STATE]</code> per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">RIGHT("Massachusetts", 4)
</code></pre></td><td>Returns the last 4 characters of a literal string.</td><td><code>"etts"</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">RIGHT(CONCATENATE("00000", [COUNTRY_CODE]), 5)
</code></pre></td><td>Left-pads ZIP with zeros then returns the last 5 characters.</td><td>Varies by row (e.g., <code>"02134"</code>).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">RIGHT([PHONE], 4)
</code></pre></td><td>Returns the last 4 characters of a phone number per row.</td><td>Varies by row.</td></tr></tbody></table>

## ROUND

The `ROUND` function rounds a number to a specified number of digits.

* If `Count` is greater than 0, the `Number` is rounded to the specified number of decimal places.
* If `Count` is 0, the `Number` is rounded to the nearest integer.
* If `Count` is less than 0, the `Number` is rounded to the left of the decimal point.
* To always round up (away from zero), use the `ROUNDUP` function.
* To always round down (toward zero), use the `ROUNDDOWN` function.

### Syntax

```dax
ROUND(number, count)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>number</td><td>The number, column or mocker you want to round.</td></tr><tr><td>count</td><td>The number of digits to which you want to round the <code>number</code>.</td></tr></tbody></table>

### Return value

A number that is rounded to a certain number of decimal places.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

The following formulas round the numbers to various decimal places.

<table><thead><tr><th width="228">Formula</th><th width="361">Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(3.25, 1)
</code></pre></td><td>Rounds 3.25 to one decimal place</td><td>3.2</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(5.149, 1)
</code></pre></td><td>Rounds 5.149 to one decimal place</td><td>5.1</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(-6.745, 2)
</code></pre></td><td>Rounds -6.745 to two decimal places</td><td>-6.75</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(23.9, -1)
</code></pre></td><td>Rounds 23.9 to one decimal place to the left of the decimal point</td><td>20</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(575.9, -3)
</code></pre></td><td>Rounds 575.9 to the nearest multiple of 1000</td><td>1000</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(2.33, -1)
</code></pre></td><td>Rounds 2.33 to the nearest multiple of 10</td><td>0</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(-51.52, -2)
</code></pre></td><td>Rounds -51.52 to the nearest multiple of 100</td><td>-100</td></tr></tbody></table>

## ROUNDDOWN

The `ROUNDDOWN` function rounds a number down, toward zero.

* `ROUNDDOWN` behaves similarly to `ROUND`, except that it always rounds a number down.
* If `Count` is greater than 0, the `number` is rounded down to the specified number of decimal places.
* If `Count` is 0, the `number` is rounded down to the nearest integer.
* If `Count` is less than 0, the `number` is rounded down to the left of the decimal point.

### Syntax

```dax
ROUNDDOWN(number, count)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>number</td><td>Any real number, column or mocker that you want to round down.</td></tr><tr><td>count</td><td>The number of digits to which you want to round the <code>number</code>.</td></tr></tbody></table>

### Return value

A number rounded down, toward zero, to a certain precision.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

The following formulas round down the numbers to various decimal places.

<table><thead><tr><th width="304">Formula</th><th width="341">Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">ROUNDDOWN(5.3, 0)
</code></pre></td><td>Rounds 5.3 down to zero decimal place</td><td>5</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUNDDOWN(66.8,0)
</code></pre></td><td>Rounds 66.8 down to zero decimal places</td><td>66</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUNDDOWN(1.24279, 3)
</code></pre></td><td>Rounds 1.24279 down to three decimal places</td><td>1.242</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUNDDOWN(-1.24279, 1)
</code></pre></td><td>Rounds -1.24279 down to one decimal places</td><td>-1.2</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUNDDOWN(12427.98637, -2)
</code></pre></td><td>Rounds 12427.98637 down to 2 decimal places to the left of the decimal point.</td><td>12400</td></tr></tbody></table>

## ROUNDUP

The `ROUNDUP` function rounds a number up, away from 0 (zero).

* `ROUNDUP` behaves similarly to `ROUND`, except that it always rounds a number up.
* If `Count` is greater than 0, the `Number` is rounded up to the specified number of decimal places.
* If `Count` is 0, the `Number` is rounded up to the nearest integer.
* If `Count` is less than 0, the `Number` is rounded up to the left of the decimal point.

### Syntax

```dax
ROUNDUP(number, count)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>number</td><td>Any real number, column or mocker that you want rounded up.</td></tr><tr><td>count</td><td>The number of digits to which you want to round the <code>number</code>.</td></tr></tbody></table>

### Return value

A number rounded up, away from zero, to a certain precision.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

The following formulas round up the numbers to various decimal places.

<table><thead><tr><th width="304">Formula</th><th width="341">Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">ROUNDUP(5.3, 0)
</code></pre></td><td>Rounds 5.3 up to zero decimal place</td><td>6</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUNDUP(66.8,0)
</code></pre></td><td>Rounds 66.8 up to zero decimal place</td><td>67</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUNDUP(1.24279, 3)
</code></pre></td><td>Rounds 1.24279 up to three decimal places</td><td>1.243</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUNDUP(-1.24279, 1)
</code></pre></td><td>Rounds -1.24279 up to one decimal places</td><td>-1.3</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUNDUP(12427.98637, -2)
</code></pre></td><td>Rounds 12427.98637 up to 2 decimal places to the left of the decimal point.</td><td>12500</td></tr></tbody></table>

## SECOND

The `SECOND` function returns the seconds component of a time value, as an integer ranging from 0 to 59.

### Syntax

```dax
SECOND(time)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>time</td><td>The time, column or mocker values from which you want to extract the seconds. This can be entered as a text string (e.g., <code>"6:45 PM"</code>), as a decimal number (e.g., <code>0.78125</code> representing 6:45 PM), or as the result of other formulas or functions (e.g., <code>TIMEVALUE("6:45 PM")</code>).</td></tr></tbody></table>

### Return value

Second component of given time.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">SECOND("2:25:55 PM")
</code></pre></td><td>Extracts the seconds component from a literal time string.</td><td><code>55</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">SECOND([START_TIME])
</code></pre></td><td>Extracts the seconds component from <code>[START_TIME]</code> per row.</td><td>Varies by row (0–59).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">SECOND("6/24/2012 07:30:05")
</code></pre></td><td>Extracts the seconds component from a literal datetime string.</td><td><code>5</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(SECOND([START_TIME]) = 0, "exact_minute", "has_seconds")
</code></pre></td><td>Flags whether the time is exactly on a minute boundary.</td><td>Either <code>"exact_minute"</code> or <code>"has_seconds"</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">SECOND([START_TIME]) + (MINUTE([START_TIME]) * 60)
</code></pre></td><td>Converts the seconds within the hour (minute*60 + second).</td><td>Varies by row (0–3599).</td></tr></tbody></table>

## SQRT

The `SQRT` function returns the positive square root of a given number.

### Syntax

```dax
SQRT(number)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>number</td><td>The number, column or mocker for which you want to find the square root. If the <code>number</code> is negative, <code>SQRT</code> returns an error value.</td></tr></tbody></table>

### Return value

The positive square root of a number.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">SQRT(49)
</code></pre></td><td>Returns the square root of a literal number.</td><td><code>7</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">SQRT([PAYER_COVERAGE])
</code></pre></td><td>Returns the square root of <code>[PAYER_COVERAGE]</code> per row.</td><td>Varies by row (and errors for negative values).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">SQRT(POWER([PAYER_COVERAGE], 2))
</code></pre></td><td>Squares the value then takes the square root (returns absolute value for non-negative inputs).</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(SQRT([PAYER_COVERAGE]), 2)
</code></pre></td><td>Computes the square root then rounds to 2 decimals.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF([PAYER_COVERAGE] &#x3C; 0, 0, SQRT([PAYER_COVERAGE]))
</code></pre></td><td>Guards against negative inputs by returning <code>0</code> instead.</td><td>Either <code>0</code> or a square-root value per row.</td></tr></tbody></table>

## SUBSTITUTE

The `SUBSTITUTE` function replaces specific text within a text string with new text. Use `SUBSTITUTE` when you want to replace specific occurrences of text in a string; use `REPLACE` when you want to replace text based on its position in the string.

### Syntax

```dax
SUBSTITUTE(text, old_text, new_text, occurrence)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>text</td><td>The text string, column or mocker containing the text where you want to substitute characters.</td></tr><tr><td>old_text</td><td>The text you want to replace.</td></tr><tr><td>new_text</td><td>The text that will replace <code>Old_text</code></td></tr><tr><td>occurrence</td><td>Specifies which occurrence of <code>Old_text</code> you want to replace with <code>New_text</code>. If you provide <code>Occurrence</code>, only that specific instance of <code>Old_text</code> is replaced. If omitted, all occurrences of <code>Old_text</code> in the text are replaced with <code>New_text</code>.</td></tr></tbody></table>

### Return value

String where occurrences of `Old_text` are replaced by `New_text`. It replaces only specific occurrence if last parameter is provided.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">SUBSTITUTE("April 2, 2012", "2", "3", 3)
</code></pre></td><td>Replaces the 3rd occurrence of <code>"2"</code> with <code>"3"</code>.</td><td><code>"April 2, 2013"</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">SUBSTITUTE([PASSPORT], "X", "S", 1)
</code></pre></td><td>Replaces the first <code>"X"</code> in <code>[PASSPORT]</code> with <code>"S"</code> per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">SUBSTITUTE("john.doe@example.com", ".", "_")
</code></pre></td><td>Replaces all dots with underscores.</td><td><code>"john_doe@example_com"</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">SUBSTITUTE(LOWER([EMAIL]), " ", "")
</code></pre></td><td>Lowercases <code>[EMAIL]</code> then removes all spaces.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">SUBSTITUTE([ADDRESS], CHAR(10), " ")
</code></pre></td><td>Replaces line breaks with spaces.</td><td>Varies by row.</td></tr></tbody></table>

## SWITCH

The `SWITCH` function evaluates one value (referred to as the expression) against a list of values and returns the result corresponding to the first matching value. If no match is found, an optional default value may be returned.

### Syntax

```dataweave
SWITCH(expression1, value1, result2, value2..., result_n, default)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>expression1</td><td>The value (such as a number, date, text or column) that will be compared against <code>value_1</code> through <code>value_N</code>.</td></tr><tr><td>value1, value2, ... value_N</td><td>The columns to be compared against <code>Expression1</code> through <code>expression_n</code>.</td></tr><tr><td>result1…result_n</td><td>The values to be returned when the corresponding <code>value1</code> through <code>value_N</code> arguments match result<code>1</code> through <code>result_n</code>. Each <code>result_n</code> must be provided for each corresponding <code>value_N</code> argument.</td></tr><tr><td>default</td><td><p>The column to return if no matches are found in the <code>value_N</code> expressions. The <code>Default</code> argument is identified by having no corresponding <code>result_n</code> expression. It must be the final argument in the function.</p><p>Note: The <code>Default</code> value must <em>not</em> exist in the referenced column.</p></td></tr></tbody></table>

### Return value

A list of arguments, consisting of an expression followed by a value.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">SWITCH(3, 1, "Dog", 2, "Cat", 3, "Horse", "?")
</code></pre></td><td>Maps the expression <code>3</code> to the matching value.</td><td><code>"Horse"</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">SWITCH([STATUS], 1, "New", 2, "Active", 3, "Closed", "Unknown")
</code></pre></td><td>Maps numeric status codes to labels, with <code>"Unknown"</code> as default.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">SWITCH([GENDER], "F", "female", "M", "male", "other")
</code></pre></td><td>Maps gender codes to normalized values.</td><td>Either <code>"female"</code>, <code>"male"</code>, or <code>"other"</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">SWITCH(TRUE(), [SCORE] >= 90, "A", [SCORE] >= 80, "B", [SCORE] >= 70, "C", "D")
</code></pre></td><td>Implements a tiered grade mapping using <code>TRUE()</code> as the expression.</td><td><code>"A"</code>, <code>"B"</code>, <code>"C"</code>, or <code>"D"</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">SWITCH(LEFT([COUNTRY], 2), "US", "United States", "NL", "Netherlands", "Other")
</code></pre></td><td>Maps country prefixes to full names.</td><td>Either <code>"United States"</code>, <code>"Netherlands"</code>, or <code>"Other"</code> per row.</td></tr></tbody></table>

## TODAY

The `TODAY` function returns the current date.

### Syntax

```dax
TODAY()
```

#### Parameters <a href="#parameters" id="parameters"></a>

The `TODAY` function has no arguments.

### Return value

An integer representing the current date as the number of full days since nullDate.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">YEAR(TODAY()) - 1998
</code></pre></td><td>Approximates age in years for a person born in 1998 (based on the current year).</td><td>Varies by run date.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">TODAY()
</code></pre></td><td>Returns the current date.</td><td>Varies by run date.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATEDIFF([BIRTHDATE], TODAY(), "year")
</code></pre></td><td>Computes age in full years based on <code>[BIRTHDATE]</code>.</td><td>Varies by row and run date.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">DATEADD(TODAY(), 30, "day")
</code></pre></td><td>Returns the date 30 days from today.</td><td>Varies by run date.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(TODAY() > DATE(2025, 1, 1), "after_2025", "before_2025")
</code></pre></td><td>Labels whether today is after Jan 1, 2025.</td><td>Either <code>"after_2025"</code> or <code>"before_2025"</code> (run date dependent).</td></tr></tbody></table>

## TRIM

The The **TRIM** function removes all unnecessary whitespace from a text string, ensuring cleaner and standardized text formatting. It is commonly used to clean imported or user-entered data that may contain irregular spacing.

### Syntax

```dax
TRIM("text")
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>"text"</td><td><p>The text, column or mocker from which you want to remove extra spaces.</p><ul><li><strong>Removes</strong> all leading and trailing spaces around a text string.</li></ul></td></tr></tbody></table>

### Return value

A cleaned text string with:

* No leading or trailing whitespace.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">TRIM(" Senatus Populusque Romanus ")
</code></pre></td><td>Removes leading and trailing spaces from a literal string.</td><td><code>"Senatus Populusque Romanus"</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">TRIM([ADDRESS])
</code></pre></td><td>Removes leading and trailing spaces from <code>[ADDRESS]</code> per row.</td><td>Varies by row (e.g., <code>" 123 Main St "</code> → <code>"123 Main St"</code>).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">TRIM(MOCK_ADDRESS)
</code></pre></td><td>Trims whitespace from a generated address.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">TRIM(CONCATENATE([FIRSTNAME], " ", [LASTNAME]))
</code></pre></td><td>Concatenates first and last name and trims the result.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">LOWER(TRIM([EMAIL]))
</code></pre></td><td>Trims <code>[EMAIL]</code> first, then lowercases it.</td><td>Varies by row.</td></tr></tbody></table>

## TRUE

The `TRUE` function returns the logical value `TRUE`. Use this function to return `TRUE` based on a condition.

For example:

* `IF(2=2,TRUE(), FALSE())` returns `TRUE` if the value 2 is equal to 2.

You can also enter the value `TRUE` directly into formulas without using the function.

For example:

* `IF(2=2,TRUE, FALSE)` also returns `TRUE` if the condition is met.

If the condition is not met, both examples return `FALSE`.

### Syntax

```dax
TRUE()
```

#### Parameters <a href="#parameters" id="parameters"></a>

The `TRUE` function does not require any arguments.

### Return value

The logical value that is set to TRUE.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">IF(2 = 2, TRUE, FALSE)
</code></pre></td><td>Returns <code>TRUE</code> because the condition is always true.</td><td><code>TRUE</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">TRUE()
</code></pre></td><td>Returns the logical value <code>TRUE</code>.</td><td><code>TRUE</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF([IS_ACTIVE] = 1, TRUE(), FALSE())
</code></pre></td><td>Returns <code>TRUE</code> when <code>[IS_ACTIVE]</code> equals <code>1</code>; otherwise <code>FALSE</code>.</td><td><code>TRUE</code> or <code>FALSE</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">AND(TRUE(), [IS_ACTIVE] = 1)
</code></pre></td><td>Shows that <code>AND</code> with <code>TRUE()</code> behaves like the other condition.</td><td><code>TRUE</code> or <code>FALSE</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(TRUE(), MOCK_FIRST_NAME, MOCK_LAST_NAME)
</code></pre></td><td>Always returns the <code>then</code> branch because the condition is always true.</td><td>Varies by row.</td></tr></tbody></table>

## UPPER

The `UPPER` function converts text to uppercase.

### Syntax

```dax
UPPER("text")
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>"text"</td><td>The text, column or mocker you want to convert to uppercase. This can be a reference or a text string.</td></tr></tbody></table>

### Return value

The text that is converted to uppercase.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">UPPER("hello world!")
</code></pre></td><td>Converts a literal string to uppercase.</td><td><code>"HELLO WORLD!"</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">UPPER([ADDRESS])
</code></pre></td><td>Converts <code>[ADDRESS]</code> to uppercase per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">UPPER(MOCK_COMPANY_EMAIL)
</code></pre></td><td>Converts a generated company email to uppercase.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">UPPER(LOWER([EMAIL]))
</code></pre></td><td>Normalizes <code>[EMAIL]</code> by lowercasing first, then uppercasing.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">CONCATENATE(UPPER([STATE]), "-", RIGHT(CONCATENATE("00000", [COUNTRY_CODE]), 5))
</code></pre></td><td>Builds <code>"STATE-#####"</code> from state and a left-padded ZIP.</td><td>Varies by row (e.g., <code>"MASSACHUSETTS-02134"</code>).</td></tr></tbody></table>

## VAR

The `VAR` function calculates the variance based on a sample of a population.

* `VAR` assumes that its arguments represent a sample of the population. If your data represents the entire population, use the `VARP` function instead.
* Arguments can be numbers, names, arrays, or values that contain numbers.
* Logical values and text representations of numbers entered directly into the argument list are included in the calculation.
* If an argument is an array, only the numbers within that array are considered; empty values, logical values, text, or error values in the array are ignored.
* Arguments that contain error values or text that cannot be converted into numbers will cause the function to return an error.
* If you want to include logical values and text representations of numbers in the calculation, use the `VARA` function.

### Syntax

```dataweave
VAR(value1, value2, ... value30)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>value1</td><td>The first number, column or mocker corresponding to a sample of a population.</td></tr><tr><td>value2, ... value30</td><td>Additional arguments corresponding to a sample of a population, up to 30 total arguments.</td></tr></tbody></table>

### Return value

Variance of a sample.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">VAR(10, 22, 45, 65, 6, 63, 23, 89, 46, 54)
</code></pre></td><td>Computes the sample variance of the provided literal values.</td><td><code>~714.23</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">VAR([AMOUNT_COVERED], [AMOUNT_UNCOVERED], [COVERED_ENCOUNTERS], [COVERED_PROCEDURES])
</code></pre></td><td>Computes the sample variance across multiple numeric columns for each row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">VAR([AMOUNT_COVERED], 0, 100, 250)
</code></pre></td><td>Computes the sample variance of a row value combined with constants.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(VAR([AMOUNT_COVERED], [AMOUNT_UNCOVERED], [COPAY_AMOUNT]), 2)
</code></pre></td><td>Computes sample variance across three columns, then rounds to 2 decimals.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">SQRT(VAR(10, 20, 30, 40, 50))
</code></pre></td><td>Computes the square root of the sample variance (i.e., sample standard deviation) of the provided values.</td><td><code>~15.811</code></td></tr></tbody></table>

## VAR.P

The `VAR.P` function calculates the variance based on the entire population, ignoring logical values and text.

* `VAR.P` assumes that its arguments represent the entire population. If your data represents only a sample of the population, use the `VAR.S` function instead.
* Arguments can include numbers, names, arrays, or values that contain numbers.
* Logical values and text representations of numbers entered directly into the argument list are included in the calculation.
* If an argument is an array, only the numbers within that array are considered; empty values, logical values, text, or error values are ignored.
* Arguments that contain error values or text that cannot be converted into numbers will cause the function to return an error.
* If you want to include logical values and text representations of numbers in the calculation, use the `VARPA` function.

### Syntax

```dax
VAR.P(value1, value2, ... value30)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>value1</td><td>The first number, column or mocker corresponding to the population.</td></tr><tr><td>value1, value2, ... value30</td><td>Additional arguments corresponding to the population, up to 30 total arguments.</td></tr></tbody></table>

### Return value

Variance of a population.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">VAR.P(10, 22, 45, 65, 6, 63, 23, 89, 46, 54)
</code></pre></td><td>Computes the population variance of the provided literal values.</td><td><code>~642.81</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">VAR.P([AMOUNT_COVERED], [AMOUNT_UNCOVERED], [COVERED_ENCOUNTERS], [COVERED_PROCEDURES])
</code></pre></td><td>Computes the population variance across multiple numeric columns for each row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">VAR.P([AMOUNT_COVERED], [AMOUNT_UNCOVERED])
</code></pre></td><td>Computes the population variance across two columns for each row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">VAR.P([AMOUNT_COVERED], [AMOUNT_UNCOVERED], [COPAY_AMOUNT])
</code></pre></td><td>Computes the population variance across three columns for each row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(VAR.P(10, 22, 45, 65, 6), 2)
</code></pre></td><td>Computes population variance, then rounds to 2 decimals.</td><td><code>497.84</code></td></tr></tbody></table>

## VAR.S

The `VAR.S` function estimates variance based on a sample, ignoring logical values and text.

* `VAR.S` assumes that its arguments represent a sample of the population. If your data represents the entire population, use the `VAR.P` function instead.
* Arguments can include numbers, names, arrays, or values that contain numbers.
* Logical values and text representations of numbers entered directly into the argument list are included in the calculation.
* If an argument is an array, only the numbers within that array are considered; empty values, logical values, text, or error values are ignored.
* Arguments that contain error values or text that cannot be converted into numbers will cause the function to return an error.
* If you want to include logical values and text representations of numbers in the calculation, use the `VARA` function.

### Syntax

```dax
VAR.S(value1, value2, ... value30)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>value1</td><td>The first number, column or mocker corresponding to a sample of the population.</td></tr><tr><td>value2, ... value30</td><td>Additional arguments corresponding to a sample of the population, up to 30 total arguments.</td></tr></tbody></table>

### Return value

Variance of a sample.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">VAR.S(10, 22, 45, 65, 6, 63, 23, 89, 46, 54)
</code></pre></td><td>Computes the sample variance of the provided literal values.</td><td><code>~714.23</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">VAR.S([AMOUNT_COVERED], [AMOUNT_UNCOVERED], [COVERED_ENCOUNTERS], [COVERED_PROCEDURES])
</code></pre></td><td>Computes the sample variance across multiple numeric columns for each row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">VAR.S([HEALTHCARE_EXPENSES], [HEALTHCARE_COVERAGE], [COPAY_AMOUNT])
</code></pre></td><td>Computes the sample variance across three columns for each row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">VAR.S([AMOUNT_COVERED], [AMOUNT_UNCOVERED], [COPAY_AMOUNT], 0)
</code></pre></td><td>Computes the sample variance across three columns plus a constant <code>0</code> for each row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(VAR.S(10, 22, 45, 65, 6), 2)
</code></pre></td><td>Computes sample variance, then rounds to 2 decimals.</td><td><code>622.30</code></td></tr></tbody></table>

## VARA

The `VARA` function estimates variance based on a sample, considering numbers, text representations of numbers, and logical values.

* `VARA` assumes that its arguments represent a sample of the population. If your data represents the entire population, use the `VARPA` function instead.
* Arguments can include numbers, names, arrays, text representations of numbers, or logical values such as `TRUE` and `FALSE`.
* Logical values and text representations of numbers entered directly into the argument list are included in the calculation.
  * Arguments containing `TRUE` evaluate as 1.
  * Arguments containing text or `FALSE` evaluate as 0.
* If an argument is an array, only the values within that array are considered. Empty values and text within the array are ignored.
* Arguments that contain error values or text that cannot be converted into numbers will cause the function to return an error.
* If you do not want to include logical values and text representations of numbers in the calculation, use the `VAR` function.

### Syntax

```dataweave
VARA(value1, value2, ... value30)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>value1, value2, ... value30</td><td><code>value1</code> is required, and subsequent values are optional. You can provide 1 to 30 values corresponding to a sample of the population.</td></tr></tbody></table>

### Return value

Variance of a sample.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">VARA(10, 22, 45, 65, 6, 63, 23, 89, 46, 54)
</code></pre></td><td>Computes sample variance, including any logical/text values (none in this example).</td><td><code>~714.23</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">VARA([AMOUNT_COVERED], [AMOUNT_UNCOVERED], [COVERED_ENCOUNTERS], [COVERED_PROCEDURES])
</code></pre></td><td>Computes sample variance across multiple columns for each row, including any logical/text values.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">VARA(TRUE, FALSE, 10, "7")
</code></pre></td><td>Computes sample variance including booleans and text numbers (<code>TRUE=1</code>, <code>FALSE=0</code>, <code>"7"=7</code>).</td><td><code>23</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">VARA(TRUE, FALSE, [FLAG_A], [FLAG_B], [FLAG_C])
</code></pre></td><td>Computes sample variance including booleans and per-row flag columns.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(VARA(10, 20, 30, TRUE, "7"), 2)
</code></pre></td><td>Computes <code>VARA</code> including booleans/text, then rounds to 2 decimals.</td><td><code>131.30</code></td></tr></tbody></table>

## VARP

The `VARP` function calculates the variance based on the entire population.

* `VARP` assumes that its arguments represent the entire population. If your data represents a sample of the population, use the `VAR` function instead.
* Arguments can include numbers, names, arrays, or values that contain numbers.
* Logical values and text representations of numbers entered directly into the argument list are included in the calculation.
* If an argument is an array, only the numbers within that array are considered; empty values, logical values, text, or error values in the array are ignored.
* Arguments that contain error values or text that cannot be converted into numbers will cause the function to return an error.
* If you want to include logical values and text representations of numbers in the calculation, use the `VARPA` function.

### Syntax

```dataweave
VARP(value1, value2, ... value30)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>value1</td><td>The first number, column or mocker corresponding to the population.</td></tr><tr><td>value2, ... value30</td><td>Additional arguments corresponding to the population, up to 30 total arguments.</td></tr></tbody></table>

### Return value

Variance of a population.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">VARP(10, 22, 45, 65, 6, 63, 23, 89, 46, 54)
</code></pre></td><td>Computes the population variance of the provided literal values.</td><td><code>~642.81</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">VARP([AMOUNT_COVERED], [AMOUNT_UNCOVERED], [COVERED_ENCOUNTERS], [COVERED_PROCEDURES])
</code></pre></td><td>Computes the population variance across multiple numeric columns for each row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">VARP([AMOUNT_COVERED], [AMOUNT_UNCOVERED])
</code></pre></td><td>Computes the population variance across two columns for each row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">VARP([AMOUNT_COVERED], [AMOUNT_UNCOVERED], [COPAY_AMOUNT])
</code></pre></td><td>Computes the population variance across three columns for each row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">ROUND(VARP(10, 22, 45, 65, 6), 2)
</code></pre></td><td>Computes population variance, then rounds to 2 decimals.</td><td><code>497.84</code></td></tr></tbody></table>

## WEEKNUM

The `WEEKNUM` function returns the week number of a specific date. For example, the week containing January 1 is considered the first week of the year and is numbered as week 1.

### Syntax

```dax
WEEKNUM(date, [return_type])
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>date</td><td>A date, column or mocker within the week you want to identify. Dates should be entered using the <code>DATE</code> function or as the result of other formulas or functions. Entering dates as text may cause errors.</td></tr><tr><td>[return_type]</td><td>A number that determines which day of the week the week starts on. The default value is 1 (where Sunday is the first day of the week).</td></tr></tbody></table>

### Return value

The week number in the year.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">WEEKNUM("2012-08-24")
</code></pre></td><td>Returns the week number for a literal date string.</td><td>A week number (locale dependent).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">WEEKNUM([BIRTHDATE])
</code></pre></td><td>Returns the week number for <code>[BIRTHDATE]</code> per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">WEEKNUM(DATE(2024, 1, 1), 2)
</code></pre></td><td>Returns the week number for Jan 1, 2024 with Monday as the week start (<code>return_type=2</code>).</td><td><code>1</code> (depending on week-numbering rules).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">WEEKNUM(TODAY())
</code></pre></td><td>Returns the current week number.</td><td>Varies by run date.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">WEEKNUM(DATEADD([ORDER_DATE], 7, "day"), 2)
</code></pre></td><td>Shifts <code>[ORDER_DATE]</code> by 7 days, then returns week number with Monday as week start.</td><td>Varies by row.</td></tr></tbody></table>

## XOR

The `XOR` function returns a logical Exclusive Or (XOR) of all the provided arguments.

* The arguments must evaluate to logical values such as `TRUE` or `FALSE`, or be in arrays or references that contain logical values.
* If an array or reference argument contains text or empty cells, those values are ignored.
* If the specified range contains no logical values, `XOR` returns the `#VALUE!` error.
* You can use an XOR array formula to check if a value occurs in an array.
* The result of `XOR` is `TRUE` when the number of `TRUE` inputs is odd and `FALSE` when the number of `TRUE` inputs is even.

### Syntax

```dataweave
XOR(Logicalvalue1, Logicalvalue2 ... Logicalvalue30)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>Logicalvalue1, Logicalvalue2 ... Logicalvalue30</td><td><code>Logical1</code> is required, and subsequent logical values are optional. You can provide 1 to 30 conditions to test, which can be either <code>TRUE</code> or <code>FALSE</code>, and can include logical values, arrays, or references.</td></tr></tbody></table>

### Return value

TRUE if an odd number of arguments evaluates to TRUE.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">XOR([GENDER] = "M", [HEALTHCARE_COVERAGE] &#x3C; 250, [HEALTHCARE_EXPENSES] > 1000)
</code></pre></td><td>Returns <code>TRUE</code> when an odd number of conditions are true.</td><td><code>TRUE</code> or <code>FALSE</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">XOR(TRUE, FALSE)
</code></pre></td><td>Computes XOR for two literal booleans.</td><td><code>TRUE</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">XOR([FLAG_A] = 1, [FLAG_B] = 1, [FLAG_C] = 1, [FLAG_D] = 1)
</code></pre></td><td>Returns <code>TRUE</code> when 1 or 3 of the flags are true.</td><td><code>TRUE</code> or <code>FALSE</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(XOR([FLAG_A] = 1, [FLAG_B] = 1), "exactly_one", "zero_or_two")
</code></pre></td><td>Labels rows where exactly one of two flags is set.</td><td>Either <code>"exactly_one"</code> or <code>"zero_or_two"</code> per row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">XOR(AND([A] = 1, [B] = 1), OR([C] = 1, [D] = 1))
</code></pre></td><td>Computes XOR of two composite logical expressions.</td><td><code>TRUE</code> or <code>FALSE</code> per row.</td></tr></tbody></table>

## YEAR

The `YEAR` function returns the year corresponding to a given date as an integer between 1900 and 9999. The values returned by the `YEAR`, `MONTH`, and `DAY` functions will be Gregorian values, regardless of the display format for the supplied date.

### Syntax

```dax
YEAR(number)
```

#### Parameters <a href="#parameters" id="parameters"></a>

<table><thead><tr><th width="208">Parameter</th><th>Description</th></tr></thead><tbody><tr><td>number</td><td>The date, column or mocker for which you want to extract the year. Dates should be entered using the <code>DATE</code> function or as the result of other formulas or functions. Entering dates as text may cause errors.</td></tr></tbody></table>

### Return value

The year as a number according to the internal calculation rules.

### Examples <a href="#example---shifting-a-set-of-dates" id="example---shifting-a-set-of-dates"></a>

<table><thead><tr><th width="360">Formula</th><th>Description</th><th>Result</th></tr></thead><tbody><tr><td><pre class="language-dax"><code class="lang-dax">YEAR("24/8/2012")
</code></pre></td><td>Extracts the year from a literal date string.</td><td><code>2012</code> (locale dependent).</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">YEAR([BIRTHDATE])
</code></pre></td><td>Extracts the year from <code>[BIRTHDATE]</code> per row.</td><td>Varies by row.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">YEAR(DATE(2024, 12, 31))
</code></pre></td><td>Extracts the year from a constructed date.</td><td><code>2024</code></td></tr><tr><td><pre class="language-dax"><code class="lang-dax">YEAR(TODAY())
</code></pre></td><td>Extracts the current year.</td><td>Varies by run date.</td></tr><tr><td><pre class="language-dax"><code class="lang-dax">IF(YEAR([ORDER_DATE]) &#x3C; 2000, "legacy", "modern")
</code></pre></td><td>Labels orders before 2000 as legacy.</td><td>Either <code>"legacy"</code> or <code>"modern"</code> per row.</td></tr></tbody></table>

{% hint style="info" %}
**Info:** The below functions are planned to be introduced in the future releases of Syntho.
{% endhint %}

### Planned Functions for future releases

| Function   | Description                                                                                                                     | Format                                 |
| ---------- | ------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------- |
| STDEV      | Returns standard deviation of a sample. When used with column names, the value is calculated anew for each row.                 | STDEV(Value1, Value2, ... Value30)     |
| STDEV.P    | Returns standard deviation of a population. When used with column names, the value is calculated anew for each row.             | STDEV.P(Value1, Value2, ... Value30)   |
| STDEV.S    | Returns standard deviation of a sample. When used with column names, the value is calculated anew for each row.                 | STDEV.S(Value1, Value2, ... Value30)   |
| STDEVA     | Returns standard deviation of a sample. When used with column names, the value is calculated anew for each row.                 | STDEVA(Value1, Value2, ... Value30)    |
| STDEVP     | Returns standard deviation of a population. When used with column names, the value is calculated anew for each row.             | STDEVP(Value1, Value2, ... Value30)    |
| STDEVPA    | Returns standard deviation of a population. When used with column names, the value is calculated anew for each row.             | STDEVPA(Value1, Value2, ... Value30)   |
| MAX        | Returns the maximum value in a list of arguments. When used with column names, the value is calculated anew for each row.       | MAX(Number1, Number2, ... Number30)    |
| MAXA       | Returns the maximum value in a list of arguments. When used with column names, the value is calculated anew for each row.       | MAXA(Value1, Value2, ... Value30)      |
| MEDIAN     | Returns the median of a set of numbers. When used with column names, the value is calculated anew for each row.                 | MEDIAN(Number1, Number2, ... Number30) |
| MIN        | Returns the minimum value in a list of arguments. When used with column names, the value is calculated anew for each row.       | MIN(Number1, Number2, ... Number30)    |
| MINA       | Returns the minimum value in a list of arguments. When used with column names, the value is calculated anew for each row.       | MINA(Value1, Value2, ... Value30)      |
| COUNT      | Counts how many numbers are in the list of arguments.                                                                           | COUNT(Value1, Value2, ... Value30)     |
| COUNTA     | Counts how many values are in the list of arguments.                                                                            | COUNTA(Value1, Value2, ... Value30)    |
| COUNTBLANK | Returns the number of empty cells.                                                                                              | COUNTBLANK(Range)                      |
| IFNA       | Returns the value if the cell does not contain the #N/A (value not available) error value, or the alternative value if it does. | IFNA(Value, Alternate\_value)          |


# Key generators

Syntho's Primary Key (PK) and Foreign Key (FK) generation methods are designed to enhance data privacy, integrity, and uniqueness across various tables. These methods generate unique hash values that mimic input data formats, ensuring referential integrity and maintaining the data structure.

Syntho offers three key generators to handle various scenarios for managing and transforming primary and foreign keys. Below are the descriptions and recommended use cases for **Duplicate**, **Generate**, and **Hash**.

Syntho supports three types of key generators to handle referential integrity:

| Method    | Description                                                                                                                                                               | When to use                                                                                                                                                                                    | When not to use                                                                                                                                                                                                                                                       |
| --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Duplicate | Copies the original key values exactly as they appear in the source data, preserving both the correlations and referential integrity between primary and foreign keys.    | When it’s essential to maintain the original key values and relationships, particularly in de-identification scenarios where the data structure must be preserved without generating new keys. | Upsampling is not supported when using Duplicate, as the original keys are simply copied, not expanded. Additionally, it is not recommended when the keys are sensitive and need to be protected, as this method retains the original key values without obfuscation. |
| Generate  | Creates new, synthetic key values that do not correspond to the original keys. It preserves only the referential integrity, but not the correlations between key columns. | Use Generate for upsampling or creating synthetic datasets where there is no need to maintain relationships with the original data. It can also be used when creating data from scratch.       | The Generate function creates new keys independently of the original key order, which disrupts correlations. As a result, it is unsuitable for scenarios where maintaining the correlations and order is essential.                                                   |
| Hash      | Converts original key values into hashed representations. Both correlations between tables and relational integrity are maintained.                                       | Use Hash when you need to obscure the original key values, while ensuring correlations and referential integrity are preserved.                                                                | Upsampling, or situations where the original key values must be maintained for direct referencing, such as cases where exact key values are essential for business logic (e.g. country codes) or traceability in audit scenarios.                                     |

## **How to apply PK / FK generators**

### **On column level**

{% hint style="info" %}
**Syntho beta feature**
{% endhint %}

Go to **Column settings > Generation Method**, and select one of the available methods: **Duplicate**, **Generate** or **Hash**.

<figure><img src="/files/Sw8mdZO1ikuu96UzNJN7" alt="" width="563"><figcaption><p>Applying PK/FK generators on column level</p></figcaption></figure>

### **On workspace level**

{% hint style="info" %}
Setting a key column at the column level overrides the workspace's default key generation method. For example, if "Hash" is the default key generation method but "Duplicate" is set on the column level for "key\_column," then "key\_column" will use "Duplicate."
{% endhint %}

* Use the shortcut `CTRL + SHIFT + ALT + 0` to open **Workspace Default Settings** and change the key generation method according to your choice by simply replacing the value to either "duplicate", "generate" or "hash" . Please note that it will apply the generator across the entire workspace.
* Alternatively, you can add **/global\_settings** to the end of the workspace URL to open **Workspace Default Settings.**

### **On composite keys**

* The generator supports composite keys and ensures consistency across all columns that form the composite key.

{% hint style="info" %}
Setting one composite key column to use a generation method automatically sets the other composite key columns to the same method as well in UI. Attempting to change the generation method for one composite key column (e.g., from "hash" to "duplicate" or "generate") will result in all composite key columns being reset to the same generation method. The UI shows linked key generation methods, but the destination database will have the correct independent generation methods applied.
{% endhint %}

## **Data types**

* **Textual data:**
  * Hash key values ensure unique and protected data while maintaining the original text structure.
* **Numeric data:**
  * Generation methods apply unique hashes to maintain numeric data privacy and integrity.
* **Date/Time data:**
  * Composite keys involving date/time fields will see all parts of the composite key aligned to the same generation method when hashes are applied.

The examples below shows two different composite keys. In the first image, the columns "Birthdate" and "First" are applied "generate" and "hash", respectively.

<figure><img src="/files/SUk3Zl9fUIODeotz7Azc" alt=""><figcaption><p>Columns "Birthdate" and "First" are composite keys</p></figcaption></figure>

In the case below, the columns "Birthdate" and "Birthplace" are applied "hash" and "generate", respectively.

<figure><img src="/files/7MdOizsxQcFneQnytfBk" alt=""><figcaption><p>Columns "Birthdate" and "Birthplace" are composite keys</p></figcaption></figure>

## **Limitations & considerations**

* Key generator **Generate** is not supported for composite foreign keys or columns referencing a composite key (i.e. unique constraint).
* Generators are not applied to tables in **Exclude** mode.
* Consistency is enabled (same input → same output) to preserve links between PK and FK columns.
* Generation of keys for non-textual or highly complex data types, like images or large binary objects, is not supported.
* In case you want to generate keys from scratch, set the default key generation method for you workspace to **generate**.


# Differences between key generators

Here we aim to visualize the differences between various key generators and their impact on preserving referential integrity and cross-table relationships, using simplified sample data for illustration purposes.

## De-identification with key generator: duplicate

In this method, the primary keys (PKs) and foreign keys (FKs) are duplicated exactly as they are in the source data, preserving the original relationships.

<figure><img src="/files/XCwd6FSEYOutgIwjkDgL" alt=""><figcaption><p>Duplicate as key generator in de-identification</p></figcaption></figure>

**Preservation of Keys**: The primary keys (ID) and foreign keys (Patient ID) in the de-identified data are exact duplicates of those in the original data.

**Referential Integrity**: Since the keys are duplicated, the referential integrity is maintained, ensuring that each foreign key in the Medications table corresponds to an existing primary key in the Patients table.

## De-identification with key generator: hash

In this method, the primary keys (PKs) and foreign keys (FKs) are transformed using a hash function. This preserves the referential integrity while anonymizing the keys. The primary keys and foreign keys will be hashed, preserving the relationship between tables while anonymizing the keys.

<figure><img src="/files/LtpDh3FldMwbKDiBRlyb" alt=""><figcaption><p>Hash as key generator in de-identification</p></figcaption></figure>

**Preservation of Keys**: The primary keys (ID) and foreign keys (Patient ID) are transformed using a hash function, ensuring they are anonymized while preserving their referential integrity. The hash data is dependent on the data type. The relationships between the Patients and Medications tables are maintained because the hashed foreign keys in the Medications table correspond to the hashed primary keys in the Patients table.

**Referential Integrity**: The relationships between the Patients and Medications tables are preserved because the hashed foreign keys in the Medications table match the hashed primary keys in the Patients table.

**Consistency**: The hash function consistently maps the same original key to the same hashed key, ensuring consistency of hashed values across tables, databases and data generation jobs.

## De-identification with key generator: generate (not recommended)

In this method, entirely new key values are generated, which preserves referential integrity but does not maintain the original order of the key values. New keys will be generated for both primary keys and foreign keys, preserving referential integrity but not the original order.

<figure><img src="/files/eTn6fWwIWcPqQgNxkYXd" alt=""><figcaption><p>Generate as key generator in de-identification</p></figcaption></figure>

**Generation of New Keys**: New primary keys (ID) and foreign keys (Patient ID) are generated, ensuring they are unique but not maintaining their original order. This key method is generally not recommended in combination with de-identification.

**Referential Integrity**: The relationships between the Patients and Medications tables are preserved because the foreign keys in the Medications table correspond to the new primary keys in the Patients table.

**Order of Keys**: The new keys do not maintain the original order. The foreign keys in the Medications table are generated based on the primary keys in the Patients table and then uniformly assigned to the Medications table using a "tiling" method. This means the IDs will repeat in a uniform pattern, such as 101, 102, 103, 104, 105, 101, 102, 103, 104, 105, etc.

## Synthesize with single table model with key generator: generate

In this method, entirely new key values and combinations of non-key values are generated for a single table model, preserving relationships between non-key columns and statistical properties, but not maintaining the original order of non-key values. New values are generated for both primary keys and combinations of non-key values, preserving relationships between non-key columns and statistical properties, but not the original order of non-key values.

<figure><img src="/files/Hh1o911YXtSBKxl2EcLC" alt=""><figcaption><p>Generate as key generator in synthesizing single table model</p></figcaption></figure>

**Generation of New Values**: New primary keys (ID), foreign keys (Patient ID), and combinations of non-key values (Gender, Country, Medication, Reason) are generated. While some values from the original columns may still exist in the synthetic column, the generated rows (i.e., combinations of values) are entirely newly generated by the trained generative model.

**Referential Integrity**: The referential integrity is preserved, because each foreign key value in the synthetic Medications table corresponds to a primary key value in the synthetic Patients table.

**Order of Keys**: The new keys do not maintain the original order. The foreign keys in the Medications table are generated based on the primary keys in the Patients table and then uniformly assigned to the Medications table using a "tiling" method, such as 201, 202, 203, 204, 205, 201, 202, 203, 204, 205, etc., illustrating that there is no 1-to-1 relationship with the original rows.

**Statistical Properties**: Although the combinations of non-key values are entirely new, their statistical properties (e.g., frequency distribution, variance) are preserved, as are their relationships with other non-key columns.

**Relationships Between Non-Key Columns**: Relationships such as the connection between Medication and Reason are preserved in the generated data.

## Synthesize with single table model with key generator: hash / duplicate

In this method, key values are either hashed or duplicated while generating entirely new data for non-key columns. This preserves referential integrity and relationships between non-key columns, but the original order of the non-key values is not maintained. Statistical properties of the data are preserved. New values are generated for non-key columns, while keys are either duplicated or hashed, preserving relationships and statistical properties but not the original order of non-key values.

<figure><img src="/files/xln0h1a4djVL7PLrAKQh" alt=""><figcaption><p>Hash / Duplicate as key generator in synthesizing single table model</p></figcaption></figure>

**Duplication/Hashing of Keys**: The primary keys (ID) and foreign keys (Patient ID) are either duplicated or hashed, ensuring referential integrity.

**Generation of New Non-Key Values**: New rows are generated based on the learned patterns within and across non-key values (Gender, Country, Medication, Reason).

**Referential Integrity**: The referential integrity is preserved, because each foreign key value in the synthetic Medications table corresponds to a primary key value in the synthetic Patients table.

**Statistical Properties**: Although the non-key values are entirely new, their statistical properties (e.g., frequency distribution, variance) are preserved.

**Relationships Between Non-Key Columns**: Statistical relationships between columns are typically preserved in the generated data.


# JSON de-identification

{% hint style="info" %}
**Coming Soon**
{% endhint %}

The JSON de-identification feature allows users to apply targeted masking to specific sub-values within complex JSON structures. This feature enables precise and flexible data de-identification based on user-defined JSONPath expressions, helping users efficiently manage sensitive data in nested JSON structures. Below is an overview of how to use this feature, followed by detailed instructions.

## Apply a JSON de-identification

1. Open your **Workspace**.
2. From the **Main hub** or **Table view** tab, select the column where you want to apply a generator.
3. Under **Generator,** select **JSON de-identification**.
4. Either:
   * **Select the value from the cell to de-identify** (The expression input field will then be populated based on this selection), or
   * Enter a **JSONPath expression** to specify which sub-values within the JSON structure the generator should target.
5. Assign **Sub-Generators**:
   * In the **Sub-generators settings** panel, configure each sub-generator based on the specific attributes you wish to mask. Available generators in sub-generators are [**Duplicate**](/configure-a-data-generation-job/configure-column-settings/duplicate), [**Mocker**](/configure-a-data-generation-job/configure-column-settings/mockers), [**Calculated Columns**](/configure-a-data-generation-job/configure-column-settings/calculated-columns), [**Mask**](/configure-a-data-generation-job/configure-column-settings/mask), [**Exclude**](/configure-a-data-generation-job/configure-column-settings/exclude).
   * For instance, apply a numeric range mocker to `children.age`, a first name mocker to `children.first_name`, and a last name mocker to `children.last_name`.
   * Sub-generators are applied in the order they appear in the list; you can reorder them as needed by dragging each item.
6. Configure **Fallback generator**:
   * Select a fallback generator from the dropdown menu to apply to values that do not match the defined JSONPath criteria or result in errors. The default option is **Duplicate** and the other option is **Exclude**.
7. Filter by **JSON type (Optional)**:
   * To increase specificity, you can restrict the generator application to particular JSON primitive types (i.e. Number, String, Boolean and Null) to match the data structure requirements.
8. **Save** and **Preview**:
   * After configuring the JSONPath expressions, sub-generators, and fallback options, click **Preview** to view how the de-identified data will look.
   * Once satisfied, click **Confirm** to save the settings.

<figure><img src="/files/5Z1VxERqGsapmdpaVw7y" alt=""><figcaption><p>Applying JSON de-identification</p></figcaption></figure>

## JSON de-identification parameters

JSON de-identification supports selective application of generators to sub-values within JSON data. Users can define complex criteria, such as JSONPath expressions, to target specific elements within JSON arrays and objects. This functionality ensures accurate data masking while preserving the structural integrity of the JSON data.

* **Sub-generators settings**: Define multiple sub-generators and apply them sequentially to sub-values based on JSONPath expressions.
* **Fallback generator**: Specify a fallback generator to handle values that cannot be masked by the main generator due to errors or mismatches.
* **Expression validation**: When a JSONPath expression is entered, users can validate it to see how many JSON elements match the criteria.
* **Filtering by data type**: Restrict generator application to specific JSON types (e.g., string, number) to ensure relevance and efficiency.


# Consistent mapping

**Consistent mapping** allows you to generate the same mock data values for a given set of original data values every time the mocker is applied. This feature can extend across various database types, allowing for consistent results. It is especially helpful when you need to consistently generate the same mock values based on the same input values, e.g. for testing or demonstration purposes.

***

## **Applicable generators**

Consistent mapping can be applied to the following generators:

* [**Mockers**](/configure-a-data-generation-job/configure-column-settings/mockers): Enable consistent mapping by selecting "Mocker" as the generation method in the column settings.
* [**Mask**](/configure-a-data-generation-job/configure-column-settings/mask): Consistent mapping can be utilized to ensure masked values remain consistent across multiple applications.
* **Hash**: Consistent mapping is inherently enabled for the Hash generator, ensuring that the same input always results in the same hashed output.

## Enable consistent mapping

* To enable consistent mapping, open column configurations by clicking the column settings for the selected table. Please see below to find how to open the window.

<figure><img src="/files/r8E7qBDkcCWCoDUfG4Kp" alt=""><figcaption><p>Click on icons to open column settings</p></figcaption></figure>

* Alternatively, clicking the “**Configure**” button after a PII scan also opens the column settings window.
* After opening column settings window, click on the box next to the “**Consistent Mapping**” to enable consistent mapping with mocker. Keep in mind that you have to select “**Mocker**”, "**Mask**" or "**Hash**" as “**Generation Method**”.

<figure><img src="/files/iuHmNoqkA94IXoyHvO04" alt="" width="553"><figcaption><p>Toggle to enable Consistent Mapping</p></figcaption></figure>

## Advantages of consistent mapping

**Linking data**: Even if your database doesn't force certain rules on how data is connected (like making sure email addresses match up), consistency lets you match things together anyway. For example, you can mock last names to keep them private but still link related columns.

**Preserving distributions**: If you have a list of different items, like 20 job titles in a column, and you want to mix them up without losing the overall distributions (still around 20 job titles), consistency is your friend. It keeps the distributions about the same.

However, remember that while consistency keeps the variety, it might not keep every item unique. The variety won't grow, but it could change slightly. If you need each item to match one-to-one to a new unique value, you will have to make the **mocker unique**.

**Matching data across systems**: If you're dealing with data spread out over different databases, like names in one place and email addresses in another, consistency helps you keep the names the same everywhere, even after making them private. This way, everything is still matched up correctly without sharing private information.

## Consistent mapping example

Assume we have a table where the first name “Mavis612” appears twice. If you enable consistent mapping, both of these will be mapped to the same name, consistently, across the tables. See the illustrations below from MySQL tables where mockers with consistent mapping map the name “Mavis612” to “Jillian”. Please note that other names can also be mapped to Jillian. **Consistent mapping** does not suggest that only and only “Mavis612” will be replaced with “Jillian”.

<figure><img src="/files/YaV0QC4iicEFyKUIEccf" alt=""><figcaption><p>An example of consistent mapping</p></figcaption></figure>

## Understanding consistent mapping

**Consistent mapping equals predictability**: If you feed the same data into a mocker with **consistent mapping** enabled, you'll always get the same result. It's like using the same recipe every time to bake a cake; the outcome is predictable.

**Uniqueness not guaranteed**: Just because the process is consistent doesn't mean every different piece of data will come out differently. Two distinct inputs might lead to the same output. Think of it as different ingredients sometimes making a cake taste similar.

### Privacy implications

**No mappings stored**: The Syntho platform uses a [seed algorithm](https://pypi.org/project/pyfarmhash/), which means it doesn't store any information about the transformation of input to output values. So, you can see "Karl" appears 10 times, but not that "Karl" was originally "Immanuel." Also, you may change how input values are mapped to output values.

**Reduced privacy**: Using **consistent mapping** might reveal some information, like how often certain data appears. For example, if "Karl" shows up 10 times, that pattern stays the same after the data is processed.

### How consistency works across databases

**Whole database application**: Consistency applies across your entire database, not just within a single table. If you use **consistent mapping** for names in both a **Customers** and an **Employees** table, the same original name will always be updated to the same new name across both tables.

**Not automatically across multiple jobs**: By default, if you generate data more than once, consistency is automatically carried over from one generation job to the next. In case you don't want this, and you want to change consistency over jobs, you can set a different seed value. You can do this by going to the **Workspace Default Settings**, by selecting `CTRL + ALT + SHIFT + 0`, and update the `seed_value` to another integer value.

In short, consistency in data generation helps make sure your data behaves predictably, while also addressing the balance between maintaining useful patterns and protecting privacy.

### Seed

The consistent mapping scheme is based on a defined **seed** value. For example, if `seed=1` and consistency mapping is enabled for an Address generator, the same input address (e.g., "123 Main Street") will consistently map to the same output (e.g., "456 Elm Avenue"). If, after a period of using this consistent mapping scheme, the user decides to switch to a new seed (e.g., `seed=2`) for security reasons, the mapping will change. For instance, "123 Main Street" might then map to "789 Oak Drive" instead.

The seed value for a workspace can be changed under the [Workspace default settings](/setup-workspaces/workspace-default-settings).

### **Ordering and indexing considerations**

To ensure accurate ordering, it is essential for the application to have either an index or a primary key in the source table. In the absence of these, the application defaults to sorting based on the first column of the table. However, if the first column contains duplicate values, the ordering cannot be guaranteed, as it relies on the database's sorting algorithm to handle duplicate values. Adding an index to the source table will resolve this issue.

#### **Column set for "ORDER BY" clause**

{% hint style="info" %}
Hive only
{% endhint %}

In the Table Settings panel, a new dropdown field allows users to specify which columns should be used in the "ORDER BY" clause. This feature enables users to define a set of columns that ensure the uniqueness of the returned results for a given table. By selecting the appropriate columns, users can achieve deterministic ordering even in the absence of primary keys or indexes.

* **Order By Dropdown**: Located in the Table Settings panel on the right side of the Table/Job Configuration screen, this dropdown lets users choose the columns for the "ORDER BY" clause.

## Limitations

* **Consistent mapping** does not guarantee consistency for generated data under different Syntho platform versions, or when deployed on different OSes (e.g., Windows vs Linux), or CPU architectures (e.g., x86 vs ARM).


# Manage foreign keys

A foreign key is a column or columns of data in one table that refers to the unique data values - often the primary key data - in another table. **Foreign keys** link together two or more tables in a database. Syntho uses foreign keys for any data generation job, including subsetting.

Here are the articles in this section:

<table data-card-size="large" data-view="cards"><thead><tr><th></th><th data-hidden></th><th data-hidden></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Foreign key inheritance</strong></td><td></td><td></td><td><a href="/pages/9BusyijY4pD4qYUK6Fk1">/pages/9BusyijY4pD4qYUK6Fk1</a></td></tr><tr><td><strong>Add foreign keys</strong></td><td></td><td></td><td><a href="/pages/EaHlxqrwEoz5CbTBOclW">/pages/EaHlxqrwEoz5CbTBOclW</a></td></tr><tr><td><strong>Delete foreign keys</strong></td><td></td><td></td><td><a href="/pages/vtl9qj4f6jWwj0jiaFdM">/pages/vtl9qj4f6jWwj0jiaFdM</a></td></tr><tr><td><strong>Circular foreign key references</strong></td><td></td><td></td><td><a href="/pages/zoMf30sKGLZDZRD9ZdPP">/pages/zoMf30sKGLZDZRD9ZdPP</a></td></tr></tbody></table>


# Foreign key inheritance

Syntho automatically inherits all foreign keys that are stored in your source database. This means that the generated database will respect the foreign key constraints of your source database.

Foreign keys can be viewed when selecting the **Foreign keys** tab on the **Job configuration** screen and after unchecking the box **Show manually added foreign keys only**.

<figure><img src="/files/x3282i4hdpuLSsW8mWtf" alt=""><figcaption><p>Foreign keys tab</p></figcaption></figure>

## Limitations

* Composite foreign keys are not supported in the following situation:
  * when two or more columns that make up the composite foreign key *also* serve as a composite primary key in a referred other table, *AND*
  * the key generation method is set to **generate**


# Add virtual foreign keys

It is best practice to have foreign keys defined inside your database. However, there are cases when a database does not have any foreign key relationships explicitly defined in the database. For instance, it can be that those relationships are stored in an application layer that runs on top of the database. For these situations, Syntho enables you to add virtual foreign keys, possibly with the help of Syntho's foreign key scanner.


# Add virtual foreign keys

To add a virtual foreign key:

1. Select the **Foreign keys** tab on the **Job configuration** panel.
2. On the **Foreign keys** tab, select the tables containing the **foreign key** and primary **key** columns.\
   You can also add composite **foreign keys** and **primary keys** by selecting multiple columns.
3. To confirm adding the virtual **foreign key** after having selected the primary and foreign keys, select **Add foreign key**.

<figure><img src="/files/tQ2cilPQ17sjjBKGDnhH" alt=""><figcaption><p>Example columns as keys</p></figcaption></figure>


# Use foreign key scanner

You can use the foreign key scanner to make it easier to discover virtual foreign keys.

1. Select the **Foreign keys** tab on the **Job configuration** screen.
2. Press the **Scan** button to launch a foreign key scan.
3. Select any filters to apply to limit the amount of **foreign key** candidates.
4. Finally, select **Confirm** to launch your foreign key scan.

Once the foreign key scan is complete, you can view, confirm, or delete any foreign key candidates resulting from the scan.

## Filter foreign key candidates

To confirm starting a foreign key scan, you can enable or disable filters to limit the foreign key candidates - columns that are considered as possible foreign key options - for the scan.

* **Only include exact column name matches**: When enabled, only column pairs with the exact same column names are considered foreign key candidates. When disabled, column names are not considered to limit the possible foreign key candidates.
* **Only include exact data type matches**: When enabled, only column pairs with the exact same data types are considered foreign key candidates. When disabled, two columns must still have a ***compatible*** data type, but it does not need to be an exact match.
* **FK candidates must link to an existing primary key**: When enabled, foreign key candidates must always link to an existing primary key column. When disabled, foreign key candidates can also be columns that are not defined as primary keys in the database, but can be identified as primary keys based on Syntho logic, considering the cardinality of the columns.
  * **Include string values as primary key candidates**: When enabled, columns with a data type that matches a *String* type (e.g. `VARCHAR` and `TEXT`) are included as possible candidates. When disabled, String type columns are excluded as possible foreign key candidates.
* **Apply bi-directional data validation**: When enabled, column pairs are considered, with the values in column A also appearing in column B and the values in column B also appearing in column A. When disabled, one-way validation applies, requiring either the values from column A to exist in column B, and/or vice versa.

{% hint style="info" %}
Applying filters reduces the list of foreign key candidates, but is likely to result in a higher proportion of foreign key suggestions that are correct.
{% endhint %}

## Limitations of the foreign key scanner

It is important to consider several characteristics of the foreign key scanner:

* **Performance on Large Databases:** Although the foreign key scanner is designed to operate in parallel for efficiency, scanning databases with hundreds of millions of rows will still take a considerable amount of time.
* **No Support for Composite Foreign Keys:** The scanner doesn't take into account composite foreign keys; it only considers individual columns.
* **Assumptions on Database Structure:** The scanner operates based on certain assumptions about the database, such as descriptive column names and correctly defined data types. If your database doesn't follow standard design best practices, you should disable the strict matching criteria for both column names and data types to allow the identification of more potential foreign key relationships.
* **Indeterminate Foreign Key Direction:** Sometimes, Syntho may not be able to determine the direction of a foreign key relationship (i.e., whether column A points to column B or vice versa). In such cases, both options will appear in the foreign key list, and manual review is strongly advised to validate the results.

Understanding these limitations will help you use the foreign key scanner more effectively and be aware of its constraints.


# Import foreign keys via JSON

{% hint style="info" %}
**Upcoming Syntho feature**

This upcoming Syntho feature is intended for users who want to import foreign key relationships via a JSON file.
{% endhint %}

You can import a **JSON** file that contains foreign keys, to define the (virtual) key relationships in your workspace.

If you already have virtual foreign keys set up, they will be replaced by the keys contained in the uploaded **JSON** file.

To import a foreign key **JSON** file:

1. Head over to the **Foreign Keys** tab.
2. Select the **Upload foreign keys** button.
3. Use the **Browse** button to search for and select the JSON file you wish to upload.
4. After selecting the file, simply click **Import**.

The uploaded foreign keys will be added to the **Foreign Keys** list and any existing virtual foreign keys will be replaced.

The contents of **JSON** file would look like below:

```json
[
   {
      "apply":true,
      "fk_columns":[
         "FK_COLUMN_NAME"
      ],
      "fk_schema":"FK_SCHEMA_NAME",
      "fk_table":"FK_TABLE_NAME",
      "pk_columns":[
         "PK_COLUMN_NAME"
      ],
      "pk_schema":"PK_SCHEMA_NAME",
      "pk_table":"PK_TABLE_NAME"
   },
   {
      "apply":true,
      "fk_columns":[
         "FK_COLUMN_NAME"
      ],
      "fk_schema":"FK_SCHEMA_NAME",
      "fk_table":"FK_TABLE_NAME",
      "pk_columns":[
         "PK_COLUMN_NAME"
      ],
      "pk_schema":"PK_SCHEMA_NAME",
      "pk_table":"PK_TABLE_NAME"
   }
]
```

Users are required to configure the following parameters according to their needs:

* `apply`: If set to `false`, the foreign key will be added but not activated. If set to `true`, it will be activated.
* `FK_COLUMN_NAME`: The column that is designated as a foreign key.
* `FK_TABLE_NAME`: The table where the foreign key column is located.
* `FK_SCHEMA_NAME`: The schema containing the table with the foreign key.
* `PK_COLUMN_NAME`: The column that is designated as a primary key.
* `PK_TABLE_NAME`: The table where the primary key column is located.
* `PK_SCHEMA_NAME`: The schema containing the table with the primary key.

<figure><img src="/files/0Q26r28vdo9UM6hksXt2" alt=""><figcaption><p>The direct influence of configuration "apply" in Syntho UI</p></figcaption></figure>

The description above elucidates how the "apply" configuration within the foreign key JSON file affects the process. If `apply` is set to `false`, the foreign key will be added but remain inactive, illustrated by an unchecked box. Conversely, if `apply` is set to `true`, the foreign key will be activated immediately upon upload.

Additionally, a subsequent illustration will guide the reader on locating schemas, tables, and columns within an example database, such as MySQL. This information is mirrored in the foreign key **JSON** file (import file), offering valuable insights for configuration.

<figure><img src="/files/RsVcW18MXt9zcqP7tY9o" alt="" width="375"><figcaption><p>MySQL hierarchy</p></figcaption></figure>


# Export foreign keys via JSON

You can export a JSON file containing foreign keys to populate them across multiple workspaces sharing the same source data structure. To export a foreign key JSON file, follow these steps:

1. Navigate to the **Foreign Keys** tab.
2. Click the "**Download foreign keys**" button.

The downloaded JSON file will contain the foreign keys and their configurations.

Below is an example of a foreign key JSON file downloaded from the Syntho workspace:

```json
[
   {
      "apply":true,
      "fk_columns":[
         "PATIENT"
      ],
      "fk_schema":"covid_patient_sample_db_synthetic",
      "fk_table":"encounters",
      "pk_columns":[
         "Id"
      ],
      "pk_schema":"covid_patient_sample_db_synthetic",
      "pk_table":"patients"
   },
   {
      "apply":true,
      "fk_columns":[
         "PAYER"
      ],
      "fk_schema":"covid_patient_sample_db_synthetic",
      "fk_table":"encounters",
      "pk_columns":[
         "Id"
      ],
      "pk_schema":"covid_patient_sample_db_synthetic",
      "pk_table":"payers"
   },
```

In this example, the first foreign key column is "**PATIENT**," located in the "**encounters**" table under the schema "**covid\_patient\_sample\_db\_synthetic**." This foreign key is linked to the primary key "**Id**" in the "**patients**" table, which is under the same schema, "**covid\_patient\_sample\_db\_synthetic**."


# Delete foreign keys

You can delete manually added foreign keys. You cannot delete foreign keys that are defined in the source database.

To delete a manually added foreign key:

1. Go to the **Foreign Keys** tab.
2. Select the **Delete** icon next to foreign key you want to delete.




---

[Next Page](/llms-full.txt/1)