> For the complete documentation index, see [llms.txt](https://docs.syntho.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.syntho.ai/configure-a-data-generation-job/configure-column-settings/consistent-mapping.md).

# Consistent mapping

**Consistent mapping** allows you to generate the same mock data values for a given set of original data values every time the mock generator is applied. This feature can extend across various database types, allowing for consistent results. It is especially helpful when you need to consistently generate the same mock values based on the same input values, e.g. for testing or demonstration purposes.

## **Applicable generators**

Consistent mapping can be applied to the following generators:

* [**Mock**](/configure-a-data-generation-job/configure-column-settings/mockers.md)
* [**Mask**](/configure-a-data-generation-job/configure-column-settings/mask.md)
* The key generator Hash.

## Enable consistent mapping

To enable consistent mapping, open the column configurations via the Table view or Main Hub.

After opening column settings window, click on the box next to the “**Consistent Mapping**” to enable consistent mapping for a generator. Keep in mind that you have to select “**Mock generator**”, "**Mask**" or "**Hash**" as “**Generation Type**”.

<figure><img src="https://1383248054-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FU61B9DqtWCNO3Z30vnjh%2Fuploads%2FsoWqiqbaYTZCkGnIWoS8%2Fimage.png?alt=media&amp;token=a25aec19-7535-44e7-861b-74518183af32" alt=""><figcaption></figcaption></figure>

## Choose the mapping input key

The **Consistent with** dropdown value determines how rows are grouped for consistent mapping.

Rows with the same value receive the same generated output. Choose one of these key sources:

* **Current column**: Uses the value in the column being generated. This is the default behaviour.
* **Another column**: Uses a column from the same table. Use this to preserve relationships between columns.
* **Custom formula**: Uses the result of a formula as the key. Use this to normalize or derive values before mapping.

For each consistent with source, choose whether it uses source values or generated values via the **Apply on generated data** toggle.

### Use another column

Select **Another column** when the generated value should remain consistent for a shared attribute.

For example, configure an `ip_address` mock generator to use `username` as its mapping input key. Every row for `User1` then receives the same generated IP address. Rows for `User2` receive a different one.

You can also preserve customer-level relationships when adding noise. Configure `order_total` with `numeric_noise`, then select `customer_id` as the mapping input key. Every order for one customer receives the same numeric shift. For example, one `customer_id` may receive `+12.45`, while another receives `-8.20`.

Apply the same setup to a datetime column. Configure `order_date` with `datetime_noise`, then select `customer_id` as the mapping input key. Every date for one customer shifts by the same duration. For example, one customer's dates may each shift by `+17 days`, while another customer's dates shift by `-4 days`.

You can select columns from the current table only.

### Use a custom formula

Select **Custom formula** when you need to transform a value before using it as the mapping key.

The formula editor supports the same functions and column references as calculated columns. Formulas define the key only. They do not generate the output value.

For example, values may include a system-specific prefix:

```
HR_company_1
DCR_company_1
OVH_company_1
```

Use a formula that removes the prefix. The resulting key is `company_1` for each value. These rows then receive the same generated value.

## Advantages of consistent mapping

**Linking data**: Even if your database doesn't force certain rules on how data is connected (like making sure email addresses match up), consistency lets you match things together anyway. For example, you can mock last names to keep them private but still link related columns.

**Preserving distributions**: If you have a list of different items, like 20 job titles in a column, and you want to mix them up without losing the overall distributions (still around 20 job titles), consistency is your friend. It keeps the distributions about the same.

However, remember that while consistency keeps the variety, it might not keep every item unique. The variety won't grow, but it could change slightly. If you need each item to match one-to-one to a new unique value, you will have to make the **mock generator unique**.

**Matching data across systems**: If you're dealing with data spread out over different databases, like names in one place and email addresses in another, consistency helps you keep the names the same everywhere, even after making them private. This way, everything is still matched up correctly without sharing private information.

## Consistent mapping example

Assume we have a table where the first name “Mavis612” appears twice. If you enable consistent mapping, both of these will be mapped to the same name, consistently, across the tables. See the illustrations below from MySQL tables where mock generators with consistent mapping map the name “Mavis612” to “Jillian”. Please note that other names can also be mapped to Jillian. **Consistent mapping** does not suggest that only and only “Mavis612” will be replaced with “Jillian”.

<figure><img src="https://content.gitbook.com/content/U61B9DqtWCNO3Z30vnjh/blobs/oRXafKKOh3RXlcUV4LQy/pic2.png" alt=""><figcaption><p>An example of consistent mapping</p></figcaption></figure>

## Understanding consistent mapping

**Consistent mapping equals predictability**: If you feed the same data into a mock generator with **consistent mapping** enabled, you'll always get the same result. It's like using the same recipe every time to bake a cake; the outcome is predictable.

**Uniqueness not guaranteed**: Just because the process is consistent doesn't mean every different piece of data will come out differently. Two distinct inputs might lead to the same output. Think of it as different ingredients sometimes making a cake taste similar.

### Privacy implications

**No mappings stored**: The Syntho platform uses a [seed algorithm](https://pypi.org/project/pyfarmhash/), which means it doesn't store any information about the transformation of input to output values, which is more secure. So, you can see "Karl" appears 10 times, but not that "Karl" was originally "Immanuel." Also, you may change how input values are mapped to output values by changing the seed value in the Workspace Settings menu.

**Reduced privacy**: Using **consistent mapping** might reveal some information, like how often certain data appears. For example, if "Karl" shows up 10 times, that pattern stays the same after the data is processed.

### How consistency works across databases

**Whole database application**: Consistency applies across your entire database, not just within a single table. If you use **consistent mapping** for names in both a **Customers** and an **Employees** table, the same original name will always be updated to the same new name across both tables.

**Consistency across jobs**: By default, if you generate data more than once, consistency is automatically carried over from one generation job to the next. In case you don't want this, and you want to change consistency over jobs, you can set a different seed value. You can do this by going to the [Workspace settings](/setup-workspaces/workspace-default-settings.md) and update the `seed_value` to another integer value.

In short, consistency in data generation helps make sure your data behaves predictably, while also addressing the balance between maintaining useful patterns and protecting privacy.

### Seed

The consistent mapping scheme is based on a defined **seed** value. For example, if `seed=1` and consistency mapping is enabled for an Address generator, the same input address (e.g., "123 Main Street") will consistently map to the same output (e.g., "456 Elm Avenue"). If, after a period of using this consistent mapping scheme, the user decides to switch to a new seed (e.g., `seed=2`) for security reasons, the mapping will change. For instance, "123 Main Street" might then map to "789 Oak Drive" instead.

The seed value for a workspace can be changed under the [Workspace settings](/setup-workspaces/workspace-default-settings.md).

### **Ordering and indexing considerations**

To ensure accurate ordering, it is essential for the application to have either an index or a primary key in the source table. In the absence of these, the application defaults to sorting based on the first column of the table. However, if the first column contains duplicate values, the ordering cannot be guaranteed, as it relies on the database's sorting algorithm to handle duplicate values. Adding an index to the source table will resolve this issue.

#### **Column set for "ORDER BY" clause**

{% hint style="info" %}
Hive only
{% endhint %}

In the Table Settings panel, a new dropdown field allows users to specify which columns should be used in the "ORDER BY" clause. This feature enables users to define a set of columns that ensure the uniqueness of the returned results for a given table. By selecting the appropriate columns, users can achieve deterministic ordering even in the absence of primary keys or indexes.

* **Order By Dropdown**: Located in the Table Settings panel on the right side of the Table/Job Configuration screen, this dropdown lets users choose the columns for the "ORDER BY" clause.

## Limitations

* **Consistent mapping** does not guarantee consistency for generated data under different Syntho platform versions, or when deployed on different OSes (e.g., Windows vs Linux), or CPU architectures (e.g., x86 vs ARM).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.syntho.ai/configure-a-data-generation-job/configure-column-settings/consistent-mapping.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.