> For the complete documentation index, see [llms.txt](https://docs.syntho.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.syntho.ai/configure-a-data-generation-job/configure-column-settings/mask/text/automatic-pii-discovery-and-de-identification-in-free-text-columns.md).

# Text de-identification

**Text de-identification** detects sensitive information inside text columns and replaces the detected parts with mock data or entity labels. Use it for notes, comments and descriptions while keeping the surrounding text useful. You can configure built-in entities, create custom entities, extend detection rules and exclude known non-sensitive values.

{% hint style="warning" %}
Text de-identification can significantly increase generation time. Test the configuration on a representative subset before processing the full dataset.
{% endhint %}

## Configure a text column

1. Open the column settings for a supported text column.
2. Select **Mask** and then **Text de-identification**.
3. Select the **Locale** that matches the text.
4. Under **Detection model**, select an available model or **None**.
5. In **Entity types**, review the entities and configure their action, **Min confidence** and, for **Mock**, replacement generator.
6. Add custom entities or rules and configure exclusions as needed.
7. Click **Confirm** to save the column configuration. Preview or generate a representative sample and review the output.

Text de-identification is configured per column. The entity panel's bulk controls apply to entities within this configuration.

## Choose the locale and detection model

The locale determines the applicable detection rules and the model offered in the interface. Available locales include English, Dutch, German and Japanese; detection coverage differs by locale.

* **None** uses the available regex, lookup and built-in recognizers without an NLP model.
* **A selected model** adds model-based detection to that baseline. It can detect entities such as people, locations and organizations from context.

Selecting **None** does not disable regex or lookup detection, or checksum validation provided by built-in recognizers. Model-only entities are unavailable without a model.

Changing locale or model refreshes the applicable system entities and rules. Your custom rules remain saved; the rule list shows those applicable to the selected locale. Changing locale resets the model selection to **None**, so select the appropriate model again if needed.

## Choose an action for each entity

Use **Filter entities** to find an entity by name. Filtering only changes the visible list.

| Action       | Result                                                                 | What to configure                                                                                       |
| ------------ | ---------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- |
| **Mock**     | Replaces detected text with generated data.                            | Select and configure a valid replacement generator. A system entity suggests a default where available. |
| **Annotate** | Replaces detected text with its entity label, such as `<EMAIL>`.       | No replacement generator is needed.                                                                     |
| **Disable**  | Stops this entity from detecting and transforming text in this column. | No replacement generator is needed.                                                                     |

System entities default to **Annotate**. New custom entities default to **Disable**: select **Mock** or **Annotate** in every column where they should apply.

Disabling an entity does not exclude its text from every other entity. For example, another active recognizer may still detect part of the same value. Use the exclusion list when you need to exclude particular detections.

Switching away from **Mock** preserves its replacement settings for reuse when you switch back.

### Confidence

**Min confidence** is the minimum detection score accepted for an entity. It uses a whole-number percentage from 0 to 100 and defaults to **50%**. A detection must meet or exceed this threshold to be processed.

A custom regex rule starts with its configured confidence, defaulting to **100%**. This differs from the entity's minimum threshold. For example, a rule with a score of 80 passes a threshold of 50 but not 90.

Built-in recognizers can adjust scores using context, validation or partial-word matching. A high confidence score does not guarantee a correct detection. Review both missed sensitive values and false positives.

## Create a custom entity

Create an entity for business-specific information, such as a patient reference number:

1. Click **Add sensitive entity**.
2. Enter a unique **sensitive entity name**, for example `PATIENT_REFERENCE`.
3. Add one or more regular expressions. For references such as `PAT-123456`, use `\bPAT-\d{6}\b`.
4. Set the **Confidence score** for the new rules, or keep 100%.
5. Click **Save**.
6. Set the new entity to **Mock** or **Annotate**, and configure its minimum confidence and replacement generator as required.

Open **List rules** on an entity to add, edit or remove your regex rules. A custom entity requires a valid name and at least one valid rule. The interface prevents removing its last rule; remove the entity instead if it is no longer needed.

{% hint style="info" %}
Entity creation, rule edits and entity deletion update the workspace's entity definitions. Other columns using those entities are affected. Actions, minimum confidence, replacement settings and exclusions belong to the selected column and are saved with **Confirm**.
{% endhint %}

### Write precise regex rules

Rules match the text covered by the expression; that matched text is replaced. Use boundaries such as `\b` when a rule should match a complete identifier.

Rules are case-insensitive by default. For example, `PAT-\d{6}` also matches `pat-123456`. Use a scoped inline flag if case matters, for example `(?-i:PAT)-\d{6}`.

Custom rules match as written, at their configured confidence, without the checksum validation or partial-word score adjustments of shipped Syntho rules. Add any required format constraints to the expression.

For JSON-like or `key=value` text, match only the sensitive value when the surrounding keys, quotes and separators must remain intact. Check the generated structure as well as the replacement values.

### Regex rule examples

| Use case                  | Regular expression                                                                                                                   | Matches                                                                                                    | Does not match                                  |
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------- | ----------------------------------------------- |
| Patient reference         | `\bPAT-\d{6}\b`                                                                                                                      | `PAT-123456`                                                                                               | `PAT-12345`, `XPAT-123456`                      |
| Customer ID               | `\bCUST-[A-Z0-9]{8}\b`                                                                                                               | `CUST-A1B2C3D4`                                                                                            | `CUST-A1B2`, `CUST-A1B2C3D4X`                   |
| U.S. phone number         | (?\<!\w)(?:+1\[-. ]?)?(?:(\d{3})\|\d{3})\[-. ]?\d{3}\[-. ]?\d{4}(?!\w)                                                               | `+1 212-555-0199`, `(212) 555-0199`                                                                        | `212-555-019`, `212-555-01999`                  |
| Value in `key=value` text | `(?<=account_id=)[A-Z0-9-]+`                                                                                                         | `ACCT-8D21` in `account_id=ACCT-8D21`                                                                      | The `account_id=` key                           |
| Value in compact JSON     | `(?<="patient_id":")[^"]+(?=")`                                                                                                      | `P-1042` in `"patient_id":"P-1042"`                                                                        | The key and quotation marks                     |
| Number                    | `(?<!\w)[+-]?(?:[0-9]{1,3}(?:[ \u00A0\u202F][0-9]{3})+(?:[.,][0-9]+)?\|[0-9]+(?:[.,][0-9]+)*\|[.,][0-9]+)(?:[eE][+-]?[0-9]+)?(?!\w)` | `42`, `-42`, `+3.14`, `3,14`, `.5`, `1,234.56`, `1.234,56`, `1 234,56`, `6.02e23`                          | Numbers written as words; fractions such as `½` |
| Date or datetime          | `\b(?:\d{4}-\d{2}-\d{2}\|\d{2}([/-])\d{2}\1\d{4})(?:[ T]\d{2}:\d{2}(?::\d{2})?)?\b`                                                  | `2026-10-31`, `31-10-2026`, `31/10/2026`, `31/10/2026 14:30`, `2026-10-31 14:30:45`, `2026-10-31T14:30:45` | Single-digit days or months; two-digit years    |
| Dutch postal code         | `(?<!\w)[1-9][0-9]{3}\s*[A-Za-z]{2}(?!\w)`                                                                                           | `1234AB`, `1234 AB`, `1234 AB`, `1234 ab`                                                                  | Codes beginning with `0`                        |

## Extend a system entity

To detect an additional format as an existing entity:

1. Find the system entity and open **List rules**.
2. Add your regular expression.
3. Edit or remove your own additions through the same list.

Your additions extend the system entity in this workspace. Shipped rules remain visible and read-only, and the entity keeps its system name. Other workspaces retain their own configuration.

Your additions remain saved across locale or model changes. Shipped rules continue to follow Syntho updates; they are not frozen when you add your own rules.

To stop using an entity in one column, choose **Disable**. Removing an extended system entity removes the workspace extension and brings back the original system entity with its defaults; it also removes the extension's column settings and replacement links.

## Exclude non-sensitive values from detection

Use **Exclude from detection list** for known false positives, such as a non-sensitive product name.

1. Open the exclusion dropdown.
2. Add a regular expression.
3. Edit or remove entries individually as needed.
4. Choose the matching mode with **Exclude only exact matches**.
5. Click **Confirm** to save the column configuration.

Exclusions are case-insensitive in both modes and are checked against each detected text span, rather than the whole cell.

| Setting          | Behavior                                                                 | Example                                    |
| ---------------- | ------------------------------------------------------------------------ | ------------------------------------------ |
| **On** (default) | The expression must match the whole detected value.                      | `Eric` excludes `Eric`, but not `Ericson`. |
| **Off**          | A match anywhere inside a detected value excludes that entire detection. | `Eric` excludes both `Eric` and `Ericson`. |

Exact-match mode still accepts patterns. For example, `.*@corp\.example` excludes whole email detections at that domain. In partial-match mode, a short entry such as `ic` would also exclude `Epic`; use broad patterns carefully.

An exclusion suppresses only detections it matches. If an email detection is excluded but its domain is separately detected as a URL, that URL detection can still be transformed. Check overlapping detections in your sample output.

An excluded detection is neither annotated nor mocked. Empty or invalid expressions must be corrected before saving.

## Apply bulk actions and remove entities

Select entity checkboxes and use **Bulk edit** to set their action to **Mock**, **Annotate** or **Disable**. The header checkbox selects the currently filtered entities. Review each selected entity's replacement generator when switching to **Mock**; entities without a valid replacement configuration cannot be saved as Mock.

Use the trash icon for one custom entity or **Remove sensitive entities** for selected custom entities. Review the confirmation and any affected-column warning before proceeding. Deletion removes the entity's rules and references from affected column configurations. Unmodified system entities cannot be deleted.

The delivered interface manages detection rules and exclusions individually. Standalone rule-list file upload and export are not available in this flow. To transfer a complete configuration, see [Import and export workspaces](https://docs.syntho.ai/setup-workspaces/import-and-export-workspaces.md).

## Keep replacements consistent

For an entity using **Mock**, enable **Consistent mapping** on its replacement generator. The mapping uses the detected value, the replacement generator and its settings, and the workspace seed.

The same detected value receives the same replacement across entities, columns and tables when those settings match. A structured column can use the same replacement too. For example, an email inside a note and the same email in an `email` column can stay aligned when both use the same Email generator, settings and seed.

For separate workspaces or systems, use the same seed and matching generator settings. The exact input must also match: a detected full name and a separate first name are different inputs. Different generators can produce different outputs for the same text.

See [Consistent mapping](https://docs.syntho.ai/configure-a-data-generation-job/configure-column-settings/consistent-mapping.md) for seed configuration and deployment considerations.

{% hint style="warning" %}
The updated consistent text replacement behavior changes the values produced compared with Syntho 2026.2.1. When upgrading to this behavior, regenerate related text and structured datasets together if their replacements must stay aligned.
{% endhint %}

## Detection behavior and limitations

Regex, lookup dictionaries, built-in recognizers and an optional NLP model contribute to detection. Syntho applies confidence thresholds and resolves overlapping detections before applying the selected actions.

For identical spans, the higher-confidence entity takes precedence; when a detection contains another, the outer span takes precedence. Adjacent or overlapping matches of one entity can be combined. Do not assume each matching rule creates a separate replacement.

Rare-category protection, when enabled for this generator, runs before sensitive-text processing and retains its existing threshold and replacement behavior.

Detection coverage depends on the locale, model, rules and text domain. Review representative output before using it downstream. For the predefined catalog and custom-entity guidance, see [Supported sensitivity entities](https://docs.syntho.ai/configure-a-data-generation-job/manage-personally-identifiable-information-pii/supported-pii-entities.md).

## Use other NLP models

Syntho ships spaCy models for English (`en_core_web_md`), Dutch (`nl_core_news_md`), German (`de_core_news_md`) and Japanese (`ja_core_news_md`).

Additional compatible spaCy models require deployment where preview and generation jobs run. Syntho does not automatically download them. Other model types require alignment with Syntho and have limited support. Model choice affects detection quality and performance.

An administrator can configure a deployed spaCy model through the generator's `text_processor_model` parameter, for example:

```json
"text_processor_model": "en_core_web_trf"
```

The model must match the text language. A missing model causes preview or generation to fail when Syntho tries to load it.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.syntho.ai/configure-a-data-generation-job/configure-column-settings/mask/text/automatic-pii-discovery-and-de-identification-in-free-text-columns.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
