> For the complete documentation index, see [llms.txt](https://docs.syntho.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.syntho.ai/configure-a-data-generation-job/configure-column-settings/duplicate/automatic-pii-discovery-and-de-identification-in-free-text-columns.md).

# Free text PII obfuscation

{% hint style="warning" %}
**Caution**: this feature will slow down your data generation jobs significantly. Consider using a GPU or reducing the number of input records to speed up your job.
{% endhint %}

For more information about the PII entities that Syntho supports, see [Supported PII entities](/configure-a-data-generation-job/manage-personally-identifiable-information-pii/supported-pii-entities.md).

## Use free text PII obfuscation

Use **Free text PII obfuscation** as a **Mask** generator for text columns.

To apply it:

1. Under **Column settings** > **Generator**, select **Mask**.
2. Under **Mask generator**, select **Free text PII obfuscation**.
3. Select the correct **Locale** for the language in your text column.
4. Optionally, enable **Replace PII with mock data**.
   * Enabled: detected PII is replaced with mock values.
   * Disabled: detected PII is annotated with a PII label.

When you apply the PII text scanner to specific columns, Syntho automatically scans for PII elements in those columns. Identified PII elements can then be replaced with mock data. Syntho employs a variety of algorithms and methods to improve the scanning process.

Here's an overview of the steps taken in the detection process, in chronological order:

1. **Regex**: for pattern recognition.
2. **Named Entity Recognition** **(NER)**: to recognize natural language PII entities.
3. **Checksums**: to validate detected patterns.
4. **Context words**: to increase detection certainty.
5. **Label**: to label detected PII entity with a descriptor of the entity.
6. **(Optional) Obfuscate**: to replace detected PII descriptors with mock data.

Under **Encoding type > Locale**, you can define the locale used by the text processing models for text columns containing PII.

Syntho supports detection and de-identification of PII fields for the languages **English** and **Dutch** in columns containing free text data.

Syntho allows adding **NLP (natural language processing)** models with limited support for different languages (see next section).

## Configure to use other NLP models (limited support)

{% hint style="info" %}
**Note**: using non-default NLP models requires having an active internet connection to retrieve those models.
{% endhint %}

Syntho uses NLP engines for two main tasks: NER-based PII identification, and feature extraction for custom rule based logic (such as leveraging context words for improved detection).

By default, with each deployment, Syntho ships the following open-source models from spaCy:

* `en_core_web_md` for English.
* `nl_core_news_md` for Dutch.
* `de_core_news_md` for German.

These models can be replaced by leveraging other NLP models, either public or proprietary. As its internal NLP engine, Syntho supports both [spaCy](https://spacy.io/usage/models) and [Stanza](https://github.com/stanfordnlp/stanza).

This feature can be enabled via the workspace default settings. Hold **CTRL + SHIFT + ALT + 0** to open the **Workspace Default Settings** and enable the model by setting the **model\_name** to any model name as defined in [spaCy](https://spacy.io/usage/models) or [Stanza](https://github.com/stanfordnlp/stanza). For example, to use the English transformer spaCy model:

```
"text_processor_model_settings": {
    "models": [
        {
            "lang_code": "en",
            "model_name": "en_core_web_trf"
        }, ...
    ],
    "nlp_engine_name": "spacy",
     "gpu": false
    }
```

Optionally, if you have configured a GPU in your deployment setup, the `"gpu"` parameter can be set to `true` for faster results.

### Other model requests

Other NLP models, such as [transformer models](https://github.com/huggingface/transformers), can be added on request with limited support from Syntho. It is important to remember that using other models will impact the PII detection flow and its performance.

* **PII Detection and Confidence Score:** The PII text scanner may identify multiple potential **Personally Identifiable Information (PII)** entities within a text column. When this occurs, the entity with the highest confidence score is presented to the user. However, it's important to understand that a high confidence score doesn't guarantee accuracy. This could result in mislabeling the type of PII detected.
* **Internet Requirement for Non-Default NLP Models:** If you opt to use specialized **Natural Language Processing (NLP)** models to accommodate different languages or regions, an active internet connection is necessary to download these models.
* **Detection Methods:** The scanner employs a multi-method approach for PII detection, including the use of **Regex** patterns, **Named Entity Recognition (NER)** models, checksum validation, and examination of context words. Note that the effectiveness of the NER models can vary in different context it's being used. For instance, a NER model trained on Wikipedia text may not perform well when applied to medical data.

By understanding these details, you can better navigate how free text PII obfuscation works and what its limitations may be.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.syntho.ai/configure-a-data-generation-job/configure-column-settings/duplicate/automatic-pii-discovery-and-de-identification-in-free-text-columns.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.