---
url: >-
  https://adk.nht.io/api/@nhtio/adk/batteries/context/thrift/relevance/functions/contentTokens.md
---

# Function: contentTokens()

```ts
function contentTokens(text: string): Set<string>;
```

Defined in: [src/batteries/context/thrift/relevance.ts:102](https://github.com/NHTIO/ADK/blob/v1.20260713.1/src/src/batteries/context/thrift/relevance.ts#L102)

Extract the "content word" tokens of a string for lexical overlap scoring: lowercased alphanumeric
runs of length >= 4, deduplicated into a set. Deliberately crude (no stemming, no stopword list
beyond the length-4 floor) — this is a cheap, zero-model-call, zero-dependency relevance signal, not
a semantic one; the calibration in [RELEVANCE\_FLOOR\_MIN](../variables/RELEVANCE_FLOOR_MIN.md)/[RELEVANCE\_FLOOR\_MAX](../variables/RELEVANCE_FLOOR_MAX.md) was run
against exactly this scoring function, so changing it invalidates those constants.

## Parameters

| Parameter | Type     | Description                               |
| --------- | -------- | ----------------------------------------- |
| `text`    | `string` | Text to extract content-word tokens from. |

## Returns

[`Set`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set)<`string`>

The distinct content words found, lowercased.