# Arabic search that finds أحمد when you type احمد

People type Arabic inconsistently. أحمد comes in as احمد, مدرسة as مدرسه, numbers as ٢٥ or 25, sometimes with diacritics. A plain text search treats each spelling as a different word.

My Android reminders app handles it with one rule from its `arabic-rtl` skill: the stored text is never changed, a normalized copy is used only for matching, and exactly one function produces it. That function is called in two places: when a message is written to the index, and when a query is built. A third copy, or an inline variant, is a bug.

## The normalizer

From the app's `ArabicNormalizer.kt`:

```kotlin
object ArabicNormalizer {
    fun normalize(raw: String): String = buildString(raw.length) {
        for (c in raw) {
            when (val cp = c.code) {
                in 0x0610..0x061A -> {}                              // Quranic/Arabic signs: strip
                in 0x064B..0x065F -> {}                              // harakat (tashkeel): strip
                0x0670 -> {}                                         // superscript alef: strip
                0x0640 -> {}                                         // tatweel (kashida): strip
                0x0623, 0x0625, 0x0622 -> append(0x0627.toChar())    // أ/إ/آ → ا
                0x0629 -> append(0x0647.toChar())                    // ة → ه
                0x0649 -> append(0x064A.toChar())                    // ى → ي
                in 0x0660..0x0669 -> append('0' + (cp - 0x0660))     // ٠–٩ → 0–9
                in 0x06F0..0x06F9 -> append('0' + (cp - 0x06F0))     // ۰–۹ (Persian/Urdu) → 0–9
                else -> append(c)
            }
        }
    }.lowercase()   // for the Latin side of mixed text
}
```

The logic uses code points, not Arabic literals, so it can't change meaning if an editor mangles the file's encoding. Only the comments carry Arabic. The skill stops at these foldings: anything more, like ء variants or ک/ك, counts as a spec change, and the agent has to ask first.

## The query builder

From `SearchQuery.kt`:

```kotlin
object SearchQuery {
    private val separators = Regex("[^\\p{L}\\p{N}]+")

    fun build(raw: String): String? {
        val tokens = ArabicNormalizer.normalize(raw)
            .split(separators)
            .filter { it.isNotEmpty() }
        if (tokens.isEmpty()) return null
        return tokens.joinToString(" ") { "\"$it\"*" }
    }
}
```

Splitting on anything that isn't a letter or digit drops the characters FTS reads as operators (`" * ( ) : -`), so user punctuation can't cause a syntax error. Each token is quoted and gets a `*`, which makes it find-as-you-type. Blank or punctuation-only input returns `null`, and the app skips the query. The code's comment says an empty MATCH would be an error. That's true of FTS5, and the app ended up on FTS4, where it just returns nothing.

The six tests in `SearchQueryTest` cover the builder: `"foo* -bar:"` becomes `"foo"* "bar"*`, `٢٥` becomes `"25"*`, `*:-()"` gives `null`. All six passed on the run of 2026-07-06. They test the builder and that it goes through the normalizer, not a search against the real index on a device.

## Room

The index is an external-content FTS4 table over the normalized column:

```kotlin
@Fts4(contentEntity = MessageEntity::class, tokenizer = FtsOptions.TOKENIZER_UNICODE61)
@Entity(tableName = "message_fts")
data class MessageFts(val normalized_text: String?)
```

The app's spec asked for FTS5. Room has [an annotation for FTS4](https://developer.android.com/reference/androidx/room/Fts4) and generates the sync triggers itself; FTS5 would mean hand-written DDL, triggers and raw queries. The skill records the call: for a personal reminders app the result is the same, and moving to FTS5 later is a data rebuild, not a schema trap. `unicode61` only splits words, because the normalizer has already done the Arabic part on both sides.

## For your own skill

These rules combine the skill's search section with what the code does, so an agent working on the app keeps them intact:

````markdown
## Arabic search: one normalizer, two call sites

Raw text is never changed. Normalization produces a derived copy used only for matching.

- Exactly ONE normalization function. It is called in exactly two places: writing the
  normalized column on create/edit, and building the FTS query. A third copy or an inline
  variant is a bug.
- Foldings: strip tashkeel, superscript alef and tatweel; أ/إ/آ → ا; ة → ه; ى → ي;
  Arabic-Indic and Persian digits → ASCII; lowercase for Latin. Anything beyond that
  (ء variants, ک/ك) is a spec change: ask, don't slip it in.
- Display always renders the raw text.
- Query: normalize, split on non-letter/non-digit runs (this removes FTS operator
  characters), quote each token and add `*` for prefix search. Blank or punctuation-only
  input runs no query.
- Accept Arabic-Indic and Latin digits in every numeric input. Parse liberally, store
  canonically; the normalizer's digit fold doubles as input scrubbing.
````

see also: [Rtl Arabic Web Skill](/articles/rtl-arabic-web-skill), [Kotlin For React Devs Skill](/articles/kotlin-for-react-devs-skill)
