polingual v0, research notes
Every word in every language, one meaning.
Abstract
polingual is a cross-lingual dictionary and an open research project. It holds 6.5 million words in 35 languages. Each word is a record with an English meaning, two vectors, and links to the same idea in other languages.
Speak two languages or more? Get early access to the learner tools.
1The record
We call each record a photon. A photon has one id, one surface form in its own script, and one meaning written in English. Each dictionary sense gets its own record. Translations stay separate records joined by links.
{
"id": "photon:word:en:gravity",
"kind": "word",
"lang": "en",
"surface": "gravity",
"meaning_en": "The attractive force between masses.",
"ipa": "/ˈɡɹævɪti/",
"pos": "noun",
"tier": "nucleus",
"relations": [
{ "predicate": "derives_from", "to": "photon:word:la:gravitas" },
{ "predicate": "translates", "to": "photon:word:de:schwerkraft" },
{ "predicate": "translates", "to": "photon:word:ja:重力" }
]
}2The math
Notation: is a word record, the set of all records, a query, and concatenation. Each word gets two unit vectors. The semantic vector embeds the text “surface: meaning” with LaBSE, a multilingual sentence model, into . The phonetic vector is built by hand from the IPA transcription into .
Two words mean the same thing when their semantic vectors point the same way. LaBSE was trained on translation pairs, and every record carries an English gloss, so light, אור, luce and valo sit close together with no translation step at query time.
The phonetic vector answers a separate question: which words sound alike, whatever they mean. Its 64 slots concatenate four blocks built from the IPA segments , where is the segment’s articulatory features (vowel height, backness, rounding; consonant place, manner, voicing):
| Slots | Block |
|---|---|
| 0 to 19 | average vowel height, backness and rounding; consonant place, manner and voicing |
| 32 to 45 | the first, middle and last segment, plus vowel and consonant ratios |
| 48 to 49 | word length and stress marks |
| 50 to 63 | sound bigrams hashed into 14 buckets |
Slots 6 to 7, 20 to 31 and 46 to 47 are reserved and stay zero.
Sound search finds loanwords and shared roots across scripts. Words close on both vectors are the cognates: gravitas, gravity, gravedad. A joint score mixes the two distances with a weight :
Lower is closer. is pure meaning search and is pure sound search. Choosing per query, and checking it against human judgment, is open work.
3The network
Records link to each other through typed edges. Following translates edges gives a translation table. Following derives_from edges backward gives an etymology: English to Latin to Greek, or Japanese back through Old Chinese readings.
| Predicate | Meaning | Example |
|---|---|---|
| translates | Same meaning in another language | gravity → Schwerkraft |
| derives_from | Etymological ancestor | gravity → gravitas |
| synonym_of | Same language, same meaning | entropy → disorder |
| attested_at | First appearance in a corpus | gravity → 1644 |
Every record carries a confidence tier: nucleus for settled entries, functional for sourced ones, edge for contested ones, and unverified for raw imports. The same graph powers bucket.foundation, which reads it as research claims. polingual reads it as words.
4Search at scale
The full index holds 6,564,942 records across 35 languages. 1,979,896 of them have IPA and a phonetic vector. Both vector columns live in Postgres with pgvector and an HNSW index, so a nearest-neighbor query walks a layered graph and visits about nodes instead of scanning all rows.
| Query | What it does | Latency |
|---|---|---|
| lookup | exact surface form in one language | 1 ms |
| semantic | nearest meaning vectors | 3 ms |
| phonetic | nearest sound vectors | 1.5 ms |
| translate | nearest meaning, filtered to a target language | 3 ms |
| spelling | trigram match on the surface form | ~160 ms |
Latencies are single warm queries against the full 6.5M-row index on one local Postgres server with pgvector, measured 2026-06-16 through the query API. The search box on the homepage covers a 209,000-word subset in 27 languages with text search: exact surface match, then prefix, then substring, then English full-text over the meanings, merged in that order until it has 30 results.
Semantic search works best for words with close neighbors: gold finds silver and golden, king finds monarch. Isolated scientific terms such as entropy get weak neighbors, and the dictionary has gaps.
5Sources
Every record keeps its source URL and capture date. Dictionary data comes from Wiktionary through Kaikki, under CC BY-SA 4.0.
| Source | Coverage | Status |
|---|---|---|
| Wiktionary via Kaikki | 35 languages, English glosses, IPA | live, CC BY-SA |
| Open Multilingual WordNet | ~30 languages | planned |
| PanLex | 1,300+ languages | planned |
| Tatoeba | 400+ languages, parallel sentences | planned |
6Open questions
- Polysemy
- bank the river and bank the institution are two records with two meanings. Today each Wiktionary sense becomes its own record. Picking the right sense from context comes later.
- Compounds
- "second law of thermodynamics" is stored as one phrase record and also as word records linked to it, so both phrase and word queries work.
- Re-embedding
- When the embedding model changes, every vector is rebuilt and ids stay fixed. Old vectors are kept so we can measure how far each word moved.
- Learner signal
- A speaker of two languages can say when two words line up and when they drift. We want that judgment as labeled data for the joint score.
7Join the learner list
We are building learning tools for people who already speak two or more languages: learn the next language through the ones you know. Leave your email, and tell us your languages if you like. We will write when the first tools open, and we may ask you to rate a few word pairs.