Languages

Languages#

Reference for the world’s spoken languages. Around 7,000 living languages are cataloged; only a few hundred have meaningful written or digital corpora, and roughly forty cover the lion’s share of speakers, internet content, and operator-relevant signal.

Quick lookups: ISO codes operators reach for, the top languages by speaker count, major language families, script systems, and the high-coverage NLP-supported set. For the exhaustive catalog, follow the Ethnologue / Glottolog references at the end.

Lookup conventions#

The standards every NLP toolchain, OS locale, and translation API reaches for. ISO 639-1 (two-letter) is the most common; 639-2 / 639-3 cover languages that don’t fit in 639-1. BCP 47 layers region, script, and variant on top:

Code	Example D	escription
ISO 639-1	`en` T	wo-letter language code; ~184 languages.
ISO 639-2/B	`eng` T	hree-letter; bibliographic. Library catalogs.
ISO 639-2/T	`eng` T	hree-letter; terminological. ISO standards.
ISO 639-3	`eng` T	hree-letter; ~7900 living and extinct languages.
ISO 639-5	`gem` L	anguage families and groups (e.g. Germanic).
BCP 47	`zh-Hant-TW`	Language + script + region; what browsers send.
ISO 15924	`Latn` S	cript code (Latin, Cyrillic, Hangul, …).
M49 region	`001` U	N region code; pairs with BCP 47.
Glottocode	`stan1293` G	lottolog identifier (more linguistic detail).

Top by speakers#

Approximate first-language plus total-speaker counts (2024-2026 sources: Ethnologue 27, CIA World Factbook). Numbers vary by source and methodology; treat them as orders of magnitude. L1 = first-language speakers; Total = L1 + L2.

Language	639-1	639-3	L1 / Total speakers (millions)
English	en	eng	380 / 1,500
Mandarin	zh	cmn	940 / 1,140
Hindi	hi	hin	345 / 610
Spanish	es	spa	485 / 560
Arabic (MSA)	ar	ara	310 / 380 (incl. all dialects)
French	fr	fra	80 / 320
Bengali	bn	ben	240 / 270
Portuguese	pt	por	230 / 265
Russian	ru	rus	150 / 255
Urdu	ur	urd	70 / 230
Indonesian	id	ind	45 / 200 (with Malay together)
German	de	deu	75 / 135
Japanese	ja	jpn	125 / 125
Marathi	mr	mar	85 / 100
Telugu	te	tel	80 / 95
Turkish	tr	tur	85 / 90
Tamil	ta	tam	75 / 85
Punjabi	pa	pan	115 / 125 (Eastern + Western)
Korean	ko	kor	80 / 82
Vietnamese	vi	vie	77 / 82
Persian/Farsi	fa	fas	70 / 110 (incl. Dari, Tajiki)
Italian	it	ita	65 / 67
Hausa	ha	hau	50 / 80
Swahili	sw	swa	18 / 200 (East African lingua franca)
Javanese	jv	jav	82 / 85
Thai	th	tha	21 / 60
Burmese	my	mya	33 / 43
Polish	pl	pol	40 / 45
Yoruba	yo	yor	47 / 50
Pashto	ps	pus	60 / 60

Language families#

Most living languages cluster into a small number of families. Each family shares a common ancestor; mutual intelligibility inside a branch is sometimes high (Spanish / Portuguese, Norwegian / Swedish), often low (Hindi / Bengali). Important for choosing language-ID models and translation pivot strategies:

Family	Notable members
Indo-European	English, Spanish, Russian, Hindi, Persian, Greek, Albanian, Armenian, all Romance, all Germanic, all Slavic, all Indo-Aryan, all Iranian.
Sino-Tibetan	Mandarin, Cantonese, Tibetan, Burmese, dozens of Tibeto-Burman.
Afro-Asiatic	Arabic (all dialects), Hebrew, Amharic, Tigrinya, Hausa, Berber, Somali.
Niger-Congo	Swahili, Yoruba, Igbo, Zulu, Shona, ~1,500 Bantu and West African languages.
Austronesian	Indonesian / Malay, Tagalog / Filipino, Javanese, Hawaiian, Maori, Malagasy.
Dravidian	Tamil, Telugu, Kannada, Malayalam.
Turkic	Turkish, Azeri, Uzbek, Kazakh, Uyghur, Kyrgyz.
Japonic	Japanese, Ryukyuan languages.
Koreanic	Korean (often classed as language isolate).
Tai-Kadai	Thai, Lao, Zhuang.
Austroasiatic	Vietnamese, Khmer, Mon.
Uralic	Finnish, Estonian, Hungarian, Sami.
Mongolic	Mongolian, Buryat, Kalmyk.
Quechuan	Quechua varieties (Andes).
Mayan	Yucatec, K’iche’, Q’eqchi’, Mam.
Eskimo-Aleut	Inuktitut, Yup’ik, Aleut.
Na-Dene	Navajo, Apache, Tlingit.
Nilo-Saharan	Luo, Maasai, Dinka, Nubian.
Khoisan (areal)	Khoekhoe, ǃXóõ (click languages).
Kartvelian	Georgian, Mingrelian, Svan.
Northeast Caucasian	Chechen, Avar, Lezgian.
Northwest Caucasian	Abkhaz, Circassian.
Language isolates	Basque (eu), Korean (debated), Ainu, Burushaski.
Constructed	Esperanto, Klingon, Toki Pona.

Scripts (writing)#

The script a language uses determines tokenization, OCR engine selection, font fallbacks, and whether mixed-script attacks (homoglyph spoofing) are a real risk:

ISO 15924	Script	Languages
Latn	Latin	English, Spanish, French, Vietnamese, Indonesian, Turkish, Swahili, most African and European.
Cyrl	Cyrillic	Russian, Ukrainian, Belarusian, Bulgarian, Serbian, Macedonian, Kazakh, Kyrgyz, Tajik, Mongolian.
Arab	Arabic (Nasta’liq)	Arabic, Persian, Urdu, Pashto, Uyghur, Kurdish.
Deva	Devanagari	Hindi, Marathi, Nepali, Sanskrit.
Hans / Hant	Han (Simplified / Tra	ditional)Mandarin (Simplified PRC, Traditional TW/HK), used in Japanese (kanji), Korean (hanja, rare).
Hira / Kana	Hiragana / Katakana	Japanese (mixed with kanji).
Hang	Hangul	Korean.
Beng	Bengali	Bengali, Assamese.
Taml	Tamil	Tamil.
Telu	Telugu	Telugu.
Knda	Kannada	Kannada.
Mlym	Malayalam	Malayalam.
Gujr	Gujarati	Gujarati.
Guru	Gurmukhi	Punjabi (Eastern).
Thai	Thai	Thai.
Laoo	Lao	Lao.
Mymr	Myanmar	Burmese, Shan.
Khmr	Khmer	Khmer (Cambodian).
Hebr	Hebrew	Hebrew, Yiddish.
Geor	Georgian	Georgian.
Armn	Armenian	Armenian.
Ethi	Ethiopic	Amharic, Tigrinya.
Sinh	Sinhala	Sinhala.
Tibt	Tibetan	Tibetan, Dzongkha.
Mong	Mongolian	Traditional Mongolian.
Latn-IPA	IPA	International Phonetic Alphabet (notation).

NLP coverage#

A practical guide to which languages have production-grade support in 2026 NLP toolchains. Tier-1 languages have everything; tier-3 means the analyst is doing more work:

Tier	Languages
Tier 1 (full coverage)	English, Mandarin, Spanish, French, German, Russian, Portuguese, Italian, Dutch, Japanese, Korean, Arabic (MSA), Hindi, Indonesian, Vietnamese, Polish. ► spaCy / Stanza / NLLB / Whisper + every LLM.
Tier 2 (good coverage)	Turkish, Greek, Hebrew, Czech, Romanian, Ukrainian, Hungarian, Finnish, Swedish, Norwegian, Danish, Thai, Bengali, Tamil, Telugu, Persian, Urdu, Swahili. ► spaCy or Stanza, NLLB-200, Whisper good, fewer fine-tuned models.
Tier 3 (partial coverage)	Most Niger-Congo, Dravidian outside the big four, Tibeto-Burman, Caucasian, regional Arabic dialects, Pashto, Kurdish, Sinhala, Khmer, Lao, Mongolian. ► language-ID works; ASR is patchy; translation usable with NLLB-200 or LLM zero-shot.
Tier 4 (low resource)	Most of the ~7000 living languages, pidgins, creoles, indigenous languages, sign languages. ► research-grade only; field linguists’ tooling.

Family tree (overview)#

A schematic of how the major language families connect. The tree is reconstructed by historical linguists from systematic sound correspondences, shared inherited vocabulary, and grammatical agreement; debates over deep groupings (Nostratic, Eurasiatic, Dene-Yeniseian) remain active.

        flowchart LR
  ROOT[Language Families]

  ROOT --> IE[Indo-European]
  IE --> II[Indo-Iranian]
  II --> Indic["Indic<br/>Hindi, Bengali, Punjabi,<br/>Marathi, Gujarati, Sinhala, Romani"]
  II --> Iranian["Iranian<br/>Persian, Pashto, Kurdish,<br/>Tajik, Ossetic"]
  IE --> Slavic["Slavic<br/>Russian, Polish, Czech,<br/>Ukrainian, Serbian, Bulgarian"]
  IE --> Germanic["Germanic<br/>English, German, Dutch,<br/>Swedish, Norwegian, Danish, Yiddish"]
  IE --> Romance["Romance<br/>Spanish, French, Italian, Portuguese,<br/>Romanian, Catalan, Galician, Occitan"]
  IE --> Celtic["Celtic<br/>Irish, Welsh, Scottish Gaelic, Breton"]
  IE --> Hellenic["Hellenic (Greek)"]
  IE --> IEother["Albanian, Armenian,<br/>Baltic (Lithuanian, Latvian)"]
  IE --> IEext["<i>extinct: Anatolian (incl. Hittite),<br/>Tocharian</i>"]

  ROOT --> ST[Sino-Tibetan]
  ST --> Sinitic["Sinitic<br/>Mandarin, Cantonese, Wu, Min, Hakka, Xiang"]
  ST --> TB["Tibeto-Burman<br/>Tibetan, Burmese, Karen, Newar"]

  ROOT --> AA[Afro-Asiatic]
  AA --> Semitic["Semitic<br/>Arabic, Hebrew, Amharic,<br/>Tigrinya, Aramaic, Maltese"]
  AA --> Berber["Berber (Tamazight)"]
  AA --> Cushitic["Cushitic<br/>Somali, Oromo, Afar"]
  AA --> Chadic["Chadic (Hausa)"]
  AA --> AAext["<i>extinct: Egyptian, Akkadian</i>"]

  ROOT --> NC[Niger-Congo]
  NC --> Bantu["Bantu<br/>Swahili, Zulu, Shona,<br/>Kongo, Lingala, Kinyarwanda"]
  NC --> Atlantic["Atlantic (Wolof, Fula)"]
  NC --> Mande["Mande (Bambara, Mandinka)"]
  NC --> VN["Volta-Niger (Yoruba, Igbo)"]
  NC --> NCmore["<i>many more sub-branches</i>"]

  ROOT --> AN[Austronesian]
  AN --> MP["Malayo-Polynesian<br/>Indonesian, Malay, Tagalog, Javanese,<br/>Hawaiian, Maori, Malagasy, Fijian"]
  AN --> Formosan["Formosan (Taiwan)"]

  ROOT --> Dravidian["Dravidian<br/>Tamil, Telugu, Kannada,<br/>Malayalam, Brahui"]
  ROOT --> Turkic["Turkic<br/>Turkish, Azeri, Uzbek, Kazakh,<br/>Kyrgyz, Uyghur, Turkmen, Yakut"]

  ROOT --> Uralic[Uralic]
  Uralic --> FP["Finno-Permic<br/>Finnish, Estonian, Sami, Komi"]
  Uralic --> Ugric["Ugric (Hungarian, Khanty, Mansi)"]

  ROOT --> Mongolic["Mongolic (Mongolian, Buryat, Kalmyk)"]
  ROOT --> Tungusic["Tungusic (Manchu, Evenki)"]
  ROOT --> Japonic["Japonic (Japanese, Ryukyuan)"]
  ROOT --> Koreanic["Koreanic (Korean)"]
  ROOT --> TK["Tai-Kadai (Thai, Lao, Zhuang, Shan)"]
  ROOT --> Aus["Austroasiatic<br/>Vietnamese, Khmer, Mon, Khasi, Munda"]
  ROOT --> HM["Hmong-Mien (Hmong, Yao)"]

  ROOT --> Andean["Quechuan, Aymaran (Andean)"]
  ROOT --> Meso["Mayan, Oto-Manguean (Mesoamerica)"]
  ROOT --> Tupian["Tupian (Guarani, Tupi)"]
  ROOT --> SAm["Arawakan, Cariban, Macro-Jê"]
  ROOT --> EA["Eskimo-Aleut (Inuit, Yup'ik, Aleut)"]
  ROOT --> ND["Na-Dene (Navajo, Apache, Tlingit)"]
  ROOT --> Algic["Algic (Cree, Ojibwe, Blackfoot)"]
  ROOT --> NAm["Iroquoian, Siouan, Uto-Aztecan"]

  ROOT --> NS["Nilo-Saharan<br/>Luo, Maasai, Dinka, Nubian, Songhai"]
  ROOT --> Khoisan["Khoisan (areal: Khoekhoe, ǃXóõ)"]
  ROOT --> Kart["Kartvelian (Georgian, Mingrelian)"]
  ROOT --> NEC["Northeast Caucasian (Chechen, Avar)"]
  ROOT --> NWC["Northwest Caucasian (Abkhaz, Circassian)"]
  ROOT --> PN["Pama-Nyungan (Australian, ~300)"]
  ROOT --> TNG["Trans-New Guinea (~470 languages)"]

  ROOT --> ISO["Language isolates<br/>Basque, Korean (debated), Ainu,<br/>Burushaski, Sumerian (extinct),<br/>Etruscan (extinct), Hadza, Sandawe"]

Disputed deep groupings: Nostratic, Eurasiatic, Altaic (mostly rejected), Dene-Yeniseian, Indo-Pacific.

Evolution and dating#

Quick reference for the chronology of the major families and attested ancestral languages. Dates are conservative consensus ranges; deeper proto-stages remain debated.

Stage / proto-language	Approximate dating
Proto-Indo-European	~4500-2500 BCE (steppe homeland hypothesis dominant).
Proto-Sino-Tibetan	~4000-2000 BCE.
Proto-Afro-Asiatic	~10000-8000 BCE (deep, debated).
Proto-Semitic	~4000-3000 BCE.
Proto-Bantu	~3000 BCE; expansion through sub-Saharan Africa from ~1500 BCE onward.
Proto-Austronesian	~3500 BCE (Taiwan); Malayo-Polynesian expansion from ~2000 BCE.
Proto-Uralic	~4000-2500 BCE.
Proto-Turkic	~500 BCE - 500 CE.
Proto-Dravidian	~3500-3000 BCE.
Proto-Japonic	~500 BCE - 300 CE.
Proto-Quechuan	~500-1000 CE.
Proto-Algonquian	~1000 BCE.
Old Egyptian	attested ~3000 BCE; modern descendant Coptic.
Akkadian	attested ~2500 BCE; the earliest Semitic.
Sumerian	attested ~3100 BCE; isolate; first writing system.
Hittite	attested ~1700 BCE; earliest written Indo-European.
Sanskrit (Vedic)	attested ~1500 BCE.
Mycenaean Greek	attested ~1400 BCE.
Old Chinese	attested ~1200 BCE (oracle bones).
Latin	attested ~7th c. BCE.
Old English	~5th-11th c. CE.
Middle English	~11th-15th c.
Modern English	from ~1500 CE; standardization post-1700.

Sociolinguistic processes#

Language change isn’t only branching. Six processes shape the modern map:

Sound change, regular phonetic shifts (Grimm’s law, the Great Vowel Shift); the comparative method depends on their regularity.
Borrowing, contact-induced lexical (and sometimes grammatical) transfer. English’s heavy Latin/Norman-French layer; Japanese kanji from Chinese; Persian borrowings into Hindi-Urdu and Turkish.
Pidginisation, a contact language with simplified grammar emerges from speakers needing a lingua franca.
Creolisation, a pidgin acquires native speakers and develops full grammatical complexity (Haitian Creole, Tok Pisin, Bislama, Cape Verdean, AAVE-as-creole-debate).
Koineisation, regional dialects converge into a new shared variety (Israeli Hebrew, modern Standard Arabic in some registers, Malaysian / Indonesian).
Language shift / death, speakers abandon a heritage language for a dominant one; UNESCO classifies ~3,000 languages as endangered.

Major derivative families an operator may meet:

English creoles, Tok Pisin (PNG), Bislama (Vanuatu), Pijin (Solomon Is.), Krio (Sierra Leone), Pidgin English (Cameroon, Nigeria), Jamaican Patois.
French creoles, Haitian, Mauritian, Seychellois, Réunion, Louisiana.
Portuguese creoles, Cape Verdean, Guinea-Bissau, São Tomé, Macau (Patuá).
Arabic-derived, Maltese (the only Semitic language in the EU written in Latin script).
Romani, Indo-Aryan diaspora language across Europe; many regional varieties.

Operator notes#

Locale strings, en_US.UTF-8, zh_CN.UTF-8, ar_EG.UTF-8; language + region + encoding. Set LC_ALL for predictable behavior across child processes.
Right-to-left, Arabic, Hebrew, Persian, Urdu. UI layout must mirror; logging text in mixed-direction strings is a classic source of confusion.
CJK (Chinese / Japanese / Korean), no spaces between words; tokenization needs language-specific segmenters (jieba for Chinese, MeCab / Sudachi for Japanese, KoNLPy / mecab-ko for Korean).
Diglossia, Arabic written form (MSA) differs sharply from spoken dialects (Egyptian, Levantine, Maghrebi, Gulf). Most NLP is tuned for MSA and degrades on dialect.
Romanization, pinyin (Mandarin), romaji (Japanese), Library of Congress (Russian / Arabic). Useful for selectors when the source script isn’t available.
Endangered languages, Ethnologue marks ~40% of living languages as threatened; corpus availability shrinks rapidly.

References#

Ethnologue, the standard catalog of the world’s languages (subscription).
Glottolog, open linguistic catalog with ~7900 languages and family trees.
ISO 639-3, the official three-letter language codes.
BCP 47, the IETF language tag specification.
Unicode CLDR, locale data (number formats, calendars, language names).
Universal Dependencies, treebanks for ~150 languages.
Hugging Face, language filter, model coverage by language code.
NLP, the NLP toolchain that consumes language-tagged corpora.