Languages.jl is a Julia package for working with human languages. It provides:
Lists of words from each language for basic categories:
- Indefinite Articles
- Definite Articles
These methods are supported only for English and German currently.
This package also detects the script and language for written text in a wide variety of languages.
using Languages articles(Languages.English()) stopwords(Languages.English())
All word lists are returned as vectors of UTF-8 strings.
Script detection model works by checking the unicode character ranges present within the input text
Languages.detect_script("To be or not to be") # => Languages.LatinScript()
A trigram based model is used to detect the language for the text. The model is filtered based on the detected script.
We detect 84 of the most common languages spoken around the world. This usually covers most languages with more than 10 million native speakers.
detector = LanguageDetector() detector("To be or not to be") #=> (Languages.English(), Languages.LatinScript(), 1.0)
List All Supported Languages
You can use
list_languages() to get all supported languages.
LanguageDetector model returns the language, the script, and the confidence when applied to a string.
The API of this package has been refurbished recently. If you have used this package earlier, please be aware of these changes.
- The language names have been shortened. So
EnglishLanguage. However, the language names are no longer exported. So they should be referred to with the package name:
- Every language is a type. However all functions now accept and return instances of these types, rather than the types themselves.