News

Ilan will present a poster entitled ‘Integrating Multi-layer Lexical Data in Language Model Training,’ which describes the unique characteristics of Lexicala’s multi-layer language data and how they can enhance the training and performance of LLMs and MT systems, as well as other NLP applications.

Our innovative multi-layer method converges human creation and curation with automated data processing. It starts with the meticulous exploration of the foundations of each language, and the evidence that is gathered and diagnosed enables “mapping its DNA” minutely to create a monolingual core. Then we proceed to add L2 equivalents, to produce a bilingual pair, and juxtapose more language translations forming a multilingual layer around the initial L1 core. This core and its multilingual satellites can be further cross-lingualized and extrapolated across other language networks, all relying on a comprehensive technical infrastructure and overall framework.

Besides the usual sense disambiguation, definitions, examples, and multiword expressions, the data features rich semantic and syntactic information including morphology lists linking inflected forms to main lemmas, offensive language taxonomies, domain and register classification, synonyms and antonyms, grammatical details, phonetic transcription, spelling variations, alternative scripts, cross-references, etc.

The most relevant for LLMs and MT are the usage examples that stem from traditional learner’s dictionaries. They are conceived by experts who identify typical language patterns and design short phrases and full sentences that illustrate their use for (foreign) language learners.

Incorporating such quality lexical resources in the initial stages of language model training exceeds feeding masses of data for LLM training, offering the advantages of efficiency, precision, compactness, savings (work, time, costs), and copyright issues.

Moreover, shifting the emphasis to the preliminary model training phases also moderates the needs for excessive (Automated) Post Editing and Quality Estimation and the challenges originating from LLM hallucination, bias, and inconsistency.

The poster will highlight the prime Danish language resource of Lexicala, which is based on data from the legacy Den Danske Ordbog (DDO, The Danish Dictionary), of the Society of Danish Language and Literature (DSL).

Over the years we have substantially revised and adapted the DDO entries to create the Danish monolingual layer in our Global series, and it has since served as a base for developing bilingual and multilingual layers, including a trilingual Danish-English-Korean dataset that we tailor-made in 2018-2020 for Naver corporation in Korea, with the help of Lexical Computing.

It’s been a while since my last visit to Copenhagen, well before the Covid-19 outbreak. I look forward to meeting our colleagues at CST and making new acquaintances!