{"id":2854,"date":"2020-01-17T17:27:51","date_gmt":"2020-01-17T17:27:51","guid":{"rendered":"https:\/\/www.translatorswithoutborders.org\/blog\/?p=2854"},"modified":"2024-08-22T14:28:32","modified_gmt":"2024-08-22T14:28:32","slug":"transfer-learning-approaches-machine-translation","status":"publish","type":"post","link":"https:\/\/translatorswithoutborders.org\/blog\/transfer-learning-approaches-machine-translation\/","title":{"rendered":"Transfer Learning Approaches for Machine Translation"},"content":{"rendered":"
\n
\n
\n

This article was originally posted in the TWB Tech Blog on medium.com<\/a><\/em><\/p>\n

TWB\u2019s current research focuses on bringing language technology to marginalized communities<\/strong><\/h1>\n

Translators without Borders (TWB) aims to empower people through access to critical information and two-way communication in their own language. We believe language technology such as machine translation systems are essential to achieving this. This is a challenging task given many of the languages we work with have little to no language data available to build such systems.<\/p>\n

\n
\n
\n
<\/div>\n<\/div>\n<\/div>\n<\/figure>\n

\n

In this post, I\u2019ll explain some methods for dealing with low-resource languages. I\u2019ll also report on our experiments in obtaining a Tigrinya-English neural machine translation (NMT) model.<\/p>\n

The progress in machine translation<\/em>\u00a0(MT) has reached many remarkable milestones over the last few years, and it is likely that it will progress further. However, the development of MT technology has mainly benefited a small number of languages.<\/p>\n<\/div>\n<\/div>\n<\/section>\n

\n
\n
\n

Building an MT system relies on the availability of parallel data. The more present a language is digitally, the higher the probability of collecting large parallel corpora which are needed to train these types of systems. However, most languages do not have the amount of written resources that English, German, French and a few other languages spoken in highly developed countries have. The lack of written resources in other languages drastically increases the difficulty of bringing MT services to speakers of these languages.<\/p>\n

Low-resource MT scenario<\/h2>\n

In scientific literature for machine translation, there is no particular consensus on which corpus size constitutes a low-resource scenario. But we can say roughly that a low-resource condition is when the size of the parallel training corpus is not sufficient for reaching an acceptable result with the standard MT approaches. This is usually judged with a standardized automatic evaluation metric called BLEU, which correlates with human translation assessments.<\/p>\n

Figure 2, modified from\u00a0Koehn and Knowles (2017<\/a>), shows the relationship between the BLEU score and the corpus size for the three MT approaches.<\/em><\/p>\n

\n
\n
\n
<\/div>\n<\/div>\n<\/div>\n<\/figure>\n

\n

A classic phrase-based MT model outperforms NMT for smaller training set sizes. Only after a corpus size threshold of 15M words, roughly equivalent to 1 million sentence pairs, classic NMT shows its superiority.<\/p>\n

Low-resource MT, on the other hand, deals with corpus sizes that are around a couple of thousand sentences. Although this figure shows at first glance that there is no way to obtain anything useful for low resource languages, there are ways to leverage even small data sets. One of these is a deep learning technique called transfer learning<\/em>, which makes use of the knowledge gained while solving one problem to apply it to a different but related problem.<\/p>\n

Cross-lingual transfer learning<\/h2>\n

Zoph et al. (2018)<\/a>\u00a0applied transfer learning in machine translation and proved that having prior knowledge in translation of a separate language pair can improve translating a low-resource language.<\/p>\n

Figure 3 illustrates their idea of cross-lingual transfer learning.<\/em><\/p>\n

\n
\n
\n
\n
<\/div>\n

<\/p>\n<\/div>\n

<\/div>\n<\/div>\n<\/div>\n<\/figure>\n

The researchers first trained an NMT model on a large parallel corpus \u2014 French\u2013English \u2014 to create what they call\u00a0the parent model<\/em>. In a second stage, they continued to train this model, but fed it with a considerably smaller parallel corpus of a low-resource language. The resulting\u00a0child model<\/em>\u00a0inherits the knowledge from the parent model by reusing its parameters. Compared to a classic approach of training only on the low-resource language, they record an average improvement of 5.6% BLEU over the four languages they experiment with. They further show that the child model doesn\u2019t only reuse knowledge of the structure of the high resource target language but also on the\u00a0process of translation\u00a0<\/em>itself.<\/p>\n

The high-resource language to choose as the parent source language is a key parameter in this approach. This decision is usually made in a heuristic way judging by the closeness to the target language in terms of distance in the language family tree or shared linguistic properties. A more sound exploration of which language is best to go for a given language is made in\u00a0Lin et al. (2019)<\/a>.<\/p>\n

Multilingual training<\/h2>\n

The path that was cleared by cross-lingual transfer learning led naturally to the use of multiple parent languages. The straightforward approach, first described by\u00a0Dong et al. (2015)<\/a>, mixes all the available parallel data in the languages of interest and sends them into training as illustrated in Figure 4.<\/p>\n

\n
\n
\n
\n
<\/div>\n

<\/p>\n<\/div>\n

<\/div>\n<\/div>\n<\/div>\n<\/figure>\n

What results from the example is one single model that translates from the four languages (French, Spanish, Portuguese and Italian) to English.<\/p>\n

Multilingual NMT offers three main advantages. Firstly, it reduces the number of individual training processes needed to one, yet the resulting model can translate many languages at once. Secondly, transfer learning makes it possible for all languages to benefit from each other through the transfer of knowledge. And finally, the model serves as a more solid starting point for a possible low-resource language.<\/p>\n

For instance, if we were interested in training MT for Galician, a low-resource romance language, the model illustrated in Figure 4 would be a perfect fit as it already knows how to translate well in four other high-resource romance languages.<\/p>\n

\n
\n
\n
\n
<\/div>\n

<\/p>\n<\/div>\n

<\/div>\n<\/div>\n<\/div>\n<\/figure>\n

A solid report on the use of multilingual models is given by\u00a0Neubig and Hu (2018)<\/a>. They use a\u00a0\u201cmassively multilingual\u201d<\/em>\u00a0corpus of 58 languages to leverage MT for four low-resource languages: Azeri, Belarusian, Galician, and Slovakian. With a parallel corpus size of only 4500 sentences for Galician, they achieved a BLEU score of up to 29.1% in contrast to 22.3% and 16.2% obtained with a classic single-language training with statistical machine translation (SMT) and NMT respectively.<\/p>\n

Transfer learning also enables what is called a\u00a0zero-shot translation,<\/em>\u00a0when no training data is available for the language of interest. For Galician, the authors report a BLEU score of 15.5% on their test set without the model seeing any Galician sentences before.<\/p>\n

Case of Tigrinya NMT<\/h2>\n

Tigrinya is an Ethiopian language spoken by around 7.9 million people in Eritrea and Ethiopia. It is neither supported by any commercial MT provider, nor has any publicly available models. TWB is currently developing open datasets and MT for Tigrinya in cooperation with the\u00a0Masakhane initiative<\/a>.<\/p>\n

Tigrinya is no longer in the very low-resource category thanks to the recently released JW300 dataset by\u00a0Agic and Vulic<\/a>. Nevertheless, we wanted to see if a higher resource language could help build a Tigrinya-to-English machine translation model. We used Amharic as a parent language, which is written with the same Ge\u2019ez script as Tigrinya and has larger public data available.<\/p>\n

The datasets that were available to us at the time of writing this post are listed below. After JW300 dataset, the largest resource to be found is\u00a0Parallel Corpora for Ethiopian Languages<\/em><\/a>.<\/em><\/p>\n

Our transfer-learning-based training process consists of four phases. First, we train on a dataset that is a random mix of all sets totaling up to 1.45 million sentences. Second, we fine-tune the model on Tigrinya using only the Tigrinya portion of the mix. In a third phase, we fine-tune on the training partition of our in-house data. Finally, 200 samples earlier allocated aside from this corpus are used for testing purposes.<\/p>\n

As a baseline, we skip the first multilingual training step and use only Tigrinya data to train on.<\/p>\n

We see a slight increase in the accuracy of the model on our in-house test set when we use the transfer learning approach. The results in various automatic evaluation metrics are as follows:<\/p>\n

Conclusion<\/h2>\n

Neural machine translation is a data-hungry technology. Although this severely reduces the possibility to expand it to the majority of the world\u2019s languages, we can still apply various techniques to make it available to more people than if we limited ourselves to approaches tuned towards high-resource languages. Methodologies like transfer learning and linguistically informed data mixture have a role to play in helping everyone communicate in their language.<\/p>\n<\/div>\n

Written by Alp \u00f6ktem, Computational Linguist for Translators without Borders<\/pre>\n<\/div>\n<\/section>\n","protected":false},"excerpt":{"rendered":"
This article was originally posted in the TWB Tech Blog on medium.com TWB\u2019s current research focuses on bringing language technology to marginalized communities Translators without Borders (TWB) aims to empower people through access to critical information and two-way communication in their own language. We believe language technology such as machine translation systems are essential to … Continue reading “Transfer Learning Approaches for Machine Translation”<\/span><\/a><\/p>\n","protected":false},"author":25,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[296],"tags":[290,76,356,323,46],"class_list":["post-2854","post","type-post","status-publish","format-standard","hentry","category-language-technology","tag-data","tag-machine-translation","tag-neural-machine-translation","tag-tigrinya","tag-translators"],"yoast_head":"\nTransfer Learning Approaches for Machine Translation - Translators without Borders Blog<\/title>\n<meta name=\"description\" content=\"TWB\u2019s current research focuses on bringing language technology to marginalized communities.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/translatorswithoutborders.org\/blog\/transfer-learning-approaches-machine-translation\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Transfer Learning Approaches for Machine Translation - Translators without Borders Blog\" \/>\n<meta property=\"og:description\" content=\"TWB\u2019s current research focuses on bringing language technology to marginalized communities.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/translatorswithoutborders.org\/blog\/transfer-learning-approaches-machine-translation\/\" \/>\n<meta property=\"og:site_name\" content=\"Translators without Borders Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/translatorswithoutborders\" \/>\n<meta property=\"article:published_time\" content=\"2020-01-17T17:27:51+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-08-22T14:28:32+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/miro.medium.com\/max\/640\/1*tZjQkrtQ2fPn3EzUt-uEdg.png\" \/>\n<meta name=\"author\" content=\"Translators Without Borders\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@TranslatorsWB\" \/>\n<meta name=\"twitter:site\" content=\"@TranslatorsWB\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Translators Without Borders\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/transfer-learning-approaches-machine-translation\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/transfer-learning-approaches-machine-translation\\\/\"},\"author\":{\"name\":\"Translators Without Borders\",\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/#\\\/schema\\\/person\\\/eadfd55fcdedf3bd8feded8ac5046d39\"},\"headline\":\"Transfer Learning Approaches for Machine Translation\",\"datePublished\":\"2020-01-17T17:27:51+00:00\",\"dateModified\":\"2024-08-22T14:28:32+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/transfer-learning-approaches-machine-translation\\\/\"},\"wordCount\":1284,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/transfer-learning-approaches-machine-translation\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/miro.medium.com\\\/max\\\/640\\\/1*tZjQkrtQ2fPn3EzUt-uEdg.png\",\"keywords\":[\"Data\",\"machine translation\",\"neural machine translation\",\"Tigrinya\",\"translators\"],\"articleSection\":[\"Language Technology\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/transfer-learning-approaches-machine-translation\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/transfer-learning-approaches-machine-translation\\\/\",\"url\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/transfer-learning-approaches-machine-translation\\\/\",\"name\":\"Transfer Learning Approaches for Machine Translation - Translators without Borders Blog\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/transfer-learning-approaches-machine-translation\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/transfer-learning-approaches-machine-translation\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/miro.medium.com\\\/max\\\/640\\\/1*tZjQkrtQ2fPn3EzUt-uEdg.png\",\"datePublished\":\"2020-01-17T17:27:51+00:00\",\"dateModified\":\"2024-08-22T14:28:32+00:00\",\"description\":\"TWB\u2019s current research focuses on bringing language technology to marginalized communities.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/transfer-learning-approaches-machine-translation\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/transfer-learning-approaches-machine-translation\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/transfer-learning-approaches-machine-translation\\\/#primaryimage\",\"url\":\"https:\\\/\\\/miro.medium.com\\\/max\\\/640\\\/1*tZjQkrtQ2fPn3EzUt-uEdg.png\",\"contentUrl\":\"https:\\\/\\\/miro.medium.com\\\/max\\\/640\\\/1*tZjQkrtQ2fPn3EzUt-uEdg.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/transfer-learning-approaches-machine-translation\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Transfer Learning Approaches for Machine Translation\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/\",\"name\":\"Translators without Borders Blog\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/#organization\",\"name\":\"Translators without Borders\",\"url\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/i2.wp.com\\\/translatorswithoutborders.org\\\/blog\\\/wp-content\\\/uploads\\\/2017\\\/03\\\/cropped-twblogo_3-1.png?fit=248%2C65&ssl=1\",\"contentUrl\":\"https:\\\/\\\/i2.wp.com\\\/translatorswithoutborders.org\\\/blog\\\/wp-content\\\/uploads\\\/2017\\\/03\\\/cropped-twblogo_3-1.png?fit=248%2C65&ssl=1\",\"width\":248,\"height\":65,\"caption\":\"Translators without Borders\"},\"image\":{\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/translatorswithoutborders\",\"https:\\\/\\\/x.com\\\/TranslatorsWB\",\"http:\\\/\\\/translatorswb\",\"https:\\\/\\\/www.linkedin.com\\\/company-beta\\\/1500298\\\/?pathWildcard=1500298\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/#\\\/schema\\\/person\\\/eadfd55fcdedf3bd8feded8ac5046d39\",\"name\":\"Translators Without Borders\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/797d99aadcc65779070b9bcdddee2a1ed6ef28301ee0b8e63c8dcc304cfdb418?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/797d99aadcc65779070b9bcdddee2a1ed6ef28301ee0b8e63c8dcc304cfdb418?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/797d99aadcc65779070b9bcdddee2a1ed6ef28301ee0b8e63c8dcc304cfdb418?s=96&d=mm&r=g\",\"caption\":\"Translators Without Borders\"},\"url\":\"https:\\\/\\\/translatorswithoutborders.org\\\/blog\\\/author\\\/translators-without-borders\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Transfer Learning Approaches for Machine Translation - Translators without Borders Blog","description":"TWB\u2019s current research focuses on bringing language technology to marginalized communities.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/translatorswithoutborders.org\/blog\/transfer-learning-approaches-machine-translation\/","og_locale":"en_US","og_type":"article","og_title":"Transfer Learning Approaches for Machine Translation - Translators without Borders Blog","og_description":"TWB\u2019s current research focuses on bringing language technology to marginalized communities.","og_url":"https:\/\/translatorswithoutborders.org\/blog\/transfer-learning-approaches-machine-translation\/","og_site_name":"Translators without Borders Blog","article_publisher":"https:\/\/www.facebook.com\/translatorswithoutborders","article_published_time":"2020-01-17T17:27:51+00:00","article_modified_time":"2024-08-22T14:28:32+00:00","og_image":[{"url":"https:\/\/miro.medium.com\/max\/640\/1*tZjQkrtQ2fPn3EzUt-uEdg.png","type":"","width":"","height":""}],"author":"Translators Without Borders","twitter_card":"summary_large_image","twitter_creator":"@TranslatorsWB","twitter_site":"@TranslatorsWB","twitter_misc":{"Written by":"Translators Without Borders","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/translatorswithoutborders.org\/blog\/transfer-learning-approaches-machine-translation\/#article","isPartOf":{"@id":"https:\/\/translatorswithoutborders.org\/blog\/transfer-learning-approaches-machine-translation\/"},"author":{"name":"Translators Without Borders","@id":"https:\/\/translatorswithoutborders.org\/blog\/#\/schema\/person\/eadfd55fcdedf3bd8feded8ac5046d39"},"headline":"Transfer Learning Approaches for Machine Translation","datePublished":"2020-01-17T17:27:51+00:00","dateModified":"2024-08-22T14:28:32+00:00","mainEntityOfPage":{"@id":"https:\/\/translatorswithoutborders.org\/blog\/transfer-learning-approaches-machine-translation\/"},"wordCount":1284,"commentCount":0,"publisher":{"@id":"https:\/\/translatorswithoutborders.org\/blog\/#organization"},"image":{"@id":"https:\/\/translatorswithoutborders.org\/blog\/transfer-learning-approaches-machine-translation\/#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/max\/640\/1*tZjQkrtQ2fPn3EzUt-uEdg.png","keywords":["Data","machine translation","neural machine translation","Tigrinya","translators"],"articleSection":["Language Technology"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/translatorswithoutborders.org\/blog\/transfer-learning-approaches-machine-translation\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/translatorswithoutborders.org\/blog\/transfer-learning-approaches-machine-translation\/","url":"https:\/\/translatorswithoutborders.org\/blog\/transfer-learning-approaches-machine-translation\/","name":"Transfer Learning Approaches for Machine Translation - Translators without Borders Blog","isPartOf":{"@id":"https:\/\/translatorswithoutborders.org\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/translatorswithoutborders.org\/blog\/transfer-learning-approaches-machine-translation\/#primaryimage"},"image":{"@id":"https:\/\/translatorswithoutborders.org\/blog\/transfer-learning-approaches-machine-translation\/#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/max\/640\/1*tZjQkrtQ2fPn3EzUt-uEdg.png","datePublished":"2020-01-17T17:27:51+00:00","dateModified":"2024-08-22T14:28:32+00:00","description":"TWB\u2019s current research focuses on bringing language technology to marginalized communities.","breadcrumb":{"@id":"https:\/\/translatorswithoutborders.org\/blog\/transfer-learning-approaches-machine-translation\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/translatorswithoutborders.org\/blog\/transfer-learning-approaches-machine-translation\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/translatorswithoutborders.org\/blog\/transfer-learning-approaches-machine-translation\/#primaryimage","url":"https:\/\/miro.medium.com\/max\/640\/1*tZjQkrtQ2fPn3EzUt-uEdg.png","contentUrl":"https:\/\/miro.medium.com\/max\/640\/1*tZjQkrtQ2fPn3EzUt-uEdg.png"},{"@type":"BreadcrumbList","@id":"https:\/\/translatorswithoutborders.org\/blog\/transfer-learning-approaches-machine-translation\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/translatorswithoutborders.org\/blog\/"},{"@type":"ListItem","position":2,"name":"Transfer Learning Approaches for Machine Translation"}]},{"@type":"WebSite","@id":"https:\/\/translatorswithoutborders.org\/blog\/#website","url":"https:\/\/translatorswithoutborders.org\/blog\/","name":"Translators without Borders Blog","description":"","publisher":{"@id":"https:\/\/translatorswithoutborders.org\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/translatorswithoutborders.org\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/translatorswithoutborders.org\/blog\/#organization","name":"Translators without Borders","url":"https:\/\/translatorswithoutborders.org\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/translatorswithoutborders.org\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/i2.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2017\/03\/cropped-twblogo_3-1.png?fit=248%2C65&ssl=1","contentUrl":"https:\/\/i2.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2017\/03\/cropped-twblogo_3-1.png?fit=248%2C65&ssl=1","width":248,"height":65,"caption":"Translators without Borders"},"image":{"@id":"https:\/\/translatorswithoutborders.org\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/translatorswithoutborders","https:\/\/x.com\/TranslatorsWB","http:\/\/translatorswb","https:\/\/www.linkedin.com\/company-beta\/1500298\/?pathWildcard=1500298"]},{"@type":"Person","@id":"https:\/\/translatorswithoutborders.org\/blog\/#\/schema\/person\/eadfd55fcdedf3bd8feded8ac5046d39","name":"Translators Without Borders","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/797d99aadcc65779070b9bcdddee2a1ed6ef28301ee0b8e63c8dcc304cfdb418?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/797d99aadcc65779070b9bcdddee2a1ed6ef28301ee0b8e63c8dcc304cfdb418?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/797d99aadcc65779070b9bcdddee2a1ed6ef28301ee0b8e63c8dcc304cfdb418?s=96&d=mm&r=g","caption":"Translators Without Borders"},"url":"https:\/\/translatorswithoutborders.org\/blog\/author\/translators-without-borders\/"}]}},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p8ykT2-K2","jetpack-related-posts":[{"id":1791,"url":"https:\/\/translatorswithoutborders.org\/blog\/languagematters-technology\/","url_meta":{"origin":2854,"position":0},"title":"#LanguageMatters. So Does Technology.","author":"Translators Without Borders","date":"February 28, 2018","format":false,"excerpt":"Improving access to information in the right languages for the world\u2019s poorest, most vulnerable individuals is the core mission of Translators without Borders (TWB). Often, however, there are too few translators or interpreters available, especially during times of crisis when impacted populations and humanitarian responders do not speak the same\u2026","rel":"","context":"In "How we work"","block_context":{"text":"How we work","link":"https:\/\/translatorswithoutborders.org\/blog\/category\/how-we-work\/"},"img":{"alt_text":"Translators Guinea Language Technology.","src":"https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2018\/03\/Translators-1024x851.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2018\/03\/Translators-1024x851.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2018\/03\/Translators-1024x851.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2018\/03\/Translators-1024x851.jpg?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":4005,"url":"https:\/\/translatorswithoutborders.org\/blog\/unlock-the-power-of-language-launching-twb-learning-center\/","url_meta":{"origin":2854,"position":1},"title":"Unlock the power of language with the TWB Learning Center","author":"Translators Without Borders","date":"February 21, 2023","format":false,"excerpt":"Discover our free online translator training courses: launching TWB\u2019s new-look Learning Center!\u00a0 Introducing the new TWB Learning Center \u2013 a place for community members to gain experience in humanitarian translation and learn in-demand language industry skills.\u00a0 Part of CLEAR Global, TWB brings together over 100,000 language volunteers globally, helping people\u2026","rel":"","context":"In "Language Technology"","block_context":{"text":"Language Technology","link":"https:\/\/translatorswithoutborders.org\/blog\/category\/language-technology\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2023\/02\/homepage3-scaled.jpeg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2023\/02\/homepage3-scaled.jpeg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2023\/02\/homepage3-scaled.jpeg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2023\/02\/homepage3-scaled.jpeg?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2023\/02\/homepage3-scaled.jpeg?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2023\/02\/homepage3-scaled.jpeg?resize=1400%2C800&ssl=1 4x"},"classes":[]},{"id":3258,"url":"https:\/\/translatorswithoutborders.org\/blog\/5-takeaways-geong\/","url_meta":{"origin":2854,"position":2},"title":"Language and data-driven humanitarian action: 5 takeaways from a recent global discussion","author":"Translators Without Borders","date":"December 11, 2020","format":false,"excerpt":"In one word, what comes to mind when you think about language and data collection? Challenging, expensive, necessary. These are some of the answers we heard from attendees at a roundtable discussion TWB facilitated during the 2020 GeOnG Forum.\u00a0 Earlier this month, we were joined by panelists from IMPACT Initiatives,\u2026","rel":"","context":"In \"language data\"","block_context":{"text":"language data","link":"https:\/\/translatorswithoutborders.org\/blog\/tag\/language-data\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2020\/12\/Language-data-collection-transparent-background-1.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2020\/12\/Language-data-collection-transparent-background-1.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2020\/12\/Language-data-collection-transparent-background-1.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2020\/12\/Language-data-collection-transparent-background-1.png?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":1838,"url":"https:\/\/translatorswithoutborders.org\/blog\/reflecting-gala-conference-2018\/","url_meta":{"origin":2854,"position":3},"title":"A Recap of GALA’s Boston Conference","author":"Translators Without Borders","date":"March 21, 2018","format":false,"excerpt":"The Globalization and Localization Association (GALA) 2018 conference has come to a close. Those of us who managed to make it to and from Boston - despite the two feet of snow - were rewarded with a vibrant and exciting conference. As the Monitoring, Evaluating & Learning Manager for Translators\u2026","rel":"","context":"In "Fun-draising"","block_context":{"text":"Fun-draising","link":"https:\/\/translatorswithoutborders.org\/blog\/category\/fundraising\/"},"img":{"alt_text":"Latha Sukumar, Executive Director of MCIS and TWB donor, left, with Eric DeLuca, TWB","src":"https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2018\/03\/2018-03-21-05.16.39-2-e1521650467129.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":2600,"url":"https:\/\/translatorswithoutborders.org\/blog\/marginalized-mother-languages-two-ways-to-improve-the-lives-of-the-people-who-speak-them\/","url_meta":{"origin":2854,"position":4},"title":"Marginalized mother languages – two ways to improve the lives of the people who speak them","author":"Translators Without Borders","date":"February 21, 2019","format":false,"excerpt":"21 February. This is the date chosen by UNESCO for International Mother Language Day, which has been observed worldwide since 2000. This year deserves special attention as 2019 is the International Year of Indigenous Languages. Both initiatives promote linguistic diversity and equal access to multilingual information and knowledge. Languages can\u2026","rel":"","context":"In "Language Technology"","block_context":{"text":"Language Technology","link":"https:\/\/translatorswithoutborders.org\/blog\/category\/language-technology\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2019\/02\/IMG_1722-2-1024x768.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2019\/02\/IMG_1722-2-1024x768.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2019\/02\/IMG_1722-2-1024x768.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2019\/02\/IMG_1722-2-1024x768.jpg?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":4690,"url":"https:\/\/translatorswithoutborders.org\/blog\/translation-connects-us\/","url_meta":{"origin":2854,"position":5},"title":"Translation connects us: why language inclusion matters","author":"Translators Without Borders","date":"September 30, 2023","format":false,"excerpt":"On September 30th we celebrated International Translation Day. To mark the day, we\u2019re highlighting the work of language professionals and volunteers worldwide who help us to connect with others and access information and opportunities across language barriers. We\u2019re exploring how our TWB Community of over 100,000 people works at the\u2026","rel":"","context":"In "How we work"","block_context":{"text":"How we work","link":"https:\/\/translatorswithoutborders.org\/blog\/category\/how-we-work\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2023\/10\/3.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2023\/10\/3.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2023\/10\/3.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2023\/10\/3.png?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/translatorswithoutborders.org\/blog\/wp-content\/uploads\/2023\/10\/3.png?resize=1050%2C600&ssl=1 3x"},"classes":[]}],"rttpg_featured_image_url":null,"rttpg_author":{"display_name":"Translators Without Borders","author_link":"https:\/\/translatorswithoutborders.org\/blog\/author\/translators-without-borders\/"},"rttpg_comment":1,"rttpg_category":"<a href=\"https:\/\/translatorswithoutborders.org\/blog\/category\/language-technology\/\" rel=\"category tag\">Language Technology<\/a>","rttpg_excerpt":"This article was originally posted in the TWB Tech Blog on medium.com TWB\u2019s current research focuses on bringing language technology to marginalized communities Translators without Borders (TWB) aims to empower people through access to critical information and two-way communication in their own language. We believe language technology such as machine translation systems are essential to…","_links":{"self":[{"href":"https:\/\/translatorswithoutborders.org\/blog\/wp-json\/wp\/v2\/posts\/2854","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/translatorswithoutborders.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/translatorswithoutborders.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/translatorswithoutborders.org\/blog\/wp-json\/wp\/v2\/users\/25"}],"replies":[{"embeddable":true,"href":"https:\/\/translatorswithoutborders.org\/blog\/wp-json\/wp\/v2\/comments?post=2854"}],"version-history":[{"count":4,"href":"https:\/\/translatorswithoutborders.org\/blog\/wp-json\/wp\/v2\/posts\/2854\/revisions"}],"predecessor-version":[{"id":2859,"href":"https:\/\/translatorswithoutborders.org\/blog\/wp-json\/wp\/v2\/posts\/2854\/revisions\/2859"}],"wp:attachment":[{"href":"https:\/\/translatorswithoutborders.org\/blog\/wp-json\/wp\/v2\/media?parent=2854"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/translatorswithoutborders.org\/blog\/wp-json\/wp\/v2\/categories?post=2854"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/translatorswithoutborders.org\/blog\/wp-json\/wp\/v2\/tags?post=2854"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}