IceFire Studios

IceFire Blog

Tips on using SharePoint in a multilingual environment

Microsoft Lists Templates are Unilingual

Now that list creation experience is making it easier to create lists from templates, and before the ability to create your own list templates rolls out, I thought I would point out that these Microsoft list templates are mostly unilingual.

Creating a new list from a template is very convenient, and it can be done in any language, and on any site, but the lists themselves are unilingual and are in the language of the user at the time that the list was created.

The language of the user depends on several things, including the alternate languages of the site.  The Microsoft Lists site itself can have alternate languages.  Depending on the year when your “my site” or OneDrive was created, it might have only one language or all languages activated.  You can check this by modifying the URL while on a Microsoft Lists or OneDrive site (it’s the same site) to this:

https://tenant-my.sharepoint.com/personal/(youremail)/_layouts/15/muisetng.aspx

When a list is created from a template, the names and descriptions of the list and the columns are created using the current language of the person who is creating the list, no matter the base language of the site.  If my language is English and I create a travel requests list, it is completely in English.

If I change my language and display the same list, it is also completely in English, or almost.  In the image below, where my language is French, you see that the word ‘’Today” in the date fields is replaced with the word “Aujourd’hui”


This is different from the old “create a custom list” experience, where at least the “Title” column would have a different name in different languages right from the start.

If my language is French at the time of creating a list from a template, it will create the list in French, even on a site where English is the site’s base language.

In the image below I created the list from the “Issue Tracking” template while my language was French, then changed my language to German.

The column “Anlagen” (attachment) is the only column which is translated out of the box, all the others are still in French.  It is possible to use the SharePoint MUI to add translations of the list and columns names and descriptions in all languages.  I cheated and used PointFire 365 to translate them all in a single step.  When that is done, the list headers are in the language of the user.

One thing that cannot be localized with the MUI is the values of Choice columns.  So for instance the possible values of the “Priority” column will be “Critique”, “Élevé”, “Normal”, and “Faible” in all languages.  The JSON that formats these columns is also generated in French in this case, where the formatting depends of the value of the column (in French).



You will also notice in the JSON that even though the list was created in French, the internal name of the column “Priority” is in English, and in fact is the same internal name in all languages.  That is handy when you write thing like Power Automate flows or PowerShell scripts, you don’t need to write different ones in different languages as long as you always refer to the internal name of the column, not the column title.  While you’re at it, if you’re writing JSON for formatting, make sure that everything is based on the @currentField, not @currentField.displayValue or friendly formats, because those can be language-sensitive and stop working when users have a different language.

Simple Trick to Translate the SharePoint Hub Name

SharePoint's hub navigation supports localization, that is to say if you change your language then you can see the navigation links in that language... eventually.  The navigation links are cached so it may take a while for people to see the translated versions.

That applies to the text of the links, because they support the MUI (Multilingual User Interface) where the text of the user interface element can be different for different languages, but it does not work for the "Hub name" which appears in the global navigation menu.  It is the same in all languages.

As you can see above. the navigation is localized, but the name of the hub, "Global Navigation", is not.

Fortunately, there is a simple workaround for this.  The first step is to hide the Hub name.  Go to the settings (gear icon) then Hub site settings, and select "Hidden in navigation"


The hub name will disappear from the navigation bar.  Next, we will add it back as a label.  Edit the navigation menu and create a new item, and make this item a Label, and give it the same name as the Hub name.  For the purpose of demonstration, I used a lower case for the word "navigation" so that I can tell that it is the label and not the original name.


Make sure it is the first item in the menu and save.

This label does support MUI.  You can localize it.  I used PointFire 365 to do this, but you can also do it manually.



The end result looks exactly like the hub menu, showing the hub name, except that the Hub name is localized.

Inuktitut machine translation is here. Why that's a big deal.

Azure Translator Text now supports the Inuktituk language spoken in the Inuit area in the far north of North America.

Over the years, I've received a lot of requests to provide machine translation for Inuktitut.  Despite the tools that are available, particularly from Microsoft, to train your own neural translation engine for an unsupported language using a corpus of translated documents, and a great bilingual corpus from the debates of the Nunanut legislative assembly, I knew that this would not be possible.  Other machine translation experts also agreed that it was beyond the state of the art.  On a couple of occasions I had applied for funding to push beyond the state of the art to make this possible, unsuccessfully.  Why is machine translation of Inuktitut so difficult?


Inuktitut belongs to a class called "polysynthetic languages".  Most of the languages that you know are probably "agglutinative"  There are some root words which can be modified by changing the beginning or the end of the word.  The root word is in the dictionary, but the words that are modified by adding or changing suffixes and prefixes are typically not, because eveyone knows the rules.  These agglutinative languages are part of a larger class called synthetic languages, which includes other simple rules for sticking words together, usually with a small set of rules that apply to one part of speech.  For example German can stick a lot of known nouns end-to-end to make a new word, but there is one root word and all the other words are modifying or narrowing down the sense one of the words, and the resulting word behaves like a longer version of the base word, and has the same part of speech.

Inuktitut is polysynthetic.  The combination rules are much more complex.  There can be several root concepts and root words, and it can lose its part of speech or change it because the full word is an entire sentence with subject object, verb, adjective, even subordinate clauses, all contained in a big compound word. How you join words together can vary using complex rules about what comes before and after the join.  A well known example is the word "ᖃᖓᑕᓲᒃᑯᕕᒻᒨᕆᐊᖃᓛᖅᑐᖓ" which means "I'll have to go to the airport".  Verbs, nouns, subject, object, they're all contained in the same word.

Not all but most native American languages are polysynthetic.  Unlike other languages, neural networds can't just have a dictionary and some rules and train the translation engine to see patterns of three or four words in a row that always translate to the same 3 or 4 words in another language.  Almost all the neural translation engines I have seen are word-based.  There are languages that are written without spaces, like Chinese, Japanese, Thai, and Korean, but they still have individual words and breaking them up into individual words is relatively simple.  Not so with polysynthetic languages.

I don't see any information about how Microsoft tackled the problem for Inuktitut.  I am assuming that they used a tool to break down words into morphemes.  What I would have used but didn't get funding for was the National Research Council's Uqailaut Inuktitut Morphological Analyzer, but I don't know whether Microsoft did something similar.  I am watching for any publications about it. There have been some advances lately in modeling and translating these languages, so that is not the only approach.

On the other hand, perhaps they trained a neural network to decompose into morphemes and vice-versa without a standalone processor.  If that's the case, then the same techniques could be used for various other widespread but hard to translate polysynthetic languages from the Algonquian language family like Cree and Ojibwe, or Iroquoian languages like Mohawk, or Athabascan langauges like Dene or Navajo, or Siouan languages like Dakotan.  It's a game changer.

Oh, and if you were curious, PointFire Translator now supports translation to and from Inuktitut on SharePoint sites, just use language code "iu".  Your browser should already support Canadian Aboriginal Syllabics.

Script Files That Download Rather Than Executing

It started with an intermittent problem from a client.  In some cases their users were being prompted to download a JavaScript file that had our product's name somewhere in the file name.

That file was a localization file that SPFx uses.  When you create an SPFx webpart or application customizer, there is a "loc" folder in which you put files that contain the character strings that vary according to language.  You can find more details about SPFx localization here. Each file in that folder will have a name corresponding to the locale, for example "en-us.js" and "fr-fr.js".  These files will then be packaged under a longer name, consisting of

(the name in the project)_en-us_(some long hex string).js

It turns out that this file was the one that was sometimes downloaded. Looking more deeply at why it was being downloaded sometimes when other JavaScript files weren't, it turns out that there was a difference in the http headers that were being sent.


The major difference between the .js file that was sometimes being downloaded and the ones that weren't was that the ones being downloaded had a "content-disposition" header and the the others didn't.  In theory, a content-disposition of "attachment" with a file name means that this file should be downloaded, not sent to whatever is responsible for the MIME type represented in the Content-Type header.  In practice, browsers know that web servers often get the http headers wrong, so they look for other evidence of the intent.  Most of the time they look at the content-type and only download the file if its value is "application/octet-stream".  They apparently also look at the context in which is it being called.  

In the case of the client, the download usually came when the user had just logged out and logged back in again.  Their authentication was not the usual one, so it involved some redirection to and from their on premise identity management.  If the caching of this JavaScript file had expired but other files were in the cache, it could very well be the first file being fetched from the SharePoint Online domain, so no context is available for the browser to take into account.  In that case, it seemed, it respected the incorrect Content-Disposition header. Also, they were not using and did not want to use any CDN to serve these files, unlike most of our other clients with a similar architecture.  Files served by a CDN tend to have the correct http headers.

That behaviour is easily reproduced, by going directly to the appropriate folder of the App Catalog's ClientSideAssets library and clicking on the file name.  That file prompted the user to download it, while other .js files in the same folder did not.

I won't bore you with everything that we tried to figure this out.  As some point we were convinced that this happened to with application customizers but did not happen to webparts.  As it turns out, the problem was with our naming convention.  using the words "modern application customizer" in the name rather than "modern webpart" made a difference.

The difference was the name of the JavaScript file, specifically the length of the name.  The file name was already pretty long, but the difference between the words "webpart" and "application customizer" took it over some magic threshold.  Our naming convention sabotaged the attempt to reproduce the problem with a "hello, world" test.  If looks like a file with a name of 81 characters has the correct http header, but one of 87 has the wrong one.  I'm still not sure what is the exact limit, and whether it is a limit on the file name or the path or the full URL.  None of those lengths seem close to some magic number, but it doesn't matter since shortening the part of the file name that we are responsible for fixes the problem.





A few new languages

Because of recent changes to the Azure Translator Text API, a few new languages in the Western Iranian language group are being added to PointFire Translator.


Two versions of Kurdish are being introduced, Kurdish (Central), language code "ku", also called Sorani,  and Kurdish (Northern), language code "kmr", also called Kurmanji.

In addition, two languages of Afghanistan are being added.  
Dari (prs) and Pashto (ps).  PointFire Translator had already supported Dari and Persian, and Dari is also supported by PointFire 365 and SharePoint.  However, we had been using the same language code and translation engine for both of them, because they are so similar.  
Now Dari will use "prs" and Persian will use "fa" for Farsi, another name for the same language.

SharePoint 2019 Machine Translation Service is back!

When SharePoint 2019 was first released, its Machine Translation Service did not work.  The service could be installed and was running, but most attempts to use of it would result in the error message "The service application required to complete this request is unavailable. Try this operation again later. If the problem persists, contact your administrator."

The ULS log would have a message of "Unimplemented method" with a stack trace in Microsoft.Office.Web.Conversion.Framework.

Microsoft  has now resolved that error!  If you install the July 2020 or later Cumulative Update for SharePoint 2019 then the service is working again.

The Machine Translation Service is used by two optional features of PointFire 2019: the machine translation of user interface elements and the translation of classic pages and documents in SharePoint libraries.


These functions now work as expected.  The next release of PointFire 2019 will have some performance improvements that were delayed because of the difficulty in testing improvements to a feature that very few could use.

Because of the ongoing problem, all users of PointFire 2019 have been able to get free annual licenses for PointFire Translator, which can translate documents, pages, lists, and UI elements in SharePoint 2019 with higher quality than the free Machine Translation Service (although check out this secret setting).  PointFire Translator can also translate modern pages Excel, PowerPoint, and PDFs, which the Machine Translation Service cannot do, and covers more languages.  This free license program will soon end. If your free PointFire Translator license is expiring, contact us about transitioning to the Machine Translation Service or renewing the license.

Supporting another 12 Languages

In the most recent version of PointFire Translator (beta) we are introducing new or enhanced support for 12 new languages.

Of these, four are languages that are supported by SharePoint.  Irish and Kazakh languages are now supported for machine translation.  That means if your SharePoint site supports Irish or Kazakh, PointFire Translator can now translate its pages, documents, and lists, and PointFire 365 will automatically filter and/or redirect as appropriate.  If you want to translate the user interface, contact us, one of the steps is different for those languages than for other languages.

PointFire Translator now supports European Portuguese and Brazilian Portuguese as two separate languages.  Before this, the same translation engine was used for both, a neutral Portuguese that was actually closer to the Brazilian version.

Several new languages have been added to PointFire Translator which are not supported by SharePoint, including Māori (New Zealand), and five languages from India and Pakistan: Marathi, Gujarati, Punjabi, Malayalam and Kannada.  PointFire Translator will happily translate to or from those non-SharePoint languages, but PointFire 365 will be unable to filter by that language code.

All of those new languages have Neural Network engines behind them.  Irish, Brazilian Portuguese, Marathi, Gujarathi, and Māori have a customizable engine, meaning you can re-train it with your own documents to improve the translation quality.


The other two languages, or rather one language and two scripts, are ones that PointFire Translator had supported before and which had stopped working and discontinued.  In preparation for the Galactic Collaboration Summit we decided to brush off our Klingon translator.  This is where we discovered that there had been an undocumented change to Microsoft's Klingon language codes.  So we are happy to announce that we have reinstated Klingon (Latin script) and Klingon (pIqaD script).  If you choose the pIqaD script, make sure that you download a font that supports it, and change the font on the document.  This language only has a statistical translation engine, not a neural translation engine, so the quality is not very good.  But to paraphrase Samuel Johnson, it is like a dog's walking on his hind legs. It is not done well, but you are surprised to find it done at all.

If you're keeping count, that is 73 languages in total.

Changes to Serbian language codes in SharePoint Online

One of the changes that went into general availability last week was the retiring of the "sr-Latn-CS" language code, Locale ID 2074.

"Serbian (Cyrillic, Serbia)" [LCID=10266, language code "sr-Cyrl-RS"] was renamed "Serbian (Cyrillic)" and 
"Serbian (Latin, Serbia)" [LCID=9242, language code "sr-Latn-RS"] was renamed "Serbian (Latin)"


"CS" in the sr-Latn-CS was the former country code for the State Union of Serbia and Montenegro until it was dissolved in 2006 and Serbia and Montenegro became separate countries.  "RS" is the code for "Republic of Serbia" not to be confused with Republika Srpska. I have never noticed any UI difference between sr-Latn-CS and sr-Latn-RS, they were probably identical.  Many users in countries other than Serbia where Serbian is used, such as Montenegro or Bosnia and Herzegovina, tended to use the code because it did not mention Serbia in the name.  In case you're wondering, yes the versions of Serbian used in Bosnia and in Montenegro are different from the one used in Serbia, in fact there are a few more letters in the alphabet.  However they are not supported as distinct languages in SharePoint.


The locale IDs sr-Latn-RS and sr-Latn-BA  still exist in Delve, labelled with the country name for display language and for locales, and if you choose one there is a mapping done to the supported version of Serbian that uses the same script.

SharePoint Multilingual Page Publishing Feature in Detail

The new multilingual page publishing feature for Modern sites is now available in Targeted Release.  This is a good time to go through its main features in a bit more detail.

This SharePoint Online feature enables the publishing and consumption of pages and news in multiple languages in a modern SharePoint communication site. Multi-lingual page publishing means that on modern communication sites, site owners can determine in what languages they want to translate individual pages and news posts. This is different from Variations in that the translated pages are on the same multilingual site, but similar in that it copies pages and you have to translate them to the other language.


Turning it on

This feature must be turned on before it has any effect.  The activation is not, as you might think, in the site features section, but in Language settings.



Language settings was there before, this is where you used to set the Alternate languages for a site.  You can still do that more or less like before if you don't turn the feature on, by clicking on "Show advanced settings"

The new option is called "Enable pages and news to be translated into multiple languages"

The switch to turn on that feature appears on Communication sites, either new ones or existing ones.  It does not appear on Classic sites and it does not even appear on Modern Team sites.


If you click on "Show advanced settings" without enabling the feature, you can pick the list of alternate languages like before, and like before they are all turned on by default.


Parenthesis, what is Overwrite translations at the bottom?  It’s the same as before, with a different wording.  It has to do with what happens when you change MUI-enabled text in the site default language when it has has already been translated, will it leave the translations alone, or will it overwrite them with the new text in the original language as an indication that they need to be re-translated.

The second that you turn on the feature, even if you immediately turn it off again, all the site alternate languages are unselected.  You have to turn them on again one by one, using a different interface if the feature is activated.



 

The list of languages is the same as it was before with one exception, Serbian (Latin) has been dropped.

To select alternate languages, you can type a few letters of the language name or select it from the dropdown list



Selecting a language has two effects: it adds it to the list of alternate languages which determines the languages supported by the multilingual user interface (MUI), and it adds it to the list of languages to which site pages can be translated.

For each language you can select one or more "translator".  These are accounts that are notified by email whenever a page on this site should be translated to that language.  Naming someone as the translator does not give them rights to edit the pages, it simply sends them the email that contains a link to the page.  It is possible to send the email to external users, and it is possible to send it to people with no access to the list or to the site.  Sending it to a group works as well.

However if you type an email that’s not in AAD, then it gives a message when you save “Can’t save setting now.  Try again later”.  Always remember to hit save.



Adding pages

Since this is a Communication site, pages in the site default language can be added to the Site Pages library like before using Add a Page or by creating a News post.  Jumping ahead, creating a page, even with a different language selected, or using a language-specific News web part will always create pages that are assumed to be in the site default language and are in the root directory of the Site Pages library. 



"Translate" pages

You notice in the page menu bar that there is a "translation" button.  This will open a translation pane on the right.


As it clearly explains, this does not actually translate (unless you have PointFire Translator), it creates a nearly exact copy of the page either in all languages or in one language at a time.  You can do this with a draft or a published version, but keep in mind who can and cannot see drafts.  If you do not create them all at once, you can create the others individually or for all remaining languages at once.


All created pages are drafts initially.  It is then possible to get to that page in at least four ways.

1) Follow the "view" link
2) Follow the email link in the email that is sent to translators when the page is created


3) follow the little "Available languages" page navigation dropdown in the top right corner


Notice how this page navigation reminds you which page is published and which is a draft.

4) Navigating to the folder


When a translation page is created, it is put in a language-specific folder, with the same file name.  These folders use the PointFire naming convention, a two-letter code for languages where SharePoint supports only one variant, and language-dash-country or language-dash-script where there are more variants.

When you first visit the language-specific copy, you will notice that the title is changed to “Translate into {Language name}: {Original title}”.  You will also notice a message at the top, "This page has not been translated into {Language name} yet.  Edit to start translation"


This message remains until the page has been Saved once, even if you went in edit mode and changed nothing and it autosaved.  You (or someone else) are responsible for translating the content of the page, its title and the title and content of every webpart.  Remind translators to delete, not translate, the “Translate into {Language name}: ” part of the title.


User Language and Issue with Right-to-Left languages

Depending on the user's SharePoint language, there is a possibility of the user being redirected to the version of the same page in the user's language.  This redirection is not persistent, it is easy to navigate to pages in other languages without changing your own language.

When you go to a page in a right-to-left language while your current language is one that has left-to-right layout, you will see it with a left-to-right layout while editing.  The letters are RTL but the layout is LTR


If your current language is a right-to-left one, then you will see a right-to-left layout.  The layout depends on the user’s language, not the language of the page or of the site.   For those interested in the technical details, the "lang" and "dir" properties of the page's html element follow the user language settings.


Language of the user interface

This may be a good time to point out that while the feature will let you translate and navigate to pages in other languages, what you will see is only the page content in that language, the entire user interface will not switch to that language.  That's why in the first Arabic picture above you see the UI in one language and the content in the other language.  In the second Arabic picture, the content language and the UI language match, because the user's language setting matches the page language.  Changing your own SharePoint language is no easy feat.

Does this feature translate the UI?  No.  There are a number of UI elements that support MUI, and out-of-the-box elements are mostly populated with localized elements, but any customization that you do, for example to the navigation, will not be translated unless you translate it manually.  Here is how to localize your SharePoint navigation, but the same technique should be used for all localizable UI elements, including title/description of sites, lists, columns, content types, and views.

Can you translate custom SPFx web parts?  Almost never.  Whoever developed the webpart might have given you the ability to change certain strings on the page, but developers should probably also follow the guidance to localize the webpart.  This localization will follow the user's language, not the page or site language.
 

Deleting pages

What if you make a change to the original page?  Can you re-copy it to the translations? No (again, unless you have PointFire Translator).  However now is a good time for human translators to discover the new SharePoint page version comparison feature to highlight what has changed and determine what need to be re-translated.

It won’t re-copy unless you delete the translated copy.  The feature adapts well to deleting the translated copy. The “Available languages” menu adapts, as does the translation pane.

If you delete the original, the copies stay.  However they feel some distress.  If you click on the “Available languages” menu while on one of the orphaned copies, it spins and never returns

 

If you turn off the feature, all the pages stay, but most of the language functionality is removed.  The functionality returns if the feature is turned back on again.


 News Webpart language filtering

One cool automatic feature I'm still trying to find the limits of is the filtering of News and Highlighted Content webparts by language.  If you have a News webpart on a default language page, it will only show published news items that are in the default language.  However if that same webpart is on a translated language page, the webpart only shows published news items that are in that language.  The filtering is automatic, there are no explicit filter conditions that are added or can be modified.  If you turn off the feature, then the filtering stops.  Does it work with Hub news or on the SharePoint home page?  Stay tuned, not certain yet.

  

How PointFire Translator completes the functionality

For more details see the blog post about machine translation for multilingual page publishing.  Here is a video that shows how this automatic machine translation works


Essentially, the machine translation process can be completely automated.  When you view the new page for the first time, what you will see is not a copy of the original page for you to manually translate, but a translated page for you to approve and publish.  This saves huge amounts of time and money.

SharePoint’s New Multilingual Publishing Feature and How PointFire Translator Integrates With It

(see also SharePoint Multilingual Page Publishing Feature in Detail)
The initial documentation for Microsoft’s new multilingual pages functionality for modern pages is now available online.  You can find it here.

According to the recent announcement, "this new feature enables the publishing and consumption of pages and news in multiple languages in a modern SharePoint communication site. This is an opt-in feature. Site owners must take action to enable this experience.... We will be gradually rolling out this feature to Targeted Release (entire organization) customers by the end of March 2020. The roll out will be completed for all customers, by the end of May 2020."

When it is activated, site owners will be able to select the languages to which they want modern pages to be translated, and the persons who should be notified when a translation is requested.

 

When a modern page has been created, a translation can be requested.  On the top bar of the page, select the Translation button.


This brings up a translation pane on the right of the page.  To create a page for translation in all the languages configured for your site, select "Create for all languages". Otherwise, you can select the individual languages you want by clicking "Create".

 

If you are one of the people named as a translator for that language, you will get an email that looks like this:

 


The out-of-the-box experience when clicking on the “Start translating” button is to bring you to a copy of the original page in the original language, saved in a language-specific folder of the Page library.  You can start translating this copy of the page, replacing the text in the title and in the various webparts with text in the desired language, provided that you have the required permissions.

This is where PointFire Translator integration comes in.  SharePoint's new out-of-the-box feature does not include machine translation.  If you have version 2.0 of PointFire Tranlator and someone has set up the automation or the scheduling features of PointFire Translator Server, then when you click on “Start translating” in your email you will see not the page in the original language for you to translate, but a freshly translated page for you to check, modify if required, and publish.  It’s that simple.  PointFire Translator will have translated the page, its title, the text within text and hero webparts, webpart titles, translatable text metadata, etc.

If notification was not set up in the OOTB feature, or if you have not enabled the automation or the scheduling features of PointFire Translator, then you will have to initiate the translation manually.  Go to the appropriate Pages library, then pick the folder with the language code corresponding to the desired language.


For those who have used PointFire products before, SharePoint is now using the same language codes that PointFire had been using.  Then within the folder, select the page that you want to translate, and either select “PointFire Translator” in the item menu or click on the item to activate the checkmark in the circle to the left of the item and click on the the “PointFire Translator” button.


Besides the ability to translate the modern pages created by this SharePoint OOTB feature, you can still get the other benefits of PointFire Translator, including the ability to translate classic pages, to translate documents in Word, Excel, PowerPoint, and PDF format, and to translate list items, as well as the ability to translate entire libraries and lists at once rather than single items or wildcard matches.

More detailed information will be posted here when Microsoft makes it public.

Office365™ - office.microsoft.com IceFire Studios is a Microsoft Partner SharePoint™ 2013 - Microsoft.com