Translate TypeScript projects with i18next

Introduction

Last month I had to add internationalization support for a Node.js project which had already been running in production. The project is not a typical web app but a worker process responsible for generating push notifications for a mobile application. I was looking for a simple internationalization library that could fit in without much effort and wasn't dependant on Express or other frameworks. After a quick research I found about i18next which promised to deliver all the features I was looking for. I was pleasantly surprised how fast I was able to integrate it and how well it worked for my use case.

What is i18next

i18next is a framework agnostic tool for translating JavaScript projects. It has simple API, it's easy to setup and provides thorough documentation. Most JavaScript frameworks, both front-end and back-end, have official bindings for i18next: Angular, React, Vue, Polymer, Express, Koa, Next.js and more! This is perfect for full-stack developers who can reuse the same API across projects using different frameworks. During my career as software engineer I have used angular-gettext, ngx-translate, transloco, react-intl, i18n-node, Polymer's app-localize-behavior and in-house solutions - all coming with their own APIs, workflow and documentation. Imagine how much time and effort I could have saved juts by using the same tool at all places.

What is i18next-scanner

A feature which I look after the most when deciding what internationalization library to use is the ability to automatically generate the translation files given to translators. This process consists of the following steps:

1. Scan the source code for translation strings.
2. Extract the found strings into resource files - JSON, PO or XLIF format for each language.
3. Merge the newly found strings with the existing strings.
4. Delete all unused translation strings. (optional)
5. Sort the keys alphabetically. (optional)

Having to do this manually for large code bases is painful, time consuming and error prone. That's why most internationalization frameworks provide their own tools to help developers in this task.

i18next-scanner is the scan and extract tool for i18next. There are three ways to use it:

• Command line interface
• Gulp or Grunt plugin
• Node.js API (for custom build scripts)

Integrating i18next in JavaScript projects

Let's see i18next in action by creating a simple JavaScript project.

mkdir i18next-sample && cd i18next-sample && npm init -y && touch index.mjs

I'm using Node.js 14.5.0 and the new ECMAScript modules feature instead of CommonJS modules (import/export instead of require()/module.exports). That's why I'm naming my file with the .mjs extension.

Now you must install i18next as a dependency for your project:

npm i i18next

and i18next-scanner as development dependency:

npm i -D i18next-scanner

Open the index.mjs file add the following contents:

import i18n from "i18next"
import { readFileSync } from "fs"

// load the translation files
const en = JSON.parse(readFileSync("./i18n/translations/en.json", "utf-8"))
const fr = JSON.parse(readFileSync("./i18n/translations/fr.json", "utf-8"))

i18n.init({
  lng: "en",
  resources: {
    en: {
      translation: en,
    },
    fr: {
      translation: fr,
    },
  },
})

console.log(i18n.t("greetPerson", { name: "John" }))

Now create a configuration file for the i18next-scanner tool called i18next-scanner.config.js:

module.exports = {
  /**
   * Input folder (source code)
   **/
  input: ["**/*.{mjs,js}"],
  /**
   * Output folder (translations)
   **/
  output: "./",
  options: {
    removeUnusedKeys: true,
    /**
     * Whether to sort translation keys in alphabetical order
     **/
    sort: true,
    func: {
      /**
       * List of function names which mark translation strings
       **/
      list: ["i18next.t", "i18n.t", "t", "__"],
      extensions: [".mjs", ".js"],
    },
    /**
     * List of supported languages
     **/
    lngs: ["en", "fr"],
    defaultLng: "en",
    /**
     * Default value returned for missing translations
     **/
    defaultValue: "",
    resource: {
      /**
       * Where translation files should be loaded from
       **/
      loadPath: "i18n/translations/{{lng}}.json",
      /**
       * Where translation files should be saved to
       **/
      savePath: "i18n/translations/{{lng}}.json",
      jsonIndent: 2,
      lineEnding: "\n",
    },
    keySeparator: ".",
    pluralSeparator: "_",
    contextSeparator: "_",
    contextDefaultValues: [],
    /**
     * Values surrounded by {{ }} are treated as params
     * e.g. "Hello {{ name }}" - "name" must be provided at runtime
     **/
    interpolation: {
      prefix: "{{",
      suffix: "}}",
    },
  },
}

I've taken the default options from the documentation and edited them to my likings. You can find the original defaults here.

Now let's run the scanner for the first time:

npx i18next-scanner

The scanning algorithm passed through our source code and found all usages of the i18n.t function and marked its first arguments as translation keys. It then generated the files i18n/translations/en.json and i18n/translations/fr.json which contain a single translation key with empty value:

{
  "greetPerson": ""
}

Let's add our first message which accepts a parameter called name.

{
  "greetPerson": "Hello, {{name}}!"
}

and run our application:

node index.mjs

# Output
Hello, John!

Now let's add another invocation of the i18n.t function but with a new translation key:

console.log(i18n.t("farewell"))

Running i18next-scanner again will result in the new string being added to all translation files. If we remove the last line in our source code and run i18next-scanner once again, the farewell key will be removed since it's no longer referenced in our source code.

Using i18next-scanner for TypeScript projects

Fast-forward to week ago, I had to bootstrap a new React Native application. Nowadays I'm always writing new projects in TypeScript as it catches most coding mistakes at build time and makes refactoring a breeze even in a large codebase. It's great for writing React components since it prevents incorrect usage of props and adds autocomplete capabilities in IDEs. Typechecking with PropTypes feels so clunky and inferior in comparison. Nowadays creating a new React Native project with TypeScript support is as easy as running a single command:

npx react-native init MyTypeScriptApp --template react-native-template-typescript

The next step for me was to add internationalization support. In my experience, the earlier in the project you set up internationalization, the better. I decided to use i18next to take advantage of my previous knowledge working with this library. Its i18next-react package provides React components and hooks for translating strings and changing the language at runtime. This is how you can install the library in a React project:

npm i i18next i18next-react

The library is intuitive to use in React and its hooks API is elegant. In comparison with react-intl, which I have used in other React apps, it does not require to pass the i18n object as a prop to each component which is going to use it. Instead calling the useTranslation hook inside any component will give you a reference to the i18next object instance. Neat!

So far so good, but when I ran the i18n-scanner command, to my dismay the translation strings were not extracted. I made sure that I ran the command with the correct glob pattern to match TypeScript files but that wasn't the cause. After having a look at the source code of i18n-scanner on GitHub I realized that the parser supports only JavaScript. The scanner does not understand how to work with .ts or .tsx files and it ignores them! The solution to this problem ended up being a simple one. You have to compile the project and run the scanner on the compiled JavaScript files. Let's create a command in our package.json which will take care of this hurdle.

{
  ...,
  "scripts": {
    "i18n:scan": "mkdir -p ./tmp && rm -rf ./tmp && npx tsc --jsx preserve --target ES6 --module es6 --noEmit false --outDir ./tmp && npx i18next-scanner"
  },
  ...,
}

The steps involved in this command are:

1. Before running the compiler and the scanner, delete the previous compilation artifacts.

   • mkdir -p ./tmp ensures rm -rf ./tmp does not error out with "folder does not exist"

2. Compile the source code into JavaScript and store the result in the tmp folder inside the project directory.

   • --jsx preserve ensures JSX is not compiled down to raw React.createElement() calls and instead leaves this task to Babel plugins. In our case the i18next-scanner needs to parse .jsx files for <Trans /> components which contain translation strings.
   • --target ES6 --module es6 makes sure import and export keywords are preserved and all components are in their own files (mirroring the TypeScript source files structure).
   • --noEmit false tells the compiler to actually write the compiled files on disk, otherwise the tmp folder won't be created. By default, React Native projects have noEmit: true in their tsconfig.json file!
   • --outDir ./tmp writes the compilation artifacts in the tmp folder.

3. Run the i18next-scanner on the compiled files.

This is what my i18next-scanner configuration file contains:

module.exports = {
  input: ["./tmp/**/*.{js,jsx}"],
  output: "./",
  options: {
    removeUnusedKeys: true,
    sort: true,
    func: {
      list: ["i18next.t", "i18n.t", "t", "__"],
      extensions: [".js", ".jsx"],
    },
    trans: {
      component: "Trans",
      i18nKey: "i18nKey",
      defaultsKey: "defaults",
      extensions: [".js", ".jsx"],
      fallbackKey: false,
    },
    lngs: ["en", "bg", "fr", "de"],
    defaultLng: "en",
    defaultValue: "",
    resource: {
      loadPath: "./src/i18n/translations/{{lng}}.json",
      savePath: "./src/i18n/translations/{{lng}}.json",
      jsonIndent: 2,
      lineEnding: "\n",
    },
    keySeparator: ".",
    pluralSeparator: "_",
    contextSeparator: "_",
    contextDefaultValues: [],
    interpolation: {
      prefix: "{{",
      suffix: "}}",
    },
  },
}

The translations will be extracted to src/i18n/translations/{{language}}.json:

├── src
│   ├── i18n
│   │   └── translations
│   │       ├── bg.json
│   │       ├── de.json
│   │       ├── en.json
│   │       └── fr.json

Now we can run:

npm run i18n:scan

and this will generate the translation files for all our supported languages.

Don't forget to add the tmp folder in your .gitignore file since build artifacts should not be tracked in version control systems.

And we're done! Have fun translating your TypeScript project with i18next!