How to integrate dotenv with NestJS and TypeORM

Dotenv integration with NestJS and TypeORM.

While using third-party sources in application development, there must be some involvement of SSH keys or API credentials. This becomes a problem when a project is handled by a team of developers. Thus, the source code has to be pushed to git repositories periodically. Once the code is pushed to a repository, anyone can see it with the third-party keys.

A very prominent and widely used solution for this problem is using environment variables. These are the local variables containing some useful information like API keys and are made available to the application or project.

A tool known as dotenv has made it easy to create such variables and making these variables available to the application. It is an easy-to-use tool which can be added to your project by using any package manager.

We will use yarn as a package manager.

First, add the package using terminal.

yarn add dotenv

Since we are using NestJS which is based on typescript, so we need to add the “@types” package for the same that acts as an interface between javascript and typescript package.

yarn add @types/dotenv

Since the database to be used is Postgres, so install the necessary driver for Postgres.

yarn add pg

Now install the TypeORM module to your nest project.

yarn add @nestjs/typeorm typeorm

Now, create TypeORM entities in your project folder- For this illustration, we will be creating a folder ‘db‘ inside the ‘src‘ folder of our nest project and inside this folder, create another folder ‘entities‘ and create a typescript file containing information about your TypeORM entity.

For the sake of simplicity, we will create a user-entity file. Also, we will be creating an ‘id‘ field, a ‘name‘ field and an ‘email‘ field for this entity.

#src/db/entities/user.entity.ts

import { BaseEntity, Column, Entity, PrimaryGeneratedColumn } from 'typeorm';

@Entity({name: 'UserTable'})
class UserEntity extends BaseEntity {

  @PrimaryGeneratedColumn()
  id: number;

  @Column()
  name: string;

  @Column()
  email: string;
}

export default UserEntity;

Note that this entity is given the name ‘UserTable’ which is optional but in case of migration it becomes somewhat useful. We will get to know the reason shortly.

Now create a migration file for this user entity. Migration file can be created using a command-line interface with the following command:

typeorm migration:create -n CreateUserTable

This will create a migration file with the timestamp as a substring in the name of this file.

Here, ‘CreateUserTable‘ will be the name of your migration file created by the TypeORM environment. Now we will create a folder ‘migrations’ inside the ‘db’ folder and place the migration file inside it if it is not done already.

Now create a separate file that will be used as a migration utility to decide the schema of the database. Thus, we can name this file as migrationUtil.ts

Inside this migration util file, create functions to get various types of columns namely-varchar, integer etc.

We will be creating two functions for illustration, namely ‘getIDColumn‘ and ‘getVarCharColumn‘.

#src/util/migrationUtil.ts

import { TableColumnOptions } from 'typeorm/schema-builder/options/TableColumnOptions';

class MigrationUtil {

  public static getIDColumn(): TableColumnOptions[] {
    const columns: TableColumnOptions[] = [];
    columns.push({
      name: 'userId',
      type: 'int',
      isPrimary: true,
      isNullable: false,
      isGenerated: true,
      generationStrategy: 'increment',
    });

    return columns;
  }

  public static getVarCharColumn({ name, length="255", isPrimary = false, isNullable = false, isUnique = false, defaultValue = null }): TableColumnOptions {
    return {
      name,
      length,
      isPrimary,
      isNullable,
      isUnique,
      default: `'${defaultValue}'`,
      type: 'varchar',
    };
  }
}

export default MigrationUtil;

Here, ‘TableColumnOptions’ is a type provided by typeorm out of the box.

The code for this file is pretty straight whenever each of these functions are called, they create a separate column in your entity table.

Now, back to the ‘CreateUserTable’ migration file, the file should look like this:

#src/db/migrations/1578306918674-CreateUserTable.ts

import { MigrationInterface, QueryRunner, Table } from 'typeorm';

export class CreateUserTable1578306918674 implements MigrationInterface {

    public async up(queryRunner: QueryRunner): Promise<any> {
        
    }

    public async down(queryRunner: QueryRunner): Promise<any> {
        
    }

}

Now, add a table to this migration file using our migration utility file as:

#src/db/migrations/1578306918674-CreateUserTable.ts

....
private static readonly table = new Table({
        name: 'UserTable',
        columns: [
          ...MigrationUtil.getIDColumn(),
          MigrationUtil.getVarCharColumn({name: 'name'}),
          MigrationUtil.getVarCharColumn({name: 'email'}),
        ],
    });

....

Note that the name of this table is given same as the userEntity so as to improve entity-table mapping for developers. Also, finish up the code for async ‘up’ and ‘down’ methods using QueryRunner.

The idea is to create three columns in the user table – ‘userId’, ‘name’ and ’email’.

Thus, in the end, the migration file will be looking something like this:

#src/db/migrations/1578306918674-CreateUserTable.ts

import { MigrationInterface, QueryRunner, Table } from 'typeorm';
import MigrationUtil from '../../util/migrationUtil';

export class CreateUserTable1578306918674 implements MigrationInterface {

    private static readonly table = new Table({
        name: 'UserTable',
        columns: [
          ...MigrationUtil.getIDColumn(),
          MigrationUtil.getVarCharColumn({name: 'name'}),
          MigrationUtil.getVarCharColumn({name: 'email'}),
        ],
    });

    public async up(queryRunner: QueryRunner): Promise<any> {
        await queryRunner.createTable(CreateUserTable1578306918674.table);
    }

    public async down(queryRunner: QueryRunner): Promise<any> {
        await queryRunner.dropTable(CreateUserTable1578306918674.table);
    }

}

Now, create your environment files containing environment variables. We will be creating two .env files, namely- development.env and test.env.

The environment variables for development.env will be:

#env/development.env

TYPEORM_CONNECTION = postgres
TYPEORM_HOST = 127.0.0.1
TYPEORM_USERNAME = root
TYPEORM_PASSWORD = root
TYPEORM_DATABASE = dotenv
TYPEORM_PORT = 5432
TYPEORM_ENTITIES = db/entities/*.entity{.ts,.js}
TYPEORM_MIGRATIONS = db/migrations/*{.ts,.js}
TYPEORM_MIGRATIONS_RUN = src/db/migrations
TYPEORM_MIGRATIONS_DIR = src/db/migrations
HTTP_PORT = 3001

And the environment variables for test.env will be:

#env/test.env

TYPEORM_CONNECTION = postgres
TYPEORM_HOST = 127.0.0.1
TYPEORM_USERNAME = root
TYPEORM_PASSWORD = root
TYPEORM_DATABASE = dotenv-test
TYPEORM_PORT = 5432
TYPEORM_ENTITIES = db/entities/*.entity{.ts,.js}
TYPEORM_MIGRATIONS = db/migrations/*{.ts,.js}
TYPEORM_MIGRATIONS_RUN = src/db/migrations
TYPEORM_ENTITIES_DIR = src/db/entities
HTTP_PORT = 3001

Now, create a TypeORM config file for the connection setup.

We will place this file in the ‘config‘ folder under ‘src‘ folder of the project.

#src/config/database.config.ts

import * as path from 'path';

const baseDir = path.join(__dirname, '../');
const entitiesPath = `${baseDir}${process.env.TYPEORM_ENTITIES}`;
const migrationPath = `${baseDir}${process.env.TYPEORM_MIGRATIONS}`;

export default {
  type: process.env.TYPEORM_CONNECTION,
  host: process.env.TYPEORM_HOST,
  username: process.env.TYPEORM_USERNAME,
  password: process.env.TYPEORM_PASSWORD,
  database: process.env.TYPEORM_DATABASE,
  port: Number.parseInt(process.env.TYPEORM_PORT, 10),
  entities: [entitiesPath],
  migrations: [migrationPath],
  migrationsRun: process.env.TYPEORM_MIGRATIONS_RUN === 'true',
  seeds: [`src/db/seeds/*.seed.ts`],
  cli: {
    migrationsDir: 'src/db/migrations',
    entitiesDir: 'src/db/entities',
  },
};

Here, process.env will contain all our environment variables.

Note that the environment will be specified by us during command execution and thus, anyone of the files ‘development.env’ or ‘test.env’ will be taken as environment variables supplying file.

In the same folder, create another configuration file for dotenv and we will name it as ‘dotenv-options.ts’.

#src/config/dotenv-options.ts

import * as path from 'path';

const env = process.env.NODE_ENV || 'development';
const p = path.join(process.cwd(), `env/${env}.env`);
console.log(`Loading environment from ${p}`);
const dotEnvOptions = {
  path: p,
};

export { dotEnvOptions };

The code for this file is pretty straight.

Note that the line of code containing console.log call will let us know which environment is taken by the nest while executing commands and the same file is being provided as dotenv options below it.

Now, to successfully integrate dotenv with nest, it is recommended by official nest docs to create a config service along with a config module.

Thus, create a ‘services’ folder and inside that folder- create a ‘config.service.ts’ file.

#src/Services/config.service.ts

import * as dotenv from 'dotenv';
import * as fs from 'fs';
import * as Joi from '@hapi/joi';
import { Injectable } from '@nestjs/common';
import IEnvConfigInterface from '../interfaces/env-config.interface';
import { TypeOrmModuleOptions } from '@nestjs/typeorm';
import * as path from 'path';

@Injectable()
class ConfigService {
  private readonly envConfig: IEnvConfigInterface;

  constructor(filePath: string) {
    const config = dotenv.parse(fs.readFileSync(filePath));
    this.envConfig = this.validateInput(config);
  }

  public getTypeORMConfig(): TypeOrmModuleOptions {
    const baseDir = path.join(__dirname, '../');
    const entitiesPath = `${baseDir}${this.envConfig.TYPEORM_ENTITIES}`;
    const migrationPath = `${baseDir}${this.envConfig.TYPEORM_MIGRATIONS}`;
    const type: any = this.envConfig.TYPEORM_CONNECTION;
    return {
      type,
      host: this.envConfig.TYPEORM_HOST,
      username: this.envConfig.TYPEORM_USERNAME,
      password: this.envConfig.TYPEORM_PASSWORD,
      database: this.envConfig.TYPEORM_DATABASE,
      port: Number.parseInt(this.envConfig.TYPEORM_PORT, 10),
      logging: false,
      entities: [entitiesPath],
      migrations: [migrationPath],
      migrationsRun: this.envConfig.TYPEORM_MIGRATIONS_RUN === 'true',
      cli: {
        migrationsDir: 'src/db/migrations',
        entitiesDir: 'src/db/entities',
      },
    };
  }

  /*
  Ensures all needed variables are set, and returns the validated JavaScript object
  including the applied default values.
  */  private validateInput(envConfig: IEnvConfigInterface): IEnvConfigInterface {
    const envVarsSchema: Joi.ObjectSchema = Joi.object({
      NODE_ENV: Joi.string()
        .valid('development', 'test')
        .default('development'),
      HTTP_PORT: Joi.number().required(),
    }).unknown(true);

    const { error, value: validatedEnvConfig } = envVarsSchema.validate(
      envConfig,
    );
    if (error) {
      throw new Error(`Config validation error: ${error.message}`);
    }
    return validatedEnvConfig;
  }
}

export default ConfigService;

Here, ‘IEnvConfigInterface‘ is an interface provided explicitly by us to improve the understandability of code.

export default interface IEnvConfigInterface {
  [key: string]: string;
}

The dotenv.parse will read the contents of the file containing environment variables and is made available for use. It can accept string or buffer and convert it into an object of key-value pairs.

This object is then validated by using Joi schema object which is a library provided by hapi. Under this schema, we have specified that the environment (whether test or development) will be grabbed as the NODE_ENV key in the command line.

Also, if no environment is specified, then set the environment to ‘development’. Thus, our envConfig variable is now initialized with this validated object.

Now, create a configModule and import it to app module.

#src/modules/config.module.ts

import { Global, Module } from '@nestjs/common';
import ConfigService from './Services/config.service';

@Global()
@Module({
  providers: [
    {
      provide: ConfigService,
      useValue: new ConfigService(`env/${process.env.NODE_ENV || 'development'}.env`),
    },
  ],
  exports: [ConfigService],
})
export default class ConfigModule {
}

Here config service is injected into this module. But since our config service is expecting an argument through the constructor, we will use ‘useValue’ to provide this service an argument which by default is development.env file, if no environment is explicitly provided during execution of cli commands.

Now we will create another loader file that will load all the configurations for database and dotenv.

We will create this file in ‘cli’ folder under ‘src’ folder of our project and name it as ‘loader.ts’.

import * as dotenv from 'dotenv';
import { dotEnvOptions } from '../config/dotenv-options';

// Make sure dbConfig is imported only after dotenv.config

dotenv.config(dotEnvOptions);
import * as dbConfig from '../config/database.config';

module.exports = dbConfig.default;

Note that there is a comment in the code to import dbConfig only after dotenv config is imported. This is because our database configuration will depend on the environment used by nest.

Now in our package.json file under the ‘scripts’ section, we will add two key-value pairs that will be our cli command for migration.

...

"migrate:all": "ts-node ./node_modules/typeorm/cli migration:run -f src/cli/loader.ts",
"migrate:undo": "ts-node ./node_modules/typeorm/cli migration:revert -f src/cli/loader.ts"

...

Note that this command will directly execute our loader file.

And, that’s it!

We have successfully integrated dotenv with NestJS and TypeORM.

To test this, start your database server, and then run the following cli commands one after another:

NODE_ENV=development yarn migrate:all
NODE_ENV=test yarn migrate:all