iTranslated by AI
NestJS Configuration Practices: .env and Environment Variables, Validation, and Custom Configuration Services
NestJS Configuration is an Initial Development Hurdle
When developing with an application framework, one of the first areas you want to solidify is configuration. Configuration involves things like:
- Making environment variables that differ by execution environment (local, staging, production, etc.) accessible within the application.
- Setting up client libraries used throughout the entire application.
NestJS, of course, provides methods for configuration.
https://docs.nestjs.com/techniques/configuration
Among the various setup methods available, I believe using .env files and environment variables is the most simple and clear approach. However, there are additional considerations such as overriding environment variables during testing and validation. In this article, I will introduce how to handle environment variables that vary by execution environment (local, staging, production, etc.) in a NestJS application while performing overrides for testing and validation. Let's look at this concretely, assuming a scenario immediately after project creation.
Initialization with Configuration Module
NestJS recommends incorporating or creating features in units called Modules.
Configuration is no exception and is provided as a Module. To use it, you install the @nestjs/config package.
yarn add @nestjs/config
With this, you can handle Configuration as a NestJS module. Being treated as a module means it becomes a target for DI (Dependency Injection), allowing you to pass environment variables or external files around as instances within the NestJS world.
Usage is simple: just specify it in imports within app.module.ts like any other module.
import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
@Module({
imports: [ConfigModule.forRoot({
envFilePath: ['.env.development.local'],
})],
})
export class AppModule {}
Look at envFilePath. Here, a dotenv file is specified. It seems that configuration values are being read from this file. I see—if we write all environment variables in this file, we could connect to various external services including databases. However, in reality, there are various requirements based on team policies or key management policies for external services. For example, let's say we want to satisfy the following requirements:
- List environment variables in
.env.development.localas much as possible and keep them under version control. - Expand sensitive information that cannot be version-controlled into local machine environment variables.
- Assume the deployment target is Cloud Run, and use those expanded into environment variables via integration with Secrets Manager.
- Override with environment variables for testing when running tests with Jest.
- Validate the environment variables being loaded.
- Call the loaded environment variables more intuitively, such as using
isProduction().
When it comes to these types of requirements, the official documentation alone might leave you at a standstill. While looking at what NestJS Configuration can do, let's think about how to achieve each of these.
1,2,3: NestJS Merges .env Files and Environment Variables, So Let's Leverage That
Fortunately, points 1, 2, and 3 are mostly covered by the Configuration Module. The dotenv file specified in envFilePath and the environment variables are merged and become accessible from the application. Let's assume the following situation:
NODE_ENV=development
DATABASE_URL="postgresql://postgres:password@127.0.0.1:54321/hello_db_development?schema=public"
MICROCMS_KEY=use-environment-variables
export MICROCMS_KEY=1234567890
In this case, as shown at the beginning, initialize the Module by specifying .env.development.local in envFilePath.
import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
@Module({
imports: [ConfigModule.forRoot({
envFilePath: ['.env.development.local'],
})],
})
export class AppModule {}
When you start the application with this, you can obtain the values as follows:
import { ConfigService } from '@nestjs/config';
import { Query, Resolver } from '@nestjs/graphql';
@Resolver()
export class PostsResolver {
constructor(
private configService: ConfigService,
) {}
@Query(() => String)
helloConfiguration(): string {
const nodeEnv = this.configService.get<string>('NODE_ENV'); // development
const databaseUrl = this.configService.get<string>('DATABASE_URL'); // postgresql:/...
const microCmsKey = this.configService.get<string>('MICRO_CMS_KEY'); // 1234567890
}
}
.env.development.local and environment variables are merged, and those specified as environment variables take precedence over the dotenv file for the same key name.
When a key exists both in the runtime environment as an environment variable (e.g., via OS shell exports like export DATABASE_USER=test) and in a .env file, the runtime environment variable takes precedence.
https://docs.nestjs.com/techniques/configuration#getting-started
Therefore, in local development, it is good practice to actively separate information that can be version-controlled into files like .env.development.local and manage sensitive information with direnv. In an execution environment, if you expand all necessary information into environment variables, .env.development.local can be completely ignored. It's best to use the features of your deployment target effectively here. For example, Cloud Run automatically expands Secrets Manager values into environment variables.
Manage configuration values you want to version-control with dotenv, and override everything for sensitive information and execution environment settings with environment variables. This policy should keep the setup simple.
4. Overriding with Mock Environment Variables for Jest Testing
Testing is another scenario where you might consider different environment variables. For example, while you might use a sandbox environment for external services during development, you may want to completely mock them during tests to avoid any external connections.
With Jest, you can execute an arbitrary setup file before running your tests, allowing you to write logic to override environment variables there.
import * as dotenv from 'dotenv';
import * as path from 'path';
// It would be ideal to have an option to override environment variables
// https://github.com/motdotla/dotenv/issues/517
const testEnv = dotenv.config({
path: path.join(process.cwd(), '.env.test.local'),
});
// Explicitly override environment variables
Object.assign(process.env, {
...testEnv.parsed,
});
MICROCMS_KEY=test-dummy
"jest": {
"moduleFileExtensions": [
"js",
"json",
"ts"
],
"rootDir": "src",
"testRegex": ".*\\.spec\\.ts$",
"transform": {
"^.+\\.(t|j)s$": "ts-jest"
},
"collectCoverageFrom": [
"**/*.(t|j)s"
],
"coverageDirectory": "../coverage",
"testEnvironment": "node",
+ "setupFiles": [
+ "<rootDir>/config/dotenv/unit-test-config.ts"
+ ]
}
Prepare a .env.test.local file for testing and call it as a setup file when executing Jest. Since simply loading with dotenv.config() does not automatically override existing environment variables, we explicitly override process.env using Object.assign() after loading the file.[1]
The example above is a configuration for unit tests. If you are performing E2E tests, you will likely need a separate set of environment variables specifically for E2E. Since Jest can load an arbitrary jest.config.js at runtime, you could use a dedicated setup for E2E tests, such as setupFiles: jest.e2e.json. With this, we have successfully set up environment variables for testing by leveraging Jest's built-in mechanisms.
5. Validating Environment Variables
Now we can load environment variables according to the situation. Next, we'll start using them extensively in the application. Wait... are you worried about whether the environment variables will be correctly set in the production environment? I understand. Well then, let's add a little extra step to the initialization of the Configuration Module to validate the environment variables.
import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
+import { validate } from './env-validator';
@Module({
imports: [ConfigModule.forRoot({
envFilePath: ['.env.development.local'],
+ validate,
})],
})
export class AppModule {}
It looks like something called validate has been added to ConfigModule.forRoot. The Configuration Module provides various validation methods, and this is one of them. It is a method where the developer defines a validate function and passes it. If validation fails, you can throw an exception, and the application will not start. Let's implement it right away.
import { IsEnum, IsNotEmpty, IsNumber, IsString } from 'class-validator';
import { plainToClass } from 'class-transformer';
import { validateSync } from 'class-validator';
enum NodeEnvEnum {
Development = 'development',
Production = 'production',
Test = 'test',
}
/**
* ①
* If there are environment variables you want to validate, list them here.
* If validation fails, the application will not start.
*/
export class EnvValidator {
@IsEnum(NodeEnvEnum)
NODE_ENV: NodeEnvEnum;
@IsNumber()
PORT = 3333; // Default values can also be specified
@IsNotEmpty()
@IsString()
DATABASE_URL: string;
}
/**
* ②
* @param config The Record<string, any> to be validated. In this case, it's the combination of .env.development.local and environment variables.
* @returns The validated Record<string, any>
*/
export function validate(config: Record<string, unknown>) {
const validatedConfig = plainToClass(EnvValidator, config, {
enableImplicitConversion: true,
});
const errors = validateSync(validatedConfig, {
skipMissingProperties: false,
});
if (errors.length > 0) {
throw new Error(errors.toString());
}
return validatedConfig;
}
① is a class definition to use the class-validator mechanism. By specifying decorators like @IsNumber() for each field, validation runs within plainToClass(). The reason for using class-validator is that it is one of the request validation methods provided by NestJS. By using the same mechanism for Config, we can align the context.
And ② is the validation function called by NestJS. Since it calls plainToClass(EnvValidator) here, validation is executed. If there are errors, throwing an exception prevents the application from starting with unintended environment variables. Let's try starting it by putting an invalid string in PORT.
PORT=88888invalid-port
> yarn start:dev
[10:07:33 AM] Starting compilation in watch mode...
[10:07:37 AM] Found 0 errors. Watching for file changes.
/backend/src/config/environments/env-validator.ts:47
throw new Error(errors.toString());
^
Error: An instance of EnvValidator has failed the validation:
- property PORT has failed the following constraints: isNumber
As intended, a validation error occurred and startup was interrupted.
From here, we'll talk about after passing validation. Environment variables basically come in as strings, but by returning validatedConfig, they are converted to the types defined in EnvValidator. In other words, PORT becomes usable as a number type in the application. However, please note that EnvValidator is strictly a class for validation and conversion; just because it passes this process doesn't mean you can call it like EnvValidator.DATABASE_URL.
import { ConfigService } from '@nestjs/config';
import { Query, Resolver } from '@nestjs/graphql';
@Resolver()
export class PostsResolver {
constructor(
private configService: ConfigService,
) {}
@Query(() => String)
hello(): string {
+ return this.configService.get<number>('PORT'); // 3333 (will be of type number)
}
}
6. Calling Environment Variables with Intuitive Functions like isProduction()
Having passed validation, the configuration accessible by the application is completely safe. You're at ease now, right? Wait... is it a hassle to check what kind of environment variables are set? You want to wrap them in convenient functions? Well, I agree. this.configService.get<number>('PORT'); is honestly a way of writing that I'd rather not see too frequently from a reader's perspective.
So, how about a strategy where we wrap the ConfigService provided by the Configuration Module and define convenient functions like isProduction() there? This is what's known as composition. This is the perfect moment to create a custom module. Just as you'd create a convenient interface for reading blog posts using a database client library, let's create a convenient interface for reading configurations. Let's assume we're building an application called Pb (Petabit)[2] and prepare a convenient service called PbEnv for access.
Create a module in src/config/environments. This path is just an example; it can be anywhere under src.
src/config/environments
├── env-validator.ts # Bring the validation file we just created here as well
├── pb-env.module.ts
├── pb-env.service.ts
└── pb-env.spec.ts
import { Injectable } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
/**
* Configuration values used in application modules are retrieved from here.
*/
@Injectable()
export class PbEnv {
constructor(private configService: ConfigService) {}
isProduction(): boolean {
return this.configService.get('NODE_ENV') === 'production';
}
get service() {
return this.configService;
}
get NodeEnv(): string {
return this.configService.get('NODE_ENV');
}
get Port(): number {
return this.configService.get('PORT');
}
get DbHost(): string {
return this.configService.get('DB_HOST');
}
}
We are hiding configService using getters and functions. If we can call this file from the application, it seems it will improve usability. Let's look at the module as well.
import { Global, Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
import { validate } from './env-validator';
import { PbEnv } from './pb-env.service';
@Global()
@Module({
imports: [
ConfigModule.forRoot({
envFilePath: ['.env.development.local'],
validate,
isGlobal: true,
}),
],
providers: [PbEnv],
exports: [PbEnv],
})
export class PbEnvModule {}
I moved the settings that were in app.module.ts to this file. Conversely, app.module.ts now simply imports PbEnvModule as follows:
import { Module } from '@nestjs/common';
import { PbEnvModule } from '@pb-config/environments/pb-env.module';
import { PbEnv } from '@pb-config/environments/pb-env.service';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { PostsModule } from './components/posts/posts.module';
import { PostsResolver } from './components/posts/posts.resolver';
@Module({
imports: [
+ PbEnvModule,
PostsModule,
],
controllers: [AppController],
providers: [AppService, PostsResolver],
})
export class AppModule {}
The module resolution when the application starts is as follows:
- NestJS attempts to resolve module dependencies at application startup.
- It discovers
PbEnvModule. - In
PbEnvModule,validateis specified, so it validates environment variables usingEnvValidator. - If validation is OK, the module instance is created.
-
PbEnvModuleexports thePbEnvservice, making this service available throughout the entire application.PbEnvis a wrapper forConfigServiceand serves as a place to define convenient functions for accessing environment variables.
PbEnv is used from the application as follows:
import { PbEnv } from '@pb-config/environments/pb-env.service';
import { ConfigService } from '@nestjs/config';
import { Query, Resolver } from '@nestjs/graphql';
import { PrismaService } from '@pb-components/prisma/prisma.service';
@Resolver()
export class PostsResolver {
constructor(
private prisma: PrismaService,
private configService: ConfigService,
private pbEnv: PbEnv,
) {}
@Query(() =0 String)
hello(): string {
return this.configService.get0number1('DATABASE_URL'); // Compared to this
}
+ @Query(() =0 String)
+ helloEnv(): string {
+ return this.pbEnv.DatabaseUrl; // It became much more intuitive. Seems likely to reduce mistakes.
+ }
}
It has become much more readable. It's defined in a GraphQL Query, and when executed, it looks like this:
Execution result. You can see that the database URL set in the environment variables is being retrieved.
Summary
In this article, I introduced practices for NestJS Configuration.
- Since NestJS naturally assumes the concurrent use of dotenv and environment variables, following that allows for simple configuration of environment variables for both development and execution environments.
- However, when running tests with Jest, explicitly override environment variables using
setupFiles:. - You can pass an arbitrary
validatefunction when initializing ConfigModule. By using it in combination withclass-validator, you can start the application safely while keeping the amount of code low. - By further wrapping ConfigService with another service, you can call environment variables more intuitively, such as with
isProduction().
Next: Library Initialization
Now that environment variables have become an injectable service, they can be called intuitively from various places. In the next article, I will try using this Config to initialize client libraries used throughout the application. The appeal of PbEnv is not only that it makes reading environment variables intuitive but also that it can consolidate environment variables. For example, in GraphQL initialization, you will pass several options, and there might be cases where you want to change the behavior based on NODE_ENV. You can write it in a getter like this:
get GqlModuleOptionsFactory(): GqlModuleOptions {
// Development: Generate schema from code and also use Playground.
// Since the backend code is the source of truth, use a code-first approach.
const devOptions: GqlModuleOptions = {
autoSchemaFile: path.join(
process.cwd(),
'src/generated/graphql/schema.gql',
),
sortSchema: true,
debug: true,
playground: true,
};
// Production environment: Execution only
const prdOptions: GqlModuleOptions = {
autoSchemaFile: true,
debug: false,
playground: false,
};
if (this.isProduction()) {
return prdOptions;
} else {
return devOptions;
}
}
Once defined, since it returns GqlModuleOptions, you can just pass it directly to the GraphQL Module and you're done. Config is the first hurdle, but once you overcome it successfully, application development accelerates even further. Please give it a try!
Source Code
Discussion