"No migrations pending" when attempting to run migrations , used to work with no problem - node.js

I have a web app, and I've written a migrator to create all my tables and relations, recently no matter what I try, typeorm does not appear to find this migrator and hence, does not run it.
My file structure (just the migrations)
src> Databas> Migrations>1663525805095-add_users.ts,1663529676790-make_institute_nullable.ts
ormconfig.ts
import { DataSource } from 'typeorm';
import { ConfigService } from '#nestjs/config';
import { config } from 'dotenv';
config();
const configService = new ConfigService();
const source = new DataSource({
type: 'postgres',
host: configService.get('POSTGRES_HOST'),
port: configService.get('POSTGRES_PORT'),
username: configService.get('POSTGRES_USER'),
password: configService.get('POSTGRES_PASSWORD'),
database: configService.get('POSTGRES_DB'),
synchronize: false,
logging: false,
migrations: ['src/database/migrations/*.ts'],
migrationsTableName: 'migrations',
entities: ['src/**/*.entity.ts'],
});
export default source;
In order to run this, I type yarn start:dev in order to get my Server started.
Then I run yarn migrations:run which I get:
query: SELECT * FROM current_schema()
query: SELECT version();
query: SELECT * FROM "information_schema"."tables" WHERE "table_schema" = 'public' AND "table_name" = 'migrations'
query: CREATE TABLE "migrations" ("id" SERIAL NOT NULL, "timestamp" bigint NOT NULL, "name" character varying NOT NULL, CONSTRAINT "PK_8c82d7f526340ab734260ea46be" PRIMARY KEY ("id"))
query: SELECT * FROM "migrations" "migrations" ORDER BY "id" DESC
No migrations are pending
When I look at my db, I see a migrations table with no entries.
I have tried to delete my migrator file and create it again with a more recent timestamp and that does not work either.
scripts from my package.json
"migrations:run": "yarn typeorm migration:run"
"typeorm": "typeorm-ts-node-commonjs -d ./ormconfig.ts"
"start:dev": "nest start --watch"
Other info
I'm using docker for the postgres DB and pgAdmin, it connects with no problem.
Any help would be greatly appreciated.

Related

Error when using docker-compose to generate a full-stack application

The API does not find the DB when using docker compose.
I already configured the DATABA_URL and it doesn't work.
.env file:
JWT_SECRET="palavrasecreta"
NODE_ENV=production
DATABASE_URL=postgres://postgres:1234#db:5432/postgres
docker-compose file:
version: "3"
services:
app-front-end:
build: charllenger-front/.
container_name: front-end-ui
expose:
- 3000
ports:
- 3000:3000
links:
- api
api:
container_name: charllenger-back-end-Api
build: Full-Stack-charlenger-API/.
volumes:
- ./src:/app/src
expose:
- 3001
ports:
- 3001:3001
depends_on:
- db
command: bash -c 'yarn migration:run && yarn dev'
links:
- db
db:
container_name: charllenger-Api-postgres
image: "postgres"
env_file:
- Full-Stack-charlenger-API/.env
expose:
- 5432
ports:
- 5432:5432
DATA SOURCE
import { DataSource } from "typeorm";
require("dotenv").config();
export const AppDataSource = new DataSource({
type: "postgres",
host: "database",
url: process.env.DATABASE_URL,
ssl:
process.env.NODE_ENV === "production"
? { rejectUnauthorized: false }
: false,
synchronize: false,
logging: true,
entities:
process.env.NODE_ENV === "production"
? ["src/entities/*.js"]
: ["src/entities/*.ts"],
migrations:
process.env.NODE_ENV === "production"
? ["src/migrations/*.js"]
: ["src/migrations/*.ts"],
});
AppDataSource.initialize()
.then(() => {
console.log("Data Source Initialized");
})
.catch((err) => {
console.error("Error during data source Initialization", err);
}
);
Migration
import { MigrationInterface, QueryRunner } from "typeorm";
export class initialMigration1673714934213 implements MigrationInterface {
name = 'initialMigration1673714934213'
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(`CREATE TABLE "transaction" ("transaction_id" uuid NOT NULL, "value" numeric NOT NULL, "createdAt" TIMESTAMP NOT NULL DEFAULT now(), "debitedAccountAccountId" uuid, "creditedAccountAccountId" uuid, CONSTRAINT "PK_6e02e5a0a6a7400e1c944d1e946" PRIMARY KEY ("transaction_id"))`);
await queryRunner.query(`CREATE TABLE "user" ("user_id" uuid NOT NULL, "username" character varying NOT NULL, "password" character varying NOT NULL, "account" uuid, CONSTRAINT "REL_4ab2df0a57a74fdf904e0e2708" UNIQUE ("account"), CONSTRAINT "PK_758b8ce7c18b9d347461b30228d" PRIMARY KEY ("user_id"))`);
await queryRunner.query(`CREATE TABLE "account" ("account_id" uuid NOT NULL, "balance" double precision NOT NULL, CONSTRAINT "PK_ea08b54a9d7322975ffc57fc612" PRIMARY KEY ("account_id"))`);
await queryRunner.query(`ALTER TABLE "transaction" ADD CONSTRAINT "FK_bbbfcdb3330cc4e5846f2d23200" FOREIGN KEY ("debitedAccountAccountId") REFERENCES "account"("account_id") ON DELETE NO ACTION ON UPDATE NO ACTION`);
await queryRunner.query(`ALTER TABLE "transaction" ADD CONSTRAINT "FK_4ece0117a7c2689832bab37209b" FOREIGN KEY ("creditedAccountAccountId") REFERENCES "account"("account_id") ON DELETE NO ACTION ON UPDATE NO ACTION`);
await queryRunner.query(`ALTER TABLE "user" ADD CONSTRAINT "FK_4ab2df0a57a74fdf904e0e27086" FOREIGN KEY ("account") REFERENCES "account"("account_id") ON DELETE NO ACTION ON UPDATE NO ACTION`);
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(`ALTER TABLE "user" DROP CONSTRAINT "FK_4ab2df0a57a74fdf904e0e27086"`);
await queryRunner.query(`ALTER TABLE "transaction" DROP CONSTRAINT "FK_4ece0117a7c2689832bab37209b"`);
await queryRunner.query(`ALTER TABLE "transaction" DROP CONSTRAINT "FK_bbbfcdb3330cc4e5846f2d23200"`);
await queryRunner.query(`DROP TABLE "account"`);
await queryRunner.query(`DROP TABLE "user"`);
await queryRunner.query(`DROP TABLE "transaction"`);
}
}
Error:
enter image description here
t's working on another computer, I've already changed localhost to db, I've tried other ports and it doesn't work.
In the docker-compose you define your database container as db.
db:
container_name: charllenger-Api-postgres
image: "postgres"
db: is the servicename.
If you want to connect from your api to the database container, inside your docker network, you need to use this servicename as db_host.
the section:
ports:
- 5432:5432
means: map the port 5432 inside the container to my host (your Pc where you installed docker).
So you can access the container on his port 5432, from your PC with localhost:5432 and this works fine if you execute your code localy even if postgres is running inside a container.
If your code runs in a container localhost refers to that container.
Lets say each container has it own localhost. So you can access the other containers only by its servicename. There are a few other possibilities, but lets remain to use the servicename.
In side your Typeorm configuration ensure that the connection to the database use the servicename when you run the code inside the conatiner, or localhost if you run it directly on your pc without docker.
When you deploy it to Production, again you must look where the db_host is. That depends on your deployment. If you use an external DB from a provvider it should be a full qualified domain name. If you deploy your db by yourself it depends if you use docker Kubernetes or what else.

"Cannot use import statement outside a module" in typeorm migration when run nestjs app

I have created the nestjs app. In the root app folder I have these subfolders:
dist
migration
src
test
The migration folder contains typeorm migrations.
When run application with npm run start:dev I have this error:
import {MigrationInterface, QueryRunner} from "typeorm";
^^^^^^
SyntaxError: Cannot use import statement outside a module
at Module._compile (internal/modules/cjs/loader.js:891:18)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:991:10)
at Module.load (internal/modules/cjs/loader.js:811:32)
at Function.Module._load (internal/modules/cjs/loader.js:723:14)
at Module.require (internal/modules/cjs/loader.js:848:19)
at require (internal/modules/cjs/helpers.js:74:18)
at Function.PlatformTools.load (C:\Users\dakru1\Documents\employo\employo-api\node_modules\typeorm\platform\PlatformTools.js:114:28)
at C:\Users\dakru1\Documents\employo\employo-api\node_modules\typeorm\util\DirectoryExportedClassesLoader.js:39:69
at Array.map (<anonymous>)
at Object.importClassesFromDirectories (C:\Users\dakru1\Documents\employo\employo-api\node_modules\typeorm\util\DirectoryExportedClassesLoader.js:39:10)
I understand the error message and I know how to fix it when it relates to application's code.
However, my problem is that this error come from typeorm migration file: [app-root-folder]\migration\1587067680466-Init.ts which shouldn't be used when application runs.
Why nestjs uses migration files. How can I ignore migration folder when running nestjs app?
to solve it just put the following code on your "scripts" at package.json:
"typeorm": "ts-node-dev ./node_modules/typeorm/cli.js",
After that you'll be able to run your typeorm migration:run :)
I found the solution:
This happends to me because I was using Nest and when you run the command nest start, under the hood, Nest executes node and tsc and node commands and because of that you need your files to be in JavaScript format. So, as the migrations are generated in Typescript... the statement import gives the error because is only for TypeScript.
The only thing i did to solve it is this:
migrations: ["dist/migrations/*.js"],
This is telling TypeOrm that search for the migration in the dist directory where all the JavaScript code was compiled previously.
Of course in my tsconfig.json file the output dir is pointing to dist.
"outDir": "./dist",
I had the same issue. I did the following as a workaround:
Set your ormconfig.json to look for migrations files with only js suffix:
"migrations": ["migrations/*.js"],
Install typescript globally: npm install -g typescript
After generating your migration, transpile typescript files to javascript files with typescript command:
tsc migrations/migration-file.ts
Run your migration: npm run typeorm migration:run
You can now run your application: npm run start:dev
I know this question is old, but I just came across it because I had the same problem. I was able to solve it by adding "migration/**/*.ts" to the "exclude" array in the /tsconfig.build.json file like so:
{
"extends": "./tsconfig.json",
"exclude": ["node_modules", "test", "dist", "**/*spec.ts", "migration/**/*.ts"]
}
The same issue here, and in my case the root cause was a wrong migrations path in the connection configuration, like this;
const databaseConfig: ConnectionOptions = {
name: 'default',
type: 'postgres',
host: process.env.TYPEORM_HOST || '',
port: Number(process.env.TYPEORM_PORT),
username: process.env.TYPEORM_USERNAME || '',
password: process.env.TYPEORM_PASSWORD || '',
database: process.env.TYPEORM_DATABASE || '',
logging: false,
synchronize: false,
entities: [],
migrations: ['src/migrations/*.ts'],
};
Instead, I have changed it to
const migrationPaths: string[] = process.env.TYPEORM_USE_CLI === 'true' ? ['src/migrations/*{.ts,.js}'] : [];
const databaseConfig: ConnectionOptions = {
name: 'default',
type: 'postgres',
host: process.env.TYPEORM_HOST || '',
port: Number(process.env.TYPEORM_PORT),
username: process.env.TYPEORM_USERNAME || '',
password: process.env.TYPEORM_PASSWORD || '',
database: process.env.TYPEORM_DATABASE || '',
logging: false,
synchronize: false,
entities: [],
migrations: migrationPaths,
};
another test you can try to see if the problem is really with migration is to move the migration filter out of src.
For TypeORM v0.3.0 or higher, glob patterns in migrations are DEPRECATED. It is recommended to directly include the migration classes.
entities, migrations, subscribers options inside DataSourceOptions accepting string directories support is deprecated. You'll be only able to pass entity references in the future versions.
import { DataSource, DataSourceOptions } from 'typeorm';
import { CreateUser1669370712331 } from '../../migrations/1669370712331-CreateUser';
import { UserModify1669457039074 } from '../../migrations/1669457039074-UserModify';
import { User } from '../users/entities/user.entity';
export const config: DataSourceOptions = {
type: 'sqlite',
database: 'db/database.sqlite',
logging: true,
synchronize: false,
entities: [User],
migrations: [CreateUser1669370712331, UserModify1669457039074],
};

TypeORM Entity in NESTJS - Cannot use import statement outside a module

Started new project with 'nest new' command. Works fine until I add entity file to it.
Got following error:
import { Entity, Column, PrimaryGeneratedColumn } from 'typeorm';
^^^^^^
SyntaxError: Cannot use import statement outside a module
What do I miss?
Adding Entity to Module:
import { Module } from '#nestjs/common';
import { BooksController } from './books.controller';
import { BooksService } from './books.service';
import { BookEntity } from './book.entity';
import { TypeOrmModule } from '#nestjs/typeorm';
#Module({
imports: [TypeOrmModule.forFeature([BookEntity])],
controllers: [BooksController],
providers: [BooksService],
})
export class BooksModule {}
app.module.ts:
import { Module } from '#nestjs/common';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { TypeOrmModule } from '#nestjs/typeorm';
import { Connection } from 'typeorm';
import { BooksModule } from './books/books.module';
#Module({
imports: [TypeOrmModule.forRoot()],
controllers: [AppController],
providers: [AppService],
})
export class AppModule {}
My assumption is that you have a TypeormModule configuration with an entities property that looks like this:
entities: ['src/**/*.entity.{ts,js}']
or like
entities: ['../**/*.entity.{ts,js}']
The error you are getting is because you are attempting to import a ts file in a js context. So long as you aren't using webpack you can use this instead so that you get the correct files
entities: [join(__dirname, '**', '*.entity.{ts,js}')]
where join is imported from the path module. Now __dirname will resolve to src or dist and then find the expected ts or js file respectively. let me know if there is still an issue going on.
EDIT 1/10/2020
The above assumes the configuration is done is a javascript compatible file (.js or in the TypeormModule.forRoot() passed parameters). If you are using an ormconfig.json instead, you should use
entities: ["dist/**/*.entity.js"]
so that you are using the compiled js files and have no chance to use the ts files in your code.
In the TypeORM documentation, i found a specific section for Typescript.
This section says:
Install ts-node globally:
npm install -g ts-node
Add typeorm command under scripts section in package.json
"scripts" {
...
"typeorm": "ts-node -r tsconfig-paths/register ./node_modules/typeorm/cli.js"
}
Then you may run the command like this:
npm run typeorm migration:run
If you need to pass parameter with dash to npm script, you will need
to add them after --. For example, if you need to generate, the
command is like this:
npm run typeorm migration:generate -- -n migrationNameHere
This works with my file config:
{
"type": "postgres",
"host": "yourhost",
"port": 5423,
"username": "username",
"password": "password",
"database": "your_db",
"synchronize": true,
"entities": [
"src/modules/**/*.entity.{ts,js}"
],
"migrations": [
"src/migrations/**/*.{ts,js}"
],
"cli": {
"entitiesDir": "src/modules",
"migrationsDir": "src/migrations"
}
}
Then you can run the generate command.
As Jay McDoniel explained in his answer, the problem seems to be the pattern matching of entity files in ormconfig.json file: Probably a typescript file (module) is imported from a javascript file (presumably a previously transpiled typescript file).
It should be sufficient to remove an existing ts glob pattern in the ormconfig.json, so that TypeORM will only load javascript files. The path to the entity files should be relative to the working directory where node is executed.
"entities" : [
"dist/entity/**/*.js"
],
"migrations" : [
"dist/migration/**/*.js"
],
"subscribers": [
"dist/subscriber/**/*.js"
],
I changed in tsconfig.json file next:
"module": "es6"
To:
"module": "commonjs",
It helps me
Defining the entities property in ormconfig.json as mentioned in the official documentation resolved this issue for me.
// This is your ormconfig.json file
...
"entities": ["dist/**/*.entity{.ts,.js}"]
...
This is how I've manage to fix it. With a single configuration file I can run the migrations on application boostrap or using TypeOrm's CLI.
src/config/ormconfig.ts
import parseBoolean from '#eturino/ts-parse-boolean';
import { TypeOrmModuleOptions } from '#nestjs/typeorm';
import * as dotenv from 'dotenv';
import { join } from 'path';
dotenv.config();
export = [
{
//name: 'default',
type: 'mssql',
host: process.env.DEFAULT_DB_HOST,
username: process.env.DEFAULT_DB_USERNAME,
password: process.env.DEFAULT_DB_PASSWORD,
database: process.env.DEFAULT_DB_NAME,
options: {
instanceName: process.env.DEFAULT_DB_INSTANCE,
enableArithAbort: false,
},
logging: parseBoolean(process.env.DEFAULT_DB_LOGGING),
dropSchema: false,
synchronize: false,
migrationsRun: parseBoolean(process.env.DEFAULT_DB_RUN_MIGRATIONS),
migrations: [join(__dirname, '..', 'model/migration/*.{ts,js}')],
cli: {
migrationsDir: 'src/model/migration',
},
entities: [
join(__dirname, '..', 'model/entity/default/**/*.entity.{ts,js}'),
],
} as TypeOrmModuleOptions,
{
name: 'other',
type: 'mssql',
host: process.env.OTHER_DB_HOST,
username: process.env.OTHER_DB_USERNAME,
password: process.env.OTHER_DB_PASSWORD,
database: process.env.OTHER_DB_NAME,
options: {
instanceName: process.env.OTHER_DB_INSTANCE,
enableArithAbort: false,
},
logging: parseBoolean(process.env.OTHER_DB_LOGGING),
dropSchema: false,
synchronize: false,
migrationsRun: false,
entities: [],
} as TypeOrmModuleOptions,
];
src/app.module.ts
import configuration from '#config/configuration';
import validationSchema from '#config/validation';
import { Module } from '#nestjs/common';
import { ConfigModule } from '#nestjs/config';
import { TypeOrmModule } from '#nestjs/typeorm';
import { LoggerService } from '#shared/logger/logger.service';
import { UsersModule } from '#user/user.module';
import { AppController } from './app.controller';
import ormconfig = require('./config/ormconfig'); //path mapping doesn't work here
#Module({
imports: [
ConfigModule.forRoot({
cache: true,
isGlobal: true,
validationSchema: validationSchema,
load: [configuration],
}),
TypeOrmModule.forRoot(ormconfig[0]), //default
TypeOrmModule.forRoot(ormconfig[1]), //other db
LoggerService,
UsersModule,
],
controllers: [AppController],
})
export class AppModule {}
package.json
"scripts": {
...
"typeorm": "ts-node -r tsconfig-paths/register ./node_modules/typeorm/cli.js --config ./src/config/ormconfig.ts",
"typeorm:migration:generate": "npm run typeorm -- migration:generate -n",
"typeorm:migration:run": "npm run typeorm -- migration:run"
},
Project structure
src/
├── app.controller.ts
├── app.module.ts
├── config
│ ├── configuration.ts
│ ├── ormconfig.ts
│ └── validation.ts
├── main.ts
├── model
│ ├── entity
│ ├── migration
│ └── repository
├── route
│ └── user
└── shared
└── logger
I was using Node.js with Typescript and TypeORM when I faced this issue. Configuring in ormconfig.json file worked for me.
entities: ['dist/**/*.entity.js']
My full code of ormconfig.json file:
{
"type": "mysql",
"host": "localhost",
"port": 3306,
"username": "xxxxxxxx",
"password": "xxxxxxxx",
"database": "typescript_orm",
"synchronize": true,
"logging": false,
"migrationTableName": "migrations",
"entities": [
"dist/**/*.entity.js"
],
"migrations": [
"src/migration/**/*.{ts, js}"
],
"suscribers": [
"src/suscriber/**/*.{ts, js}"
],
"cli": {
"entitiesDir": "src/model",
"migrationDir": "src/migration",
"suscribersDir": "src/suscriber"
}
}
Also check out your imports in the entities. Don't import { SomeClassFromTypeorm } from 'typeorm/browser'; since this can lead to the same error.
It happened to me after my IDE automatically imported the wrong package. Delete '/browser' from the import.
In line with other people's comments - it does in fact seem silly to have to depend on generated code for this to work. I do not take credit for this solution as it's someone else's repository, but it does in fact allow full Typescript only migrations. It relies on the .env file Typeorm values instead of ormconfig.json although I'm sure it could be translated. I found it instrumental in helping me remove the dependency on .js files.
Here is the repo:
https://github.com/mthomps4/next-now-test/tree/next-typeorm-example
Explanation as to how it's working:
Aside from your usual .env or ormconfig.json file with the proper localhost db connection in it, you also need to specify the following properly in ormconfig.json or .env file
TYPEORM_ENTITIES="entities/*.ts"
TYPEORM_MIGRATIONS="migrations/*.ts"
TYPEORM_ENTITIES_DIR="entities"
TYPEORM_MIGRATIONS_DIR="migrations"
Notice the entities and migrations globs only have *.ts. The other very important piece is how your npm scripts are setup to run with ts-node.
You need an extended tsconfig that has the following in it somewhere:
{
"extends": "./tsconfig.json",
"compilerOptions": {
"module": "commonjs"
}
}
This is what allows ts-node to "pick up" the .ts files properly while generating a migration.
This npm script (the DOTENV part is only if using .env files instead of ormconfig.json) specifies to use that tsconfig.json
"local": "DOTENV_CONFIG_PATH=./.env ts-node -P ./tsconfig.yarn.json -r dotenv/config"
Which is leveraged as a "pre-cursor" script to this:
"typeorm:local": "yarn local ./node_modules/typeorm/cli.js"
I'm not 100% sure all of that is necessary (you may could do it all inline) but it works for me. Basically this says "invoke the typrorm cli in the context of ts-node with a specific .env file and a specific tsconfig." You may be able to skip those configurations in some cases.
Lastly, this script now works:
"g:migration": "yarn typeorm:local migration:generate -n"
So by running:
npm run g:migration -- User
You will get your automatically generated migration file based on your current changed entities!
So 3 nested npm scripts later, we have a very specific way to run the "generate" migration conmmand with all the proper configuration to use only TS files. Yay - no wonder some people still rail against typescript but thankfully this does work and the example repo above has it all preconfigured if you want to try it out to see how it "just works".
Actually, typeorm was designed to work with javascript by default.
To run the migrations with typescript, you must tell typeorm to do it.
Just put in your package.json, in the scripts part this line below:
"typeorm": "ts-node-dev ./node_modules/typeorm/cli.js"
and then, try to migrate again:
yarn typeorm migration:run
I ran into this error trying to run typeorm migration:generate from a project created with the TypeORM starter kit (npx typeorm init). The issue came down to this bit that it inserted into package.json:
"scripts": {
"typeorm": "typeorm-ts-node-commonjs"
}
Change that to:
"scripts": {
"typeorm": "typeorm-ts-node-esm"
}
And you should be good to go:
npm run -- typeorm migration:generate --dataSource path/to/data-source.ts NameOfMigration
Surprised about these almost kinda hacky solutions, particularely at the accepted one...
You should never import anything from a dist folder inside your ts source code!
If the answered assumption is true, and you do this:
entities: ['src/**/*.entity.{ts,js}']
then, why don't you rather DO THIS:
import { Answer } from './entities/answer/answer.entity';
entities: [Answer]
This way you would you use your ts code (correctly) and the builded js code would get provided to the TypeOrmModule in runtime.
check your TypeOrmModule's entities
TypeOrmModule.forRoot({
type: 'postgres',
host: 'localhost',
port: 5432,
username: 'postgres',
password: '#GoHomeGota',
database: 'quiz',
**entities: ["dist/**/*.entity{.ts,.js}"],**
synchronize: true,
}),
The alternative I found for this is having two orm config files
namely orm-config.ts and cli-orm-config.ts (You can name them whatever)
//content of cli-orm-config.ts
import { DataSource, DataSourceOptions } from "typeorm"
import 'dotenv/config'
export const cliOrmConfig: DataSourceOptions = {
type: 'postgres',
host: process.env.DATABASE_HOST,
port: (process.env.PG_DATABASE_PORT as any) as number,
username: process.env.PG_DATABASE_USER,
password: process.env.PG_DATABASE_PASSWORD,
database: process.env.DATABASE_NAME,
entities: ["src/**/*/*.entity{.ts,.js}"],
migrations: ["src/**/*/*-Migration{.ts,.js}"]
}
const datasource = new DataSource(cliOrmConfig)
export default datasource
//content of orm-config.ts, this is the one I use in nest TypeOrmModule.forRoot(ormConfig)
import { DataSource, DataSourceOptions } from 'typeorm';
import 'dotenv/config'
export const ormConfig: DataSourceOptions = {
type: 'postgres',
host: process.env.DATABASE_HOST,
port: (process.env.PG_DATABASE_PORT as any) as number,
username: process.env.PG_DATABASE_USER,
password: process.env.PG_DATABASE_PASSWORD,
database: process.env.DATABASE_NAME,
entities: ["dist/src/**/*/*.entity{.ts,.js}"]
}
const datasource = new DataSource(ormConfig)
export default datasource
// My package.json relevant scripts section
"typeorm": "ts-node ./node_modules/typeorm/cli -d ./src/db/cli-orm-config.ts",
"nest:migration:generate": "npm run typeorm migration:generate ./src/db/migrations/Migration",
"nest:migration:run": "npm run typeorm migration:run"
I think as far as TypeOrm is concerned, the migration, cli parts should be teared apart from models loading and other stuffs; hence the seperation of the orm configs file for both.
Hope it helps somebody
You need to have a something.module.ts for every section of your app. It works like Angular. This is setup with GraphQL resolvers and service. REST is a bit different with a controller. Each module will probably have an entity and if GraphQL, projects.schema.graphql.
projects.module.ts
import { Module } from '#nestjs/common';
import { TypeOrmModule } from '#nestjs/typeorm';
import { ProjectsService } from './projects.service';
import { Projects } from './projects.entity';
import { ProjectsResolvers } from './projects.resolvers';
#Module({
imports: [
TypeOrmModule.forFeature([Projects])],
providers: [
ProjectsService,
ProjectsResolvers
],
})
export class ProjectsModule {}
This worked for me - no changes needed to your ormconfig.js. Run from your root directory where the node_modules are:
ts-node ./node_modules/typeorm/cli.js migration:generate -n <MirgrationName> -c <ConnectionType>
Example:
ts-node ./node_modules/typeorm/cli.js migration:create -n AuthorHasMultipleBooks -c development
Configuration to support migrations:
// FILE: src/config/ormconfig.ts
const connectionOptions: ConnectionOptions = {
// Other configs here
// My ormconfig isn't in root folder
entities: [`${__dirname}/../**/*.entity.{ts,js}`],
synchronize: false,
dropSchema: false,
migrationsRun: false,
migrations: [getMigrationDirectory()],
cli: {
migrationsDir: 'src/migrations',
}
}
function getMigrationDirectory() {
const directory = process.env.NODE_ENV === 'migration' ? 'src' : `${__dirname}`;
return `${directory}/migrations/**/*{.ts,.js}`;
}
export = connectionOptions;
// FILE package.json
{
// Other configs here
"scripts": {
"typeorm": "NODE_ENV=migration ts-node -r tsconfig-paths/register ./node_modules/typeorm/cli.js --config src/config/database.ts",
"typeorm:migrate": "npm run typeorm migration:generate -- -n",
"typeorm:run": "npm run typeorm migration:run",
"typeorm:revert": "npm run typeorm migration:revert"
}
}
I think a better solution, than the accepted one, is to create a alias in your shell of choice, that uses ts-node inside node_modules.
Note: I'm doing this in bash, with OhMyZsh, so your configuration might be totally different.
1: Open shell configuration
Open shell configuration1
nano ~/.zshrc
2: Find the place where other aliases are defined and add a new alias
alias typeorm="ts-node ./node_modules/typeorm/cli.js"
3: Close and save
Press CTRL + X to request nano to exit and press Y to confirm to save the configuration.
4: Apply the new configuration
. ~/.zshrc
5: Close terminal and open it again
You can now go to your project root and type "typeorm" which will use ts-node in conjunction with the typeorm-cli from your node_modules.
I used this solution only for production.
for development I change "../src/entity/**/*.ts" to "src/entity/**/*.ts" and then run this command: "nodemon --exec ts-node ./src/index.ts" and it works –
I solved the problem!
Create pm2.config.js file in root with below codes:
module.exports = {
apps: [
{
name: "app",
script: "./build/index.js",
},
],
};
Change entity path in ormconfig.js
{
"type": "postgres",
"host": "localhost",
"port": 5432,
"username": "postgres",
"password": "password",
"database": "db_name",
"synchronize": false,
"logging": true,
"entities": [
"../src/entity/**/*.ts", ===>>> this line is important
"./build/entity/**/*.js"
],
"migrations": [
"../src/migration/**/*.ts",===>>> this line is important
"./build/migration/**/*.js"
],
"subscribers": [
"../src/subscriber/**/*.ts",===>>> this line is important
"./build/subscriber/**/*.js"
],
"cli": {
"entitiesDir": "src/entity",
"migrationsDir": "src/migration",
"subscribersDir": "src/subscriber"
}
}
tsconfig.json with below code:
{
"compilerOptions": {
"lib": [
"es5",
"es6"
],
"target": "es5",
"module": "commonjs",
"moduleResolution": "node",
"outDir": "./build",
"emitDecoratorMetadata": true,
"experimentalDecorators": true,
"sourceMap": true,
"esModuleInterop": true
}
}
Run below command for production:
tsc =>> This command generate "build" folder
Run below command for run node app in pm2:
tsc && pm2 start pm2.config.js
Now after 2 days with this solution my app with node express & typeorm is worked!
Also my app are working on linux & nginx with pm2.
For me, changing module in my tsconfig.ts
from
"module": "esnext"
To:
"module": "commonjs",
Did the job.
If you are writing in typescript and use tsc to create a dist folder with translated js files in it, then you probably have my issue and it will get fixed here.
As it is mentioned here in the docs if you use nodemon server.js, then you will hit the entities from js perspective and it will not recognize import as it is ts and es6 related. However if you want to import entities from ts files, you should run ts-node server.ts!
Personally I believe the former node server.js is a safer one to do as it is closer to the real case application.
!!! HOWEVER !!! Be very careful as you have to delete the dist folder and rebuild it if you change an entity's name, otherwise it will throw an error or work unexpectedly.
The error happens because the tsc will try to translate the changed and created ts files and leave the deleted files so it can run faster!
I hope it helped as it will definitely help me in the future as I am almost certain I will forget about it again!
I have encountered the same problem. The only difference is that my project uses .env file instead of ormconfig.json
This is what my .env file configuration looks like.
TYPEORM_ENTITIES = src/modules/*.entity.ts
TYPEORM_MIGRATIONS = src/migrations/*.entity.ts
TYPEORM_MIGRATIONS_RUN = src/migrations
TYPEORM_ENTITIES_DIR = src/modules
TYPEORM_MIGRATIONS_DIR = src/migrations
And run by using command
nest start
The problem appears to be that TypeORM does not accept entities in the form of typescript files.
There are two approaches that can be used to solve this problem.
Use node-ts instead of nest start solved the problem without modifying the path of the entities file. From my understanding, node-ts will process the typescript file in the src folder without issue.
Change the entity and migration file paths to point to the compiled js file in the dist folder instead.
TYPEORM_ENTITIES = dist/modules/*.entity.js
TYPEORM_MIGRATIONS = dist/migrations/*.entity.js
TYPEORM_MIGRATIONS_RUN = dist/migrations
TYPEORM_ENTITIES_DIR = dist/modules
TYPEORM_MIGRATIONS_DIR = dist/migrations
with this approach, I can use nest start without any problem.
The accepted answer here (https://stackoverflow.com/a/59607836/2040160) was help me generate and run the migrations, but not to run the NestJS project. I got the same error as the author when I npm run start:dev.
What worked for me, is to just generate the migrations file in vanilla JavaScript.
My ormconfig,json file:
{
"type": "cockroachdb",
"host": "localhost",
"port": 26257,
"username": "root",
"password": "",
"database": "test",
"entities": ["dist/**/*.entity{.ts,.js}"],
"migrations": ["migration/*.js"],
"synchronize": false,
"cli": {
"migrationsDir": "migration"
}
}
The script in package.json:
"typeorm": "node --require ts-node/register ./node_modules/typeorm/cli.js"
And the command I use to generate the migrations:
npm run typeorm migration:generate -- -o -n init
The -o flag will output the migrations in vanilla JavaScript.
The error is on your ormconfig.json file. check where is your code searching for the entities, migrations, subscribers. In a dev, test environment it will search for them in your src/entities src/migrations src/subscribers. But in a production environment, if you leave it as it is, it will still search in the same path instead of your build path dist/src/entities etc.... ;)
I spent so much time in this mini compilation hell :)
Just use the autoLoadEntities option in https://docs.nestjs.com/techniques/database
v useful!!
npm run typeorm migration:generate -- -n translationLength

"ERROR: Validation error" message when executing two Sequelize commands in "pretest" script

I'm writing tests for my project. It uses Sequelize and I thought about doing the following:
"pretest": "NODE_ENV=testing yarn sequelize db:migrate && yarn sequelize db:seed:all",
"test": "mocha --require #babel/register 'src/tests/**/*.spec.js'",
"posttest": "NODE_ENV=testing yarn sequelize db:migrate:undo:all"
But the following shows:
❯ yarn test
yarn run v1.19.2
$ NODE_ENV=testing yarn sequelize db:migrate && yarn sequelize db:seed:all
$ /home/gabriel/Workspace/graphql-apollo/node_modules/.bin/sequelize db:migrate
Sequelize CLI [Node: 12.13.1, CLI: 5.5.1, ORM: 5.21.2]
Loaded configuration file "src/config/database.js".
== 20191123132531-create-users: migrating =======
== 20191123132531-create-users: migrated (0.047s)
== 20191123132658-create-messages: migrating =======
== 20191123132658-create-messages: migrated (0.028s)
$ /home/gabriel/Workspace/graphql-apollo/node_modules/.bin/sequelize db:seed:all
Sequelize CLI [Node: 12.13.1, CLI: 5.5.1, ORM: 5.21.2]
Loaded configuration file "src/config/database.js".
== 20191123132945-users: migrating =======
ERROR: Validation error
error Command failed with exit code 1.
If I execute the migration and seeding command separately, it works fine. I also tried to use concurrently, but the same happens.
"pretest": "concurrently 'NODE_ENV=testing yarn sequelize db:migrate' 'yarn sequelize db:seed:all'",
#gamofe Add the --debug flag to the command to see more info about the error. e.g.
$ sequelize db:seed:all --debug
It's very likely you are getting this error because you are running sequelize db:seed:all without undoing previous seed. If your table already contains the data you are trying to seed in which some of them have unique constraint, you'll hit the unique constraint error.
To solve this you would need to run the sequelize db:seed:undo:all before seeding.
You can find more info here https://sequelize.org/master/manual/migrations.html
I had the same problem and managed to solve it as follows:
Wrong: "pretest": "NODE_ENV=test sequelize db:migrate && sequelize db:seed:all"
Right: "pretest": "NODE_ENV=test sequelize db:migrate && NODE_ENV=test sequelize db:seed:all"
Check your queryInterface.bulkInsert() or other insert statements are passed created_at and updated_at values. The seed part of sequelize does not automatically add this in - and the database has a NOT NULL restriction on these fields.
In my case I had to change the content of my seed file to include the createdAt and updatedAt fields.
I changed:
module.exports = {
up: async (queryInterface, Sequelize) => {
await queryInterface.bulkInsert('Users', [
{ id:1, name: `God`, type: UserTypes.Person },
]);
}
}
to:
module.exports = {
up: async (queryInterface, Sequelize) => {
const currentTime = new Date(new Date().toUTCString()).toISOString();
await queryInterface.bulkInsert('Users', [
{ id:++i, name: `God`, type: UserTypes.Person, createdAt: currentTime, updatedAt: currentTime },
]);
}
}
Note: I use the underscored: true option. If you don't you will need to use created_at and updated_at in place of createdAt and updatedAt.

Run Sequelize migration after deployment to App Engine

I have create a sample project for myself to try Node.js with Sequelize on Google App Engine.
I can run the project locally and it works fine, however when I deploy it to App Engine gcloud app deploy
I get the following error:
{
name: "SequelizeDatabaseError",
message: "ER_NO_SUCH_TABLE: Table 'sql8175762.Likes' doesn't exist",
parent: {
code: "ER_NO_SUCH_TABLE",
errno: 1146,
sqlState: "42S02",
index: 0,
sql: "SELECT `id`, `code`, `likes`, `createdAt`, `updatedAt` FROM `Likes` AS `Likes`;"
},
original: {
code: "ER_NO_SUCH_TABLE",
errno: 1146,
sqlState: "42S02",
index: 0,
sql: "SELECT `id`, `code`, `likes`, `createdAt`, `updatedAt` FROM `Likes` AS `Likes`;"
},
sql: "SELECT `id`, `code`, `likes`, `createdAt`, `updatedAt` FROM `Likes` AS `Likes`;"
}
Thats because the Sequelize migration doesnt run however I do have it specified in npm start command:
"scripts": {
"deploy": "gcloud app deploy",
"start": "npm install & sequelize db:migrate & NODE_ENV=production node ./bin/www"
},
I have never used app engine for node deployment and I am not even sure if the steps I am talking are correct to deploy, migrate and run the app.
Does anyone have any tips regarding this?
I had the same experience, the problem is that for some reasons you have to add tableName and freezeTableName: true property in the defining model. for instance:
'use strict';
module.exports = (sequelize, DataTypes) => {
const File = sequelize.define('File', {
name: DataTypes.STRING,
courseId: DataTypes.INTEGER
}, {
freezeTableName: true,
tableName: 'files'
});
File.associate = function(models) {
// associations can be defined here
};
return File;
};
Im a little late to this question, but in case others are looking, it's worth noting that the example script shown uses & between the commands.
This will background each command and immediately run the next, so there's a good chance that both npm install and sequelize db:migrate are still running when node launches.
If you're on a unix system (Mac/Linux) You can test this locally by creating a file called ./test.sh containing
#!/bin/sh
echo "Starting long process ..."
sleep 5
echo "Long process done"
and then from the command line running
chmod +x test.sh
./test.sh & echo "Running second process"
Changing the commands to && should fix this, ie:
npm install && sequelize db:migrate && NODE_ENV=production node ./bin/www
I don't know your setup, but do you have sequelize-cli package dependency, also have you tried adding a .sequelizerc file in your root directory and add the paths for your config, migration and models? Example:
var path = require('path');
module.exports = {
'config': path.resolve('./', 'config/config.js'),
'migrations-path': path.resolve('./', 'src/server/migrations'),
'seeders-path': path.resolve('./', 'src/server/seeders'),
'models-path': path.resolve('./', 'src/server/models')
};
you should use && instead of & in your start command.
& means to run the command at background.
&& means "logic and" so that the shell will run the command one by one and wait for each returned code.

Resources