We're using Contentful CMS, and are interested in programmatically migrating our schema. Contentful has a migration guide available here - https://www.contentful.com/developers/docs/tutorials/cli/scripting-migrations/
The guide describes how to write migration scripts and run them against Contentful to alter its schema. However, it doesn't provide guidance around how the scripts should be run. After a migration is executed, it seems like you have to manually remember to never run it again (if executed again it will produce errors). This makes automating the schema changes difficult. Are there best practices that exist around this?
Heyooo, Contentful DevRel here.
We have a blog post about that.
What it describes is that you could set up a content type that holds the version of the last ran migration. After you ran this migration you could then update then entry and apply some logic.
When you're running bigger migrations you could also have a look at environment aliases which allow you to promote environments to master after a successful migration. This gives you much flexibility and safety.
Hope that helps. :)
Related
I want to create a workflow automation where an activity comes in and user can setup a multilevel workflow.
For frontend i am using https://reactflow.dev
How to structure things in nodejs backend. Things like database, accessing control flow statements, statements which requires crons.
You also may want to have a look at node-red.
It's an open-source product that does exactly that.
There's a set of built-in nodes.
You can develop your own nodes, or import 3rd party ones. Which are stored in NPM.
You can also just create a node with javascript or typescript code in it, on the fly.
You should check Flumejs: https://flume.dev/
https://flume.dev/docs/quick-start/
Also you should see this code sandbox example. Try to read the code
and all the dependencies: https://codesandbox.io/s/node-based-code-generation-test-forked-ll9flz?file=/src/App.tsx
I hope you find this helpful.
Apologies in advance as I am unable to find a good solution regarding this.
Background: We are using Typeorm in express app and have too many migrations because its a little old project.
Question: Is there a good way to manage migrations? Ideally I would want to delete all migrations(generated till now) and have only single file for schema creation(updated automatically after every migration run) and another file for seeding master data. Using these two files any new developer can setup the database locally in no time.
I worked in Ruby on rails and there used to be a clean approach like ruby maintains the database schema all the time in schema.rb and database could be generated using that.
I researched a lot but couldn't find a maintainable solution. May be the experts here could help me.
Thanks in advance
All your schemas describe your data model at current time, you can just delete all migrations from code and database, it will be as if you had no migrations at all. The problem will be only with old versions of your database which needs to be migrated.
Is there any tool that works similar to Django South, but for Node?
Now I'm working with Sequelize. If I got it right, Sequelize does not have an option to create migration files based on existing models.
So, to create a new model/table, the steps are:
Create model with sequelize model:create <model meta>.
Edit generated migration file - add actual code for creating tables
in DB under up section.
Run migration with sequelize db:migrate.
I'm looking for something that can create migration files based on existing models, manage it similar to what South can do for Django.
Is there any option?
I have written a step-by-step guide on how to auto-create migrations with Sequelize in another post. Here is a summary...
The closest thing with Sequelize is
Sequelize Auto Migrations.
It allows you to have an iteration cycle like the following:
Create/update model -- by hand or with sequelize-cli)
Run makemigrations to auto-generate up and down migrations
Repeat as necessary
While this is very helpful, I've found it to be lacking in some critical areas:
The down migrations can be created incorrectly. So it may try to drop a table before its dependent tables have been dropped.
There are certain configurations around multi-field indexes that it has not correctly output.
There are currently 10 outstanding PRs, so it seems like a few additional contributors are attempting to make it more production-ready... but I've yet to find anything as clean and reliable as Django Migrations (formerly Django South).
TypeORM supports model based migrations. It can sync your db to your models directly, it can also create migration files too.
I think prisma is another option. It doesn't seem like that popular but it's a promising one.
Either way, it's just ridiculous that there are no solid tools for this. I've worked on django and .net projects in the last years and creating migrations is just easy with them. But when you try to use node.js for backend, you get stuck at a lot of points.
I was using sequelize and when I saw there was no official way to create automatic migrations from models, I've dropped using it. Maintaining your models with manually written migrations becomes very hard in my experience.
Now my only alternative is TypeORM and it just bugs me there is no other alternative in case TypeORM just goes unmaintained or if I want to use another library etc.
I'm seriously thinking dropping using node.js for backend. Yet, there are good tools to create projects integrated with modern front-end tools (like Next.js), finding a good orm is a big problem.
Take a look at https://typeorm.io/#/migrations/generating-migrations. I am in the same situation you was 4 years ago.
My options:
Waterline just for ORM and diff tool (like dbdiff) to make a file with differences from a new schema (generated by waterline migration with 'drop') vs production schema. With that output, you run query by query in a safe mode.
Prev option plus knex migrations. But you have to make your own migrations files. Knex doesnt have a schema file to compare but there is a request feature https://github.com/knex/knex/issues/1086.
Using sails but change waterline for sequalize and give a try at the #paulmest answer.
Use sails but change waterline for typeorm an use that auto generated.
Over the years, my team has tried Sequelize and TypeORM, but neither of them were great experiences. TypeORM had tons of cryptic problems and thousands of issues are unaddressed/open for years on end. Sequelize was generally fine, but lack of type support made it time-consuming to work with.
For the last 1.5 years, my team has been using Hasura. It's honestly been a breath of fresh air. Hasura's migration system is pretty barebones compared to Django South, but it is straight-forward and has a clean happy path.
You can use the Hasura Console (a localhost web editor) to create tables and add/remove columns. Those changes will be automatically distilled into schema changes stored in .sql files in a migration directory. You can modify these files. You can run a migration command from the command line to apply them when you want.
Since Hasura is a GraphQL engine, it also comes with a schematized SDK that is TypeScript compatible so you get incredible editor support.
All of this and much more is available in the open source version of their product. We have not needed to pay for any higher tier.
You can find more information here: https://hasura.io/docs/latest/graphql/core/migrations/index.html
Is it possible to turn off Entity Framework using the web.config? In the application I'm developing we have the following environments
Development
Continuous Integration
Integration Testing
Production
The Integration Testing and Production databases are managed by a database administrator, so we have to send them a script to make changes to the database.
I've spent hours Googling and looking through old projects, and I can't find how to do this or remember if we ever turned off migrations on the old projects in the first place.
From the lack of information I'm doubting if what I'm asking is needed or possible, but there is something in the back of my head that's annoying me about this so I thought I'd ask the experts.
The easiest method is to simply delete the dbo._MigrationHistory table from these environments. If that table doesn't exist, then only an "initial" migration can ever be generated against that database, which will fail if someone tries to actually apply it to a database with existing tables.
You could set the database initializer in the config file as described at the bottom here, so you can have an updating initializer in the environments you want
I have a database project in Visual Studio 2012 with SSDT (latest as of this writing). In the database project, I have a schema called "UNITTEST" which contains tons of stored procedures that create, destroy, and provide other helper functionality for the unit tests. We do this because it gives us the ability to control our test data centrally rather than inside each unit test. Now that's fine and all however, I don't want to publish this schema or any of the objects inside of this schema to production.
So my question.. Is there a way to stop SSDT/VS2012 from including the UNITTEST schema in the production build deployment script?
I'm thinking there should be a way to do it depending on the solution configuration settings and publish profiles. If my configuration is set to "Release" then I want the build to perform a bit differently.
Builds are very new to me. I found this question: build-different-scripts-depending-on-build-configuration but I can't seem to get the answer to fulfill my problem. This question also doesn't help although it's very similar: bind-the-deploy-and-publish-destination.
Is anyone else managing something like this? The other developers in my team are just modifying the published script to remove these objects but I HATE manual work, there HAS to be a solution! :)
Thanks all!
One of my schemas references a lot of sys.* objects which created a lot of errors in the build. I created another project in the solution and moved that schema to the new project.
Luckily you can build and publish at the project level.
This allows me to keep the other schema in change control at least.
(It may also help to set the Properties on the SQL files to Build Action: None)
Partial/Composite projects might be useful here. Main project contains all of your necessary DB objects for your apps to run. The partial project references the main project, but then contains all of the "Test" code.
Here are a couple of options from Jamie Thomson:
http://sqlblog.com/blogs/jamie_thomson/archive/2013/03/10/deployment-of-client-specific-database-code-using-ssdt.aspx --This may be the simplest way to handle this
http://sqlblog.com/blogs/jamie_thomson/archive/2012/01/01/implementing-sql-server-solutions-using-visual-studio-2010-database-projects-a-compendium-of-project-experiences.aspx --Lots of good information in this post and most of it also applies to SSDT SQL Projects.
http://msdn.microsoft.com/en-us/library/dd193415.aspx - Composite projects for larger DBs. This could potentially work for you as well.