I have cloned this eleventy : https://github.com/anborg/eleventy-netlify-boilerplate
I'm trying to enable netlify CMS with local git repo.
I expect to see Workflow section (between Contents and Media) in the below screenshot. What setting should I change to enable Workflow ?
http://localhost:3000/admin/#/collections/blog
admin\config.yml
local_backend:
url : http://localhost:3030/api/v1
# when accessing this from other hosts
allowed_hosts : ['*']
backend:
name: github
repo: anborg/one-click-hugo-cms
branch: master # optional, defaults to master
open_authoring: true
# Uncomment below to enable drafts
publish_mode: editorial_workflow
.env
# This file is used by stat:netlifyProxy for port 3030
#### below props are for npx netlify-cms-proxy-server #######
# optional, defaults to current directory
#GIT_REPO_DIRECTORY=FULL_PATH_TO_LOCAL_GIT_REPO
# optional, defaults to 8081
PORT=3030
package.json has proxy
{
"proxy": "http://localhost:3030",
"name": "eleventy-netlify-boilerplate",
"version": "1.0.3",
"description": "A boilerplate for building a fast static website with the Eleventy static site generator, for deployment to Netlify.",
"scripts": {
"start": "run-p start:**",
"start:eleventy": "set ELEVENTY_EXPERIMENTAL=true && npx #11ty/eleventy --serve --port=3000",
"start:netlifyProxy": "npx netlify-cms-proxy-server",
"build": "npx eleventy",
"watch": "npx eleventy --watch",
"debug": "DEBUG=* npx eleventy"
},
"repository": {
It looks like this isn't working because the local_backend section isn't set up correctly, or isn't supported by the editorial workflow yet.
According to the docs, you do have to use a supported Git service: GitLab, GitHub or BitBucket, as the editorial workflow is using the specific Pull-Request features of those services.
Try removing that bit from the config and the editorial_workflow will connect via the github gateway.
Related
I'm running Postman tests via Newman using files that are in a Git repository. That works fine. My issue is I have multiple sets of tests, each test has it's own Jenkins job that triggers when a build is successful. I'm trying NOT to use multiple git repositories to host these files.
The tests are run using a package.json file like this:
{
"name": "postmanfiles",
"version": "1.0.0",
"description": "project to store postman collection",
"main": "index.js",
"scripts": { "api-tests": "newman run --verbose --timeout 8000000 --reporters 'cli,testrail' 'SearchAPI.postman_collection.json' -e 'My_Environment.postman_environment.json'" },
"keywords": [ "Postman", "CI" ],
"author": "Joe Smith",
"license": "ISC",
"dependencies": { "newman": "^6.14.10"
}
I have multiple Postman collections, for example, 'SearchAPI.postman_collection.json', 'ModifyAPI.postman_collection.json', etc for each set of tests.
In order to not use multiple Git repos for these tests I need multiple package.json files in one repository because I have five different Postman collections to run. Each Jenkins job triggers off of separate builds. An easy way to do this would be to have five different uniquely named package.json files in one Git repository.
But can I do that and if so how?
Or is there an easier way to do this?
I got this to work by specifying multiple tests inside package.json and then having unique Jenkins scripts in the repository, one for each set of tests.
"scripts": {
"api-tests": "newman run --verbose --timeout 8000000 --reporters 'cli' 'SearchAPI.postman_collection.json' -e 'CTV_Environment.postman_environment.json'",
"site-tests": "newman run --verbose --timeout 8000000 --reporters 'cli' 'SiteAPI.postman_collection.json' -e 'CTV_Environment.postman_environment.json'"
},
As far as I know you can't. Which is why we use tools like Grunt/Gulp or Webpack to get complex build workflows done.
I am deploying a Google Cloud function and it fails to deploy because it sometimes doesn't find the module #google-cloud/pubsub and when it finds it, it instead fails on a transitive dependency from the pubsub package. I have had this with #google-cloud/storage as well and never managed to solve it.
See the logs screenshot below in this post for an example of the issue. It shows two consecutive deployments.
In the first deployment, the #google-cloud/pubsub is not found although it is listed in package.json
In the second deployment, the pubsub package is found(!), but then instead it fails on the transitive dependency google-gax.
Things I tried
Deploy using Google Cloud Build, hooked up to my github repo (deplyoment fails)
Deploy from local command line. See script target deployAnalyzer in package.json below. (deployment fails with same errors a Google Cloud Builds)
Deploy the function locally to the functions framework. See script target devFunctionAnalyzer below. (Deplyoment works and the function can be invoked)
Build locally using pack to repeat the same build procedure as Google Cloud Build. Result: Image builds and runs fine!
In short: It work fine locally, both in the Functions Framework and running the docker image built using the google nodejs buildpack.
The Full Code
Work in progress: https://github.com/aweijnitz/ornitho-de-monitor
Minimalistic repo with one dependency, that fails with the same error: https://github.com/aweijnitz/cloud-functions-dependency-test-01/tree/main
EDIT
Added link to stripped down repo that illustrates the problem without all the application code. Just a single dependency now.
package.json
{
"name": "ornitho-de-monitor",
"version": "0.0.1",
"description": "Collect latest interesting observations from ornitho.de",
"main": "src/index.js",
"engines": {
"node": ">=12.13"
},
"scripts": {
"devFunctionAnalyzer": "npx functions-framework --target=analyzeObservations --signature-type=event",
"devFunctionNotifier": "npx functions-framework --target=notifyAll --signature-type=event",
"buildCloudRunContainer": "gcloud builds submit --tag gcr.io/ornitho-de-monitor/ornitho-de-scraper",
"deployCloudRunContainer": "gcloud run deploy ornitho-de-scraper --region=europe-west6 --image gcr.io/ornitho-de-monitor/ornitho-de-scraper --platform managed",
"deployAnalyzer": "gcloud functions deploy analyzeObservations --source=./src --entry-point=analyzeObservations --runtime nodejs12 --memory=256MB --max-instances=3 --trigger-topic=ornitho-bus",
"deployNotifier": "gcloud functions deploy notifyAll --source=./src --entry-point=notifyAll --runtime nodejs12 --memory=256MB --max-instances=3 --trigger-topic=ornitho-bus",
"invoke": "gcloud functions call ornitho-de-monitor --data '{\"name\":\"Keyboard Cat\"}'",
"invokeEncoded": "DATA=$(printf 'Hello!'|base64) && gcloud functions call helloPubSub --data '{\"data\":\"'$DATA'\"}'",
"viewFunctionLogs": "gcloud functions logs read ornitho-de-monitor",
"serveTestFile": "cd testdata && python -m SimpleHTTPServer 8000",
"test": "echo \"No test specified\" && exit 0"
},
"author": "Anders Weijnitz",
"license": "ISC",
"dependencies": {
"#google-cloud/pubsub": "^2.7.0",
"#google-cloud/storage": "^5.7.0",
"express": "^4.17.1",
"express-async-handler": "^1.1.4",
"express-rate-limit": "^5.2.3",
"knex": "^0.21.15",
"pg": "^8.5.1"
},
"devDependencies": {
"#google-cloud/functions-framework": "^1.7.1"
}
}
These are the relevant steps from cloudbuild.yaml
steps:
# Retrieve credentials from Fort Knox
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
args: [ '-c', "gcloud secrets versions access latest --secret=ornitho-pubsub --format='get(payload.data)' | tr '_-' '/+' | base64 -d > pubsubkey.json" ]
# Install dependencies
- name: 'gcr.io/cloud-builders/npm'
args: [ 'install' ]
dir: '.'
# Run tests
- name: 'gcr.io/cloud-builders/npm'
args: [ 'test' ]
dir: '.'
# Deploy the notifier function
- name: 'gcr.io/cloud-builders/gcloud'
args:
- 'functions'
- 'deploy'
- 'notifyAll'
- '--source=./src'
- '--entry-point=notifyAll'
- '--runtime=nodejs12'
- '--memory=128MB'
- '--max-instances=3'
- '--trigger-topic=ornitho-bus'
# Deploy the analyzer function
- name: 'gcr.io/cloud-builders/gcloud'
args:
- 'functions'
- 'deploy'
- 'analyzeObservations'
- '--source=./src'
- '--entry-point=analyzeObservations'
- '--runtime=nodejs12'
- '--memory=128MB'
- '--max-instances=3'
- '--trigger-topic=ornitho-bus'
SOLVED!
In turns out that the --source parameter should not point to the location of the application source, but to the location of the project root, where package.json resides. Bad: --source=./src Good: --source=. .
For more details, see github.com/aweijnitz/cloud-functions-dependency-test-01 –
I developed a micro-front end UI app, the application is running on local by npm start command, but now I want to deploy it, for that I need to publish it using build command, when I am trying to run npm build --prod or just npm build, its not throwing any error but I don't see any dist folder created in my app root folder.
Below is my package.json and app structure:
{
"name": "root-html-file",
"description": "The single-spa root config for coexisting-angular-microfrontends",
"main": "index.js",
"scripts": {
"start": "serve -s -l 4200",
},
"author": "Joel Denning",
"license": "MIT",
"devDependencies": {
"serve": "^11.1.0"
},
"dependencies": {}
}
Dist folder is not getting created while building a single-spa root node app
When we build the app on the azure cloud server, Azure devops will checkout the source code from the repo to the build agent. Then the output of the build pipeline will save on this agent instead of our repo.
So, we could not see the dist folder from the repo. It is saved in the local folder on the agent. We need use the copy files and publish build artifact task to publish build artifacts to Azure Pipelines.
Note: The Source Folder in the copy files task should be Agent.BuildDirectory or System.DefaultWorkingDirectory instead of directly selecting the path in the repo through the extension button.
If above not help you, please share you build log, so that we could check if the dist folder created or not.
I have simple running react Server Side App .
GitHub Link for the repo, in the local env i have a build script that runs using npm start which starts the client and server builds and then i have a watcher which listen's to changes in file. This set up works really well locally.
"scripts": {
"start": "npm run dev",
"dev": "npm-run-all --parallel dev:*",
"dev:server": "nodemon --watch build --exec \"node build/bundle.js\"",
"dev:build-server": "webpack --config webpack.server.js --watch",
"dev:build-client": "webpack --config webpack.client.js --watch"
},
I tried to deploy these changes as a prod app to heroku here for some odd reason it never works and returns 503 . I have not added any changes to the build scripts in the package.json and tried deploying as is.
I thought it should work as the build log dosent give any errors on heroku but while trying to access the app it shows error and asks me to check app log but i am not using heroku cli and not planning on using it , and
thinking of auto deployment from github.
I am quite new to script/ largely build scripts trying to learn more on them.
How can we make sure this small React SSR git repo deploys and i am
able to access the home page .
I am not using heroku cli and not planning on using it
You should and I really recommend, as it's the best way to see errors that occurs after the build, why your app crashed and see the full logs. Even though you can see the logs from your heroku dashbord (More -> View logs), this only gives you the tail of the logs.
How can we make sure this small React SSR git repo deploys and i am able to access the home page.
Make sure the server is listening on the right port
by using process.env.PORT as heroku expose a dynamic port.
const port = process.env.PORT || 3000
app.listen(port, () => {
console.log('Listening on Port ' + port);
})
On heroku the NODE_ENV environment variable is set toproduction by default,
which means heroku will prune thedevDependencies after the build,
but in your case, you still need these dependencies to start your app with webpack.
You can fix this in two ways :
Customize your build process by adding a build script:
From your github
repo
:
"start": "webpack --config webpack.client.js; webpack --config webpack.server.js; node build/bundle.js"
to
"scripts": {
"start": "node build/bundle.js",
"build": "webpack --config webpack.client.js & webpack --config webpack.server.js",
"build:start": "webpack --config webpack.client.js && webpack --config webpack.server.js && node build/bundle.js",
}
Set the NODE_ENV to any other value to skip pruning your
devDependencies
heroku config:set NODE_ENV=development //(Another reason to use the CLI)
Check this link for more details
I wanted to provide a howto on this as I couldn't find any complete information. I thought that this seemed most appropriate in Stackoverflow documentation. However it has been sunsetted *-) - Sunsetting Documentation (*).
Instead I will write this as a StackOverflow Q&A.
How to deploy a Typescript, NodeJS and Express app to Heroku
I created a project (https://gitlab.com/OehmSmith_Examples/herokumovies) that includes a README describing what needs to be done and I will reproduce that here. As a good StackOverflow practice I will also provide a copy of all the code at the bottom of this post.
Tutorial - Deploy Typescript NodeJS Express app to Heroku
This tutorial will work from https://amenallah.com/node-js-typescript-jest-express-starter/ as the base app. I have no affiliation with that site or the author. I chose it as it is simple and works. It is also an example of good OO Typescript code.
Install typescript
Most tutorials or even the official documentation in https://www.typescriptlang.org/docs/handbook/typescript-in-5-minutes.html say to install typescript globally:
> npm install -g typescript
Heroku doesn't have a global install of typescript so it needs to be kept locally. The example project does just this:
> npm i -D nodemon rimraf typescript ts-node ts-jest jest #types/jest #types/node
#types/node
In the case you have pinned your #types/node at an older version you will see something like this error:
~/AppData/Roaming/nvm/v11.15.0/node_modules/typescript/lib/lib.es2015.iterable.d.ts:41:6 - error TS2300: Duplicate identifier 'IteratorResult'.
41 type IteratorResult<T, TReturn = any> = IteratorYieldResult<T> | IteratorReturnResult<TReturn>;
~~~~~~~~~~~~~~
node_modules/#types/node/index.d.ts:170:11
170 interface IteratorResult<T> { }
~~~~~~~~~~~~~~
'IteratorResult' was also declared here.
node_modules/#types/node/index.d.ts:170:11 - error TS2300: Duplicate identifier 'IteratorResult'.
170 interface IteratorResult<T> { }
~~~~~~~~~~~~~~
~/AppData/Roaming/nvm/v11.15.0/node_modules/typescript/lib/lib.es2015.iterable.d.ts:41:6
41 type IteratorResult<T, TReturn = any> = IteratorYieldResult<T> | IteratorReturnResult<TReturn>;
~~~~~~~~~~~~~~
'IteratorResult' was also declared here.
Found 2 errors.
From TypeScript: Duplicate identifier 'IteratorResult'. And as per that you need to update your version of #types/node. This was a problem I struck as I was working with older code and wanted to include this discussion of it.
Update port for cloud service
Change the index.ts to be the following instead since the original code hard-coded port 5000:
app.listen(process.env.PORT, () => {
console.log(`server started on port ${process.env.PORT}`)
});
To allow for this I have added the PORT to the npm scripts, including adding a start:dev so you can run it like Heroku does from the compiled typescript.
"start:dev": "PORT=5000 node dist/index.js",
"dev": "PORT=5000 nodemon --exec ts-node src/index.ts --watch src",
Or it can be set in a .env file:
PORT=5000
NPM dependencies
Heroku will NOT install dev dependencies (neither will any other cloud provider). Therefore you need to move some dependencies to the main block. For example, a NestJS application has these as Dev Dependencies and they need to be moved:
#nestjs/cli
Dummy data
I added this constructor to the MoviesApi.ts:
constructor() {
// setup some dummy data
movies.push({
name: 'Pirates of the caribbean',
rating: 8.5
})
movies.push({
name: 'Star Wars: A new hope',
rating: 8.7
})
}
Heroku
Now deploy to Heroku
Setup an account if you don't already have one and create an app on Heroku
In your terminal:
heroku login
heroku create moviesheroku // this needs to be unique
You may need to add the returned git url as a remote (check with git remote -v):
git remote add heroku <git url>
Lookup or search for buildpacks with (the next step already specifies those I use):
Lookup at: https://devcenter.heroku.com/articles/buildpacks#officially-supported-buildpacks
Search with:
heroku buildpacks:search typescript
Add buildpacks:
heroku buildpacks:add zidizei/typescript
heroku buildpacks:add heroku/nodejs
Confirm buildpacks:
heroku buildpacks
Commit to your local repository
git init // if not already done
git add --all
git ci -m "Initial commit. Test project all setup and should be ready to 'serve' but not yet ready to deploy to heroku"
Run with
npm dev # OR
npm run start:dev # It depends on your npm scripts
Test with postman or similar or run this from the command-line:
curl http://localhost:5000/movies
Test that it transpiles with npm run build
Update the npm scripts so that after installation (npm install) on Heroku, it will build it before attempting to npm run start
"postinstall": "npm run build" # Depends on your npm scripts
Commit to local repository:
git add --all
git ci -m "Now it should deploy, build and run on heroku"
Deploy to heroku. It should build and start up.
git push heroku master
Test (assuming the app you heroku created is moviesheroku - adjust accordingly)
curl https://moviesheroku.herokuapp.com/movies
Variations
Procfile
I haven't specified a Procfile telling Heroku anything about the app. Fortunately it creates its own defaults by determining that this is a node + npm app. However you can explicitly define this and will need to perform this action if you have multiple apps or similar. You could add a Procfile to contain (this is the default):
web: npm start
Node and NPM version
Heroku also defaults to using one of the most recent versions of these. You could explicitly set the versions at the top-level in the package.json file like:
"engines": {
"node": "10.x",
"npm": "6.x"
},
Although if you don't specify a npm version then Heroku will use a sensible default for the version of node.
Concluding thoughts
I had this going in only a couple hours. The main issues I needed to work out is that typescript has to be local, not global. And the buildpacks. The PORT is an issue also though every cloud provider requires the use of process.env.PORT so this was obvious to me.
Azure was a nightmare and took days, but that was mainly because the workplace I was at insisted on using Windows servers. Long story and I won't go in to it.
AWS was so convoluted. I didn't get the instance I had working after trying for a day. however I do need to try again. The app I was trying used the https://tsed.io/ library. Simple Node / Typescript / Express apps should work out quite easily.
(*) - the sunsetting of documentation was a little bit surprising though given it happened over 2 years ago I guess it wasn't something I used. And I always thought that the Q&A was the easiest place for documentation.
Code
.gitignore
node_modules
dist
coverage
.jest.config.js
module.exports = {
preset: 'ts-jest',
testEnvironment: 'node'
};
package.json
{
"name": "movies",
"version": "1.0.0",
"description": "Example from https://amenallah.com/node-js-typescript-jest-express-starter/ but then modify and / or show steps for how to deploy this Typescript NodeJS Express RESTful app to Heroku.",
"main": "index.js",
"scripts": {
"build": "rimraf dist && tsc",
"postinstall": "npm run build",
"start": "node dist/index.js",
"start:dev": "PORT=5000 node dist/index.js",
"dev": "PORT=5000 nodemon --exec ts-node src/index.ts --watch src",
"test": "jest --watch",
"coverage": "jest --coverage"
},
"keywords": [],
"author": "",
"license": "ISC",
"devDependencies": {
"#types/express": "^4.17.2",
"#types/jest": "^24.0.25",
"#types/node": "^13.1.2",
"jest": "^24.9.0",
"nodemon": "^2.0.2",
"rimraf": "^3.0.0",
"ts-jest": "^24.2.0",
"ts-node": "^8.5.4",
"typescript": "^3.7.4"
},
"dependencies": {
"body-parser": "^1.19.0",
"express": "^4.17.1"
}
}
tsconfig.json
{
"compilerOptions": {
"target": "es5",
"module": "commonjs",
"outDir": "dist",
"sourceMap": false,
"allowSyntheticDefaultImports": true,
"baseUrl": ".",
"paths": {
"*": [
"node_modules/",
],
"typings/*": [
"src/typings/*"
]
},
},
"include": [
"src/**/*.ts"
],
"exclude": [
"src/test/**/*.spec.ts"
]
}
src/api/MoviesApi.ts
import IResource from "typings/IResource";
let movies: object[] = []
export default class MoviesApi implements IResource {
constructor() {
// setup some dummy data
movies.push({
name: 'Pirates of the caribbean',
rating: 8.5
})
movies.push({
name: 'Star Wars: A new hope',
rating: 8.7
})
}
create(data: any): any {
movies.push(data)
return data
}
findMany(): any[] {
return movies;
}
}
src/test/api/Movies.spec.ts
import IResource from '../typings/IResource'
import MoviesApi from '../api/MoviesApi'
const moviesApi: IResource = new MoviesApi()
describe('Movies API', () => {
it('should create a new movie', () => {
const movieData: object = {
name: 'Pirates of the caribbean',
rating: 8.5
};
const movie: object = moviesApi.create(movieData);
expect(movie).toEqual(movieData)
})
});
src/typings/IResource/index.d.ts
export default interface IResource {
create(data: any): any
findMany(): any[]
}
src/index.ts
import * as express from 'express'
import * as bodyParser from 'body-parser'
import MoviesApi from './api/MoviesApi'
const app = express();
const moviesApi = new MoviesApi();
app.use(bodyParser.json());
app.post('/movies', (req: express.Request, res: express.Response) => {
res.json(moviesApi.create(req.body))
});
app.get('/movies', (req: express.Request, res: express.Response) => {
res.json(moviesApi.findMany())
});
app.listen(process.env.PORT, () => {
console.log(`server started on port ${process.env.PORT}`)
});