How to run a Go server in Travis-CI? - node.js

I'm working on a Go project that exposes a RESTful http API.
I want to run the Go project and use Nodejs (Mocha) to test the endpoints. It seems as if the nohup command doesn't keep running in the background.
Locally everything works but I don't seem to be able to get it running in Travis-ci.
language: go
go:
- 1.8
env:
- "PATH=/home/travis/gopath/bin:$PATH"
before_install:
- go get ./...
script:
- npm install mocha -g
- npm install
- nohup go run ./cmd/server/main.go --scheme=http --port=8080 --host=127.0.0.1 &
- mocha

First, you should seriously question whether writing tests in mocha makes sense. Admittedly, there might be cases when it does make sense (i.e. if you're porting a nodejs app to Go, and already have tests written for node). But even then, you should consider this a stop-gap measure, and write all new tests, and even migrate old tests, to Go as soon as possible.
But that aside, there's no reason you shouldn't be able to just start the process in the background normally. Perhaps with a shell script called from your Travis config (can often be cleaner and easier to follow than putting all commands in the config directly):
#!/bin/sh
go run &
mocha
If you really must run your tests in an external process, there are certain advantages from starting that from a Go test anyway. Namely, you can get test coverage stats, and starting can be more easily synchronized (so you don't need a sleep). To do this, you can follow my advice in this answer. Specifically, for the mocha case:
Write a test file that executes your main() function in a go routine:
func TestMainApp(t *testing.T) {
go main() // Or whatever you need to do to start the server process
cmd := exec.Command("mocha", ...)
cmd.Start()
}
But seriously. You should write your tests in Go.

Mocha runs directly after - nohup go run ./cmd/server/main.go --scheme=http --port=8080 --host=127.0.0.1 & so the server didn't start up yet.
Adding a sleep was enough.
script:
- npm install mocha -g
- npm install
- nohup go run ./cmd/server/main.go --scheme=http --port=8080 --host=127.0.0.1 &
- sleep 4
- mocha
But, the point is taken, I will move to Go tests :-P

Related

How to write a Dockerfile to let docker build failed if the built container fails to run?

I have a nodejs webapp run in docker, the Dockerfile is just the usual stuff as in Dockerizing a Node.js web app. But I hope to do one thing differently: if the docker container fails to run, e.g. incorrect node_modules I would like docker build fails.
If I just add RUN npm run start in Dockerfile and it fails to run, docker build indeed fails. But if npm run start succeeds, docker build won't exit then.
My goal is that if docker run fails, I hope docker build fails first. Currently I can't find a way to do that, so I have to write a script to first docker build then docker run and check its result.
--- update ---
I know normally it is in build/test/release order and testing comes after the build. But I am trying to improving our CI process and after build I hope to do a quick test to make sure the container can start correctly. If that needs a manual check it is quite unproductive. So I think why not do a one step further? If container can not start why not just fails the build?
--- update 2 ---
I felt rather uncomfortable with the answer I got. The main reason is that I felt that solution was dirty. I raised another question about it at SE's softwareengineering website Improve CI process by testing against docker image and fail docker build if test fails and now I fully agree that what I proposed was not a good idea. Also github action is a good solution to reduce failed docker images.
I would approach it in this way:
First find out how long does npm run start takes for you. In normal cases it won't take more than 15-20 seconds but let's say, it takes 60 seconds for your app to successfully start. Now I would keep a conditional statement in the server.js or the main entrypoint of your app like:
if (process.argv[2] === 'BUILD') { // Call only when docker build is ran
setTimeout(() => process.exit(0), 60000) // This would wait for 60 seconds and if the app is still running it will terminate it.
}
You just need to ensure that you call RUN npm run start -- --BUILD while running the server during docker build as the -- --BUILD would pass a runtime argument to your server to ensure that it was called during the build process. For normal docker run entrypoint you can just call npm run start and it should work fine.

`npm test` never exits

I use mocha to run my unit tests, and often use npm test in order to run it. My package.json contains these script definitions:
"pretest": "NODE_ENV=test node migrate all",
"test": "DEBUG= NODE_ENV=test mocha --recursive",
If I run either one of these commands directly in my shell (i.e. not going through npm) they execute fine (790 tests takes about 2m to run, the migration script is done in under 1s). The processes also exit cleanly.
If, however, I run these via npm test, everything runs in exactly the same way but the process doesn't exit (I have to manually cancel it with ^c).
I can't work out how to debug exactly what is going on here in order to work out why the process isn't exiting.
It's worth noting that if I test a subdirectory (npm test ./test/queue) which does not interact at all with the database, then the process exits fine. These tests do, however, interact with an AMQP broker so there is some activity over sockets. This suggests to me that the database connection is causing the problem. I am using knex to connect to a postgres9.6 server. This also suggests that the pretest script is not the problem. If I try to run a suite of tests which do interact with the database, the process never exits (so presumably the open sockets are preventing it from doing so, but why this should only happen in the case of npm test rather than direct execution is beyond me).
Extra info:
node version: 8.7.0
mocha version: 4.0.1
npm version: 5.6.0
knex version: 0.14.0
I am using async/await in my codebase
I was mistaken in thinking this to be an npm problem. Rather, when I was running mocha in my shell it was using the globally installed mocha, which is of a different version to the locally installed one. Running with this version causes the same hanging issue.
This change in behaviour appears to be coming from this issue, which is described thus in the changelog:
By default, Mocha will no longer force the process to exit once all tests complete. This means any test code (or code under test) which would normally prevent node from exiting will do so when run in Mocha. Supply the --exit flag to revert to pre-v4.0.0 behavior (#ScottFreeCode, #boneskull)

How to start node server before running end to end tests using npm run?

I'm writing end to end tests for an express site, and I want to add a "test" command into package.js
This command needs to:
run eslint
compile typescript
start node server
run unit tests against that server and show output.
once done testing, close the server.
I know how to execute all those commands individually, but not all at once.
What I have now is :
npm run compile && npm run build && node ./dist/server.js --db=test && npm run test
It works to the point of: "&& npm run test"
since node server is running, it won't continue on to the next command, and if it closes then tests wouldn't run.
Any help is appreciated.
One thing that I have found to help with reliable, maintainable end-to-end tests is to separate concerns:
Test suite assumes that the server is already running
Orchestrator calls into separate commands to bring up your test stack then run the tests
In CI, this could look like
npm start-e2e-test-stack --port=XXXX --db=test
npm test --port=XXXX --db=test
npm teardown-e2e-test-stack
In my experiences, having the end-to-end tests operate against any server helps to allow them to verify all environments, local, dev, qa, staging, production.

atom + mocha on windows = spawn mocha ENOENT

My goal is to run mocha unit tests by Atom, which is installed on Windows and also my src code resides. this should work independently from my Meteor App which is running on a different (Linux) machine.
Basically my setup is like this:
I have my repo and sourcecode:
c:\Users\Me\repos\meteor
My tests are inside:
c:\Users\Me\repos\meteor\tests
I have Node:
c:\Program Files\nodejs
installed with "npm i -g mocha --save-dev"
And i try to use this package https://github.com/Tabcorp/atom-mocha-test-runner but i can switch to another package if necessary.
What I've tried so far:
I edited my settings for the atom-mocha-test-runner:
Mocha command: C:\Program Files\nodejs\node_modules\mocha\.bin\mocha
Mocha command: C:\Program Files\nodejs\npm mocha
But each time i try to run my test via dropdown menu (Run Mocha Test), i get this error:
Mocha Test Results:
Node binary: C:\Program Files\nodejs\node.exe
Root folder: C:\Source\Repos
Mocha command: undefined
Path to mocha: mocha
Debug-Mode: false
Test file: tests\unit\first.js
Selected test: should return url
Failed to run Mocha
spawn mocha ENOENT
Anyone know what i miss or do wrong?
Still having no idea about why the package isn't working, I'm going to give a cop-out answer. If we figure out how to make it work, you can accept that answer instead. process-palette gives you the ability to run highly specific command-line instructions from Atom commands. Here's an example of a command that runs mocha in the project path for the current file with the same hotkey and also conveniently organized into its own menu item:
The disadvantage of this approach is that you have to know how to use the external program yourself. Packages like mocha-test-runner are designed to remove that need from the user, but as we can see here, sometimes the package doesn't know what it needs to be doing. The disadvantage is mitigated by the fact that you only have to learn the command for long enough to set up the configuration to run it, and from that point on it's very easy.
Advantages versus other packages include the ability to precisely control what's going on. Say you have multiple top-level folders in the current project, and they have different test suites. A package like mocha-test-runner can get the path from the active file, or from the project. If the developer chose to grab the project path, then you're going to have trouble running individual test suites. With the configuration I've shared, the command will always be run in the absolute path of the current file's project folder, so the tests will be run for whichever file you're working on at the time.

how can I debug a node app that is started via the command line (cli) like forever or supervisor?

I'm familiar with debugging my own node apps (usually with node-inspector). Today I'd like to debug someone else's program. I'm trying to track down an issue with supervisor. So naturally I just add a --debug (or debug-brk) to the command call, but it passes that to the code that it is supervising.
I've tried adding debugger lines to the js file for supervisor but that didn't work (probably because no debugger was attached at that time). There's a bit of a race here -- I need to start the debugger and attach it to the supervisor process after it starts but before it processes its arguments from the command line.
What I really want to do here is stop supervisor and debug it before it processes its command line arguments. How can I do this?
I had the same problem while developing my hexo blog. The documentation isn't all that complete yet so I find myself needing to reverse engineer at times.
The basic idea is that in Node.js even your cli apps are simply normal node apps that you are exposing to the OS command line interface. On Unix systems you are using this line:
#!/usr/bin/env node
To allow the environment to execute the script.
Many cli based node apps try to insist that you install them globally with the -g option.
npm install -g node-inspector
I personally prefer to have as much control of my development environment as I can get, so I prefer to break some conventions and check my node_modules in to source control along with installing everything I can locally by dropping the -g.
npm install node-inspector
Now you don't have to do this in order to make this work, I'm just describing this setup because it relates to your question. When I run node-inspector I can't simply use:
node-inspector
Instead I must explicitly invoke it from within my project. I do this by executing the symlink in my node_modules/.bin folder:
node_modules/.bin/node-inspector
Now I'm running node-inspector just like you.
Next all I need to do is start the cli process in debug and optionally pass params to it:
node --debug-brk node_modules/.bin/hexo generate
Note I am explicitly calling the symlink here and not simply:
node --debug-brk hexo generate
If I tried the line above I would get an error: "Error: Cannot find module".
I hope this helps.

Resources