I'm using https://github.com/apocas/dockerode to manage docker containers from a node.js app and have written the following function:
run: function(opts) {
var self = this;
return self.createContainer(opts).catch(function(e) {
return self.pull(opts.Image).then(function(stream) {
return self.followProgress(stream);
}).then(function() {
return self.createContainer(opts);
});
}).then(function(o) {
Bluebird.promisify(o.start, {context: o})();
return o;
}).then(function(o) {;
var inspect = Bluebird.promisify(
o.inspect, {context: o}
);
return inspect();
});
},
which I call like this:
bitcoind: function(opts) {
return this.run({
name: 'bitcoind',
Image: 'seegno/bitcoind:latest',
Env: ['BITCOIN_DATA=/data'],
Binds: [process.cwd() + '/bitcoind:/data'],
HostConfig: {
PortBindings: {
'8332/tcp': [{ HostPort: '8332' }]
}
},
Cmd: [
'-server',
'-rpcuser=' + opts.user,
'-rpcpassword=' + opts.pass,
'-printtoconsole'
]
});
},
and everything works perfectly, almost. if I start my bitcoind server like this:
docker.bitcoind(config.bitcoind).then(function(info) {
var p = info.NetworkSettings.Ports;
console.log(p);
});
the console shows "null". the ports didn't get set. the command line shows me proper values. running
$ docker inspect bitcoind
I see:
"Ports": {
"18332/tcp": null,
"18333/tcp": null,
"18444/tcp": null,
"8332/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "8332"
}
],
"8333/tcp": null
},
which is what I expected. so out of curiosity I then ran:
docker.inspect('693f32482b46').then(function(info) {
console.log(info);
});
and that contains the port information! so I'm puzzled as to why the inspection I ran at the end of the run() would return everything but the port mappings. had I failed to wait correctly for the process to start, inspection would have returned an empty value.
thoughts as to what might be going on here or what I should do differently?
ok, the answer is I must wait for the start method to complete
Related
Update
It seems like webpack is causing the issues.
If I replace the dist/background.js with:
console.log("background is running"); // Now visible ✅
const handler = (req, sender, sendResponse) => {
switch (req.type) {
case "message":
sendResponse({ data: "hi" });
break;
default:
break;
}
};
chrome.runtime.onMessage.addListener(handler);
Both the console log (in service worker) and response (in popup) are observed. Also, there are no errors.
Time to investigate further
Update #2
Upon further inspection, I noticed that the webpack output is wrapped in a function, but never called:
{
/***/ "./src/background.ts":
/*!***************************!*\
!*** ./src/background.ts ***!
\***************************/
/***/ (function () {
console.log("background is running");
const handler = (req, sender, sendResponse) => {
switch (req.type) {
case "message":
sendResponse({ data: "hi" });
break;
default:
break;
}
};
chrome.runtime.onMessage.addListener(handler);
/***/
}) // <==== adding () will make it an IIFE and everything works!
},
Question is how to automate this?
Update #3
Seems like the IIFE trick I mentioned above only works when there are no imports in background.js. As soon as I add any import, I get an error that the background script is not valid.
Adding module type property to background does not help:
// manifest.json
{
...
"background": {
"service_worker": "background.js",
"type": "module"
},
...
}
Update #4
Turns out this was caused by vendor splitting optimization in webpack:
// webpack.config.json
{
...
optimization: {
runtimeChunk: "single",
splitChunks: {
cacheGroups: {
vendor: {
test: /[\\/]node_modules[\\/]/,
name: "vendors",
enforce: true,
chunks: "all",
},
},
},
},
...
}
Once I removed this, everything started working properly!
Would be nice to keep this around, but it is just an optimization after all, so if it breaks things, best to get rid of it.
How I figured this out? Well as I mentioned, everything worked a couple of commits ago. Back then I didn't have this optimization, so I tried removing it again, and everything started working again like magic.
Original Question
I had this working previously, so I am sure my setup is correct, but regardless of what I try, I now get
Unchecked runtime.lastError: Could not establish connection. Receiving end does not exist.
Also, I cannot find a solution online which indicates anything that differs from my setup.
Here is a MWE
dist folder structure:
dist/background.js
dist/index.html
dist/manifest.json
dist/popup.js
dist/runtime.js
dist/vendors.js
other misc files
// manifest.json
{
...
"background": {
"service_worker": "background.js"
},
...
}
// src/components/App.tsx
export default function App(): JSX.Element {
...
useEffect(() => {
chrome.runtime.sendMessage({ type: 'someMessage' }, ({ data }) => {
console.log(data);
});
}, []);
...
}
// src/background.ts
import { TSentResponse } from "./typings/background";
import { executeResponse } from "./utils/background";
console.log('sanity check'); // <=== does not fire 🤔
// also doesn't seem to be called 😥
const handleMessage = (req: { type: string }, sender: chrome.runtime.MessageSender, res: (response?: unknown) => void) => {
switch (req.type) {
case 'someMessage':
// an IIFE (worked fine before)
break;
default:
break;
}
return true; // due to asynchronous nature
};
chrome.runtime.onMessage.addListener(handleMessage);
My service worker is registered properly:
When I said above that I cannot see the logs for background, I mean when I check in the service worker dev tools (from above image), not the popup dev tool.
When I open the popup, I get the following errors:
Unchecked runtime.lastError: Could not establish connection. Receiving end does not exist.
Error handling response: TypeError: Cannot destructure property 'data' of 'undefined' as it is undefined.
I also don't see the service worker actually being registered - it did register before...
Is this a bug with MV3?
My repository (not fully up to date, but can be used to quickly check the above)
Follow these steps:
To send a message from the popup.js to the background service worker, first you need to get current tab id. to get the current tab id do as below:
popup.js
const messageKey = 'key-message-from-popup-to-background';
// Listen to get current tab info from the content_popup
document.addEventListener('_listener_getCurrentTabInfo', function (e) {
const tab = e.detail.response;
// Send a message to the background,js
chrome.tabs.sendMessage(tab.id, messageKey);
});
// Send message to content_popup to get current tab info
document.dispatchEvent(new CustomEvent('_dispatch_getCurrentTabInfo'));
content_popup.js
const config = {
scripts: [
// add ".js" files to web_accessible_resources in manifest.json
"popup/popup.js"
]
};
// Listen to get current tab info from the background.js
document.addEventListener('_dispatch_getCurrentTabInfo', function (e) {
// Key to help
const type = 'get_current_tab_info';
// Send a message to the background.js to get current tab info
chrome.runtime.sendMessage({type: type}, response => {
document.dispatchEvent(new CustomEvent('_listener_getCurrentTabInfo', {detail: {'response': response}}));
});
});
// prepare and add scripts
var scriptList = config['scripts'];
for (var i = 0; i < scriptList.length; i++) {
var s = document.createElement('script');
s.src = chrome.runtime.getURL(scriptList[i]);
s.onload = function () {
this.remove();
};
(document.head || document.documentElement).appendChild(s);
}
background.js
// Get messages
chrome.runtime.onMessage.addListener(function (request, sender, sendResponse) {
// If request related to fetch data from API
if (request.url && request.url !== '') {
// Fetch http request and send back the response
fetch(request.url, request.init).then(function (response) {
return response.text().then(function (text) {
sendResponse([{
body: text,
status: response.status,
statusText: response.statusText,
}, null]);
});
}, function (error) {
sendResponse([null, error]);
});
// If request do not related to fetch data from API
} else {
detectMessageType(request, sender, sendResponse);
}
return true;
});
function detectMessageType(request, sender, sendResponse) {
// Check background request type
if (request && request.type) {
switch (request.type) {
case 'get_current_tab_info': {
getCurrentTabInfo(tab => {
// Send current tab info back to content_popup
sendResponse(tab);
});
break;
}
}
}
});
function getCurrentTabInfo(callback) {
chrome.tabs.query({
active: true,
currentWindow: true
}, function (tab) {
callback(tab[0]);
});
}
Get message in contentScript
chrome.runtime.onMessage.addListener((message, sender, response) => {
const message = message;
// Write your codes
});
This is the project structure (partial)
Project
-- popup
---- popup.js
---- content_popup.js
---- popup.html
---- popup.css
-- content_extension.js
-- background.js
-- manifest.js
Add the following code to the end of the body tag inside the popup.html file
<script src="content_popup.js"></script>
Update the manifest.json file like below
"manifest_version": 3,
.
.
.
"content_scripts": [
{
"js": [
"content_extension.js"
]
}
],
"action": {
"default_icon": ...,
"default_title": ...,
"default_popup": "popup/popup.html"
},
"background": {
"service_worker": "background.js"
},
"web_accessible_resources": [
{
"resources": [
"popup/popup.html",
"popup/popup.js"
]
}
],
...
I'm using the VM2 package to run user code. I'm trying to intercept console output and have set the NodeVM object's console property to 'redirect':
// Create a new sandbox VM for this request
const vm = new NodeVM( {
console: 'redirect',
timeout: 30000,
sandbox: { request, state, response },
require: {
external: true
}
});
According to the documentation that redirects console output to 'events'. I'm new to NodeJS, how do I hook into those events to capture the console.log messages executed inside the Sandbox?
After digging through the source code, I found this file where the event emit is occuring:
sandbox.js
if (vm.options.console === 'inherit') {
global.console = Contextify.readonly(host.console);
} else if (vm.options.console === 'redirect') {
global.console = {
log(...args) {
vm.emit('console.log', ...Decontextify.arguments(args));
return null;
},
info(...args) {
vm.emit('console.info', ...Decontextify.arguments(args));
return null;
},
warn(...args) {
vm.emit('console.warn', ...Decontextify.arguments(args));
return null;
},
error(...args) {
vm.emit('console.error', ...Decontextify.arguments(args));
return null;
},
dir(...args) {
vm.emit('console.dir', ...Decontextify.arguments(args));
return null;
},
time: () => {},
timeEnd: () => {},
trace(...args) {
vm.emit('console.trace', ...Decontextify.arguments(args));
return null;
}
};
}
All you need to do to listen to these events is to bind an event listener on the vm you've created:
// Create a new sandbox VM for this request
const vm = new NodeVM( {
console: 'redirect',
require: {
external: ['request']
}
});
vm.on('console.log', (data) => {
console.log(`VM stdout: ${data}`);
});
Likewise, you can bind to console.log, console.info, console.warn, console.error, console.dir, and console.trace. Hopefully this will save someone else some time.
I want to use node.js to manipulate docker containers. And I found a module called dockerode, which seems like the tool I need.
However, I haven't found any way to add --health-cmd and --health-interval options to createContainer function.
These 2 options are used in docker run command. You can find them by docker run --help | grep health:
--health-cmd string Command to run to check health
--health-interval duration Time between running the check (ms|s|m|h) (default 0s)
I've tried the following code, but the 2 options don't work inside the code.
const Docker = require('dockerode')
const docker = new Docker({socketPath: '/var/run/docker.sock'})
docker.createContainer({
Image: 'mysql:5.7',
name: 'myContainer',
env: ['MYSQL_ROOT_PASSWORD=mypassword'],
'health-cmd': 'mysqladmin ping --silent',
'health-interval': '1s',
HostConfig: {
PortBindings: {
'3306/tcp': [
{
HostPort: '3306'
}
]
}
}
}, (err, container) => {
if (err) {
console.error(`create err:${err}`)
} else {
container.start((err, data) => {
if (err) {
console.error(`start err: ${err}`)
} else {
console.log(`data: ${data}`)
}
})
}
})
Thanks to the help of this post. I have found the document of Docker Engine API and figured out the correct way to add the 2 options:
const Docker = require('dockerode')
const docker = new Docker({socketPath: '/var/run/docker.sock'})
docker.createContainer({
Image: 'mysql:5.7',
name: 'myContainer',
env: ['MYSQL_ROOT_PASSWORD=mypassword'],
Healthcheck: {
Test: ['CMD-SHELL', 'mysqladmin ping'],
Interval: 1000000000
},
HostConfig: {
PortBindings: {
'3306/tcp': [
{
HostPort: '3306'
}
]
}
}
}, (err, container) => {
if (err) {
console.error(`create err:${err}`)
} else {
container.start((err, data) => {
if (err) {
console.error(`start err: ${err}`)
} else {
console.log(`data: ${data}`)
}
})
}
})
You can find that createContainer calls POST /containers/create from the source code located in node_modules/dockerode/lib/docker.js.
When I write the following code in jupyter opened from my system, it is working, but when I open it from another system it is not. The problem is with require(it is not running). I have put some consoles to see where it is stopping and it is running till console.log(require.s.contexts._.defined). I have checked the modules with this console and angular seems to be missing in this. Thanks in advance.
%%javascript
require.config({
paths: {
velocity: "https://cdn.jsdelivr.net/velocity/1.2.3/velocity.min",
interact: "https://cdn.jsdelivr.net/interact.js/1.2.6/interact.min",
angular: "https://cdnjs.cloudflare.com/ajax/libs/angular.js/1.5.8/angular.min"
},
shim: {
'angular': {
exports: 'angular'
}
}
});
function start_application(data) {
// console.log("data ready")
console.log(require.s.contexts._.defined)
require(['angular', 'jquery'], function(angular,$) {
console.log("require running")
$.ajax({url: "http://localhost:8889/tree/Jupyter_Angular/DQAForm.html",
success: function(result) {
// console.log("SUCCESS")
// console.log(result);
$("#DQAFormContainer").html(result)
var el = document.getElementById("dqaBrickContent");
if(angular.element(el).injector()){
angular.element(el).injector().get('$rootScope').$destroy()
}
var dqaBrick = angular.module('dqaBrick', []);
dqaBrick.controller('dqaBrickCtrl', ['$scope', function ($scope) {
$scope.myVariable = 'Starting new DQA Brick';
console.log($scope.myVariable);
$scope.showPanel = 1;
$scope.openPanel = function (panelNum) {
$scope.showPanel = panelNum;
// console.log(panelNum);
}
}]);
angular.element(document).ready(function() {
angular.bootstrap(el, ['dqaBrick']);
});
}
});
})
}
var callbacks = {
iopub : {
output : start_application
}
}
var kernel = IPython.notebook.kernel
kernel.execute('print("This is the starting")', callbacks)
I'm running Nightwatch after launching a child process that starts up my local servers. Nightwatch runs the tests, they complete successfully, and the browser windows all close, but the nightwatch process continues to run after printing the message "OK. 10 total assertions passed.".
I thought it may have something to do with how I'm watching events on the nightwatch process, but as far as I can tell I am watching all events that would indicate Nightwatch is exiting.
The method shutdown() in runner.js is never called. How do I get Nightwatch to terminate when the tests finish?
Update
If I remove the last test in sign-in.js then Nightwatch exits as expected.
runner.js
import spawn from 'cross-spawn'
// 1. start the dev server using production config
process.env.NODE_ENV = 'testing'
let servers
function shutdown (result) {
console.log('HERE', result)
try {
// Passing a negative PID to kill will terminate all child processes, not just the parent
if (servers) process.kill(-servers.pid)
} catch (e) {
console.error('Unable to shutdown servers, may need to be killed manually')
}
if (result) {
console.error(result)
process.exit(1)
} else {
process.exit(0)
}
}
function watch (child) {
child.on('close', shutdown)
child.on('disconnect', shutdown)
child.on('error', shutdown)
child.on('exit', shutdown)
child.on('uncaughtException', shutdown)
}
try {
servers = spawn('yarn', ['run', 'dev-all'], { cwd: '..', stdio: 'inherit', detached: true })
watch(servers)
// 2. run the nightwatch test suite against it
// to run in additional browsers:
// 1. add an entry in test/e2e/nightwatch.conf.json under "test_settings"
// 2. add it to the --env flag below
// or override the environment flag, for example: `npm run e2e -- --env chrome,firefox`
// For more information on Nightwatch's config file, see
// http://nightwatchjs.org/guide#settings-file
var opts = process.argv.slice(2)
if (opts.indexOf('--config') === -1) {
opts = opts.concat(['--config', 'e2e/nightwatch.conf.js'])
}
if (opts.indexOf('--env') === -1) {
opts = opts.concat(['--env', 'chrome'])
}
var runner = spawn('./node_modules/.bin/nightwatch', opts, { stdio: 'inherit' })
watch(runner)
watch(process)
} catch (error) {
shutdown(error)
}
nightwatch.conf.js
require('babel-register')
var config = require('../../frontend/config')
// http://nightwatchjs.org/guide#settings-file
module.exports = {
src_folders: ['e2e/specs'],
output_folder: 'e2e/reports',
custom_assertions_path: ['e2e/custom-assertions'],
selenium: {
start_process: true,
server_path: 'node_modules/selenium-server/lib/runner/selenium-server-standalone-3.0.1.jar',
host: '127.0.0.1',
port: 4444,
cli_args: {
'webdriver.chrome.driver': require('chromedriver').path
}
},
test_settings: {
default: {
selenium_port: 4444,
selenium_host: 'localhost',
silent: true,
globals: {
devServerURL: 'http://localhost:' + (process.env.PORT || config.dev.port)
}
},
chrome: {
desiredCapabilities: {
browserName: 'chrome',
javascriptEnabled: true,
acceptSslCerts: true
}
},
firefox: {
desiredCapabilities: {
browserName: 'firefox',
javascriptEnabled: true,
acceptSslCerts: true
}
}
}
}
sign-in.js (one of the tests)
import firebase from 'firebase-admin'
import uuid from 'uuid'
import * as firebaseSettings from '../../../backend/src/firebase-settings'
const PASSWORD = 'toomanysecrets'
function createUser (user) {
console.log('Creating user', user.uid)
let db = firebase.database()
return Promise.all([
firebase.auth().createUser({
uid: user.uid,
email: user.email,
emailVerified: true,
displayName: user.fullName,
password: PASSWORD
}),
db.ref('users').child(user.uid).set({
email: user.email,
fullName: user.fullName
}),
db.ref('roles').child(user.uid).set({
instructor: false
})
])
}
function destroyUser (user) {
if (!user) return
console.log('Removing user', user.uid)
let db = firebase.database()
try { db.ref('roles').child(user.uid).remove() } catch (e) {}
try { db.ref('users').child(user.uid).remove() } catch (e) {}
try { firebase.auth().deleteUser(user.uid) } catch (e) {}
}
module.exports = {
'Sign In links exist': browser => {
// automatically uses dev Server port from /config.index.js
// default: http://localhost:8080
// see nightwatch.conf.js
const devServer = browser.globals.devServerURL
browser
.url(devServer)
.waitForElementVisible('#container', 5000)
browser.expect.element('.main-nav').to.be.present
browser.expect.element('.main-nav a[href^=\'https://oauth.ais.msu.edu/oauth/authorize\']').to.be.present
browser.expect.element('.main-nav a[href^=\'/email-sign-in\']').to.be.present
browser.end()
},
'Successful Sign In with Email shows dashboard': browser => {
const devServer = browser.globals.devServerURL
firebase.initializeApp(firebaseSettings.appConfig)
let userId = uuid.v4()
let user = {
uid: userId,
email: `${userId}#test.com`,
fullName: 'Test User'
}
createUser(user)
browser.url(devServer)
.waitForElementVisible('.main-nav a[href^=\'/email-sign-in\']', 5000)
.click('.main-nav a[href^=\'/email-sign-in\']')
.waitForElementVisible('button', 5000)
.setValue('input[type=text]', user.email)
.setValue('input[type=password]', PASSWORD)
.click('button')
.waitForElementVisible('.main-nav a[href^=\'/sign-out\']', 5000)
.end(() => {
destroyUser(user)
})
}
}
After the tests complete successfully, I see the following:
grimlock:backend egillespie$ ps -ef | grep nightwatch
501 13087 13085 0 1:51AM ttys000 0:02.18 node ./node_modules/.bin/nightwatch --presets es2015,stage-0 --config e2e/nightwatch.conf.js --env chrome
I was not explicitly closing the Firebase connection. This caused the last test to hang indefinitely.
Here's how I am closing the connection after doing test cleanup:
browser.end(() => {
destroyUser(user).then(() => {
firebase.app().delete()
})
})
The destroyUser function now looks like this:
function destroyUser (user) {
if (!user) return Promise.resolve()
let db = firebase.database()
return Promise.all([
db.ref('roles').child(user.uid).remove(),
db.ref('users').child(user.uid).remove(),
firebase.auth().deleteUser(user.uid)
])
}
In my case (nightwatch with vue/vuetify) after each test like so:
afterEach:function(browser,done){
done();
}
AfterAll(async()=>{
await closeSession();
await stopWebDriver();
}
place this in the config file #Erik Gillespie
Nightwatch still have this issue with the browser.end()
If you run Nightwatch with node.js you can stop the process
by doing something like that:
browser.end(() => {
process.exit();
});
It will close the browser and end the process.
I have tried the following method:
In "nightwatch.conf.js",
"test_settings" {
"default" {
"silent": true,
...
},
...
}
I set "silent" from true to false.
It lead to becoming verbose in the console. And the chromedriver.exe will exit peacefully after running the tests
I was using the vue template from: https://github.com/vuejs-templates/pwa
My platform:
Windows 7 (64bit)
node v8.1.3
"nightwatch": "^0.9.16",
"selenium-server": "^3.6.0",
"chromedriver": "^2.33.1"