Selenium on a Docker container - node.js

I have a simple javascript file that uses selenium to launch Chrome, open some website, and collect some data from that website. Now, I want to put that file and run it inside a Docker container.
To do this I wrote the following Dockerfile:
FROM alpine
RUN apk add --update nodejs npm chromium
COPY . /src
WORKDIR /src
RUN npm install -g chromedriver
RUN npm install
ENTRYPOINT ["node", "index.js"]
The image builds with no errors, but when I attempt to run the container from it I get:
/src/node_modules/selenium-webdriver/remote/index.js:248
reject(Error(e.message))
^
Error: Server terminated early with status 1
at /src/node_modules/selenium-webdriver/remote/index.js:248:24
at processTicksAndRejections (node:internal/process/task_queues:96:5)
There's only one dependency for my index.js file - "selenium-webdriver": "^4.1.1", and the file itself looks like this:
index.js
const {Builder, By} = require('selenium-webdriver');
(async function example() {
let driver = await new Builder().forBrowser('chrome').build();
try {
// Navigate to Url
await driver.get('https://www.example.com');
// Get all the elements available with tag 'p'
let elements = await driver.findElements(By.css('p'));
for(let e of elements) {
console.log(await e.getText());
}
}
finally {
await driver.quit();
}
})();
What am I doing wrong? How can I make the container run successfully?

I think the problem is with selenium-webdriver/chrome,
since the builder is getting stuck on mine.

Related

How can I execute a command inside a docker from a node app?

I have a node app running, and I need to access a command that lives in an alpine docker image.
Do I have to use exec inside of javascript?
How can I install latex on an alpine container and use it from a node app?
I pulled an alpine docker image, started it and installed latex.
Now I have a docker container running on my host. I want to access this latex compiler from inside my node app (dockerized or not) and be able to compile *.tex files into *.pdf
If I sh into the alpine image I can compile '.tex into *.pdf just fine, but how can I access this software from outside the container e.g. a node app?
If you just want to run the LaTeX engine over files that you have in your local container filesystem, you should install it directly in your image and run it as an ordinary subprocess.
For example, this Javascript code will run in any environment that has LaTeX installed locally, Docker or otherwise:
const { execFileSync } = require('node:child_process');
const { mkdtemp, open } = require('node:fs/promises');
const tmpdir = await mkdtemp('/tmp/latex-');
let input;
try {
input = await open(tmpdir + '/input.tex', 'w');
await input.write('\\begin{document}\n...\n\\end{document}\n');
} finally {
input?.close();
}
execFileSync('pdflatex', ['input'], { cwd: tmpdir, stdio: 'inherit' });
// produces tmpdir + '/input.pdf'
In a Docker context, you'd have to make sure LaTeX is installed in the same image as your Node application. You mention using an Alpine-based LaTeX setup, so you could
FROM node:lts-alpine
RUN apk add texlive-full # or maybe a smaller subset
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY ./ ./
CMD ["node", "main.js"]
You should not try to directly run commands in other Docker containers. There are several aspects of this that are tricky, including security concerns and managing the input and output files. If it's possible to directly invoke a command in a new or existing container, it's also very straightforward to use that permission to compromise the entire host.

Deploying Strapi V4 Serverless via Docker ECR container

I've been following this blog post:
But with a slight twist as following the above hits the 250mb limit AWS Lambda limit (no idea how the original author managed to do without).
So my alternative was to try using ECR/Docker to deploy instead.
my docker image is building and deploying to serverless fine, however when I try to access the application on a route i.e. /dev/api/users my serverless log keeps throwing this error:
INFO Cold starting Strapi
/bin/sh: hostname: command not found
[2022-10-03 17:43:49.672] debug: ⛔️ Server wasn't able to start properly.
[2022-10-03 17:43:49.674] error: The public folder (/var/task/public) doesn't exist or is not accessible. Please make sure it exists.
Error: The public folder (/var/task/public) doesn't exist or is not accessible. Please make sure it exists.
at module.exports (/var/task/node_modules/#strapi/strapi/lib/core/bootstrap.js:29:11)
at async Strapi.register (/var/task/node_modules/#strapi/strapi/lib/Strapi.js:380:5)
at async Strapi.load (/var/task/node_modules/#strapi/strapi/lib/Strapi.js:474:5)
at async startStrapi (/var/task/app.js:16:7)
at async Runtime.module.exports.strapiHandler [as handler] (/var/task/app.js:36:5)
My Dockerfile image:
FROM amazon/aws-lambda-nodejs:16
WORKDIR /var/task
RUN npm i python3
COPY src /var/task
COPY build /var/task
COPY config /var/task
COPY database /var/task
COPY public /var/task
COPY package*.json /var/task
COPY .env.example /var/task
RUN npm install
RUN chmod -R 777 /var/task
CMD [ "app.strapiHandler" ]
Handler function in app.js
module.exports.strapiHandler = async (event, context) => {
let workingDir = process.cwd();
if (process.env.LAMBDA_TASK_ROOT && process.env.IS_OFFLINE !== "true") {
workingDir = process.env.LAMBDA_TASK_ROOT;
}
if (!global.strapi) {
console.info("Cold starting Strapi");
Strapi({ dir: workingDir });
}
if (!global.strapi.isLoaded) {
await startStrapi(global.strapi);
}
const handler = serverless(global.strapi.server.app);
return handler(event, context);
};
I am copying my public directory to the /var/task folder. But the error keeps persisting. Feel like I've hit a wall and it's business critical I get this solved.
Any insight would be appreciated.
NOTE: the chmod -R 777 is temporary whilst I try and fix this. Figured it may have been a permission issue.
EDIT:
Managed to solve the above. If anyone wants to know how please reach out happy to help anyone on a similar path with Serverless and headless CMS.
New issue:
error: Forbidden access
ForbiddenError: Forbidden access
at Object.verify (/var/task/node_modules/#strapi/plugin-users-permissions/server/strategies/users-permissions.js:94:11)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async /var/task/node_modules/#strapi/strapi/lib/services/server/compose-endpoint.js:31:5
at async serve (/var/task/node_modules/koa-static/index.js:59:5)
at async returnBodyMiddleware (/var/task/node_modules/#strapi/strapi/lib/services/server/compose-endpoint.js:52:18)
at async policiesMiddleware (/var/task/node_modules/#strapi/strapi/lib/services/server/policy.js:24:5)
at async /var/task/node_modules/#strapi/strapi/lib/middlewares/body.js:58:9
at async /var/task/node_modules/#strapi/strapi/lib/middlewares/logger.js:25:5
at async /var/task/node_modules/#strapi/strapi/lib/middlewares/powered-by.js:16:5
at async cors (/var/task/node_modules/#koa/cors/index.js:61:32)
I'm a super user so not letting me access my own /api/users route.
EDIT 2:
Also managed to solve the Forbidden access issue.
My latest issue is that when I go to content-builder-plugin in the admin is saying that it needs autoReload to be enabled. Anyone know how to config so that this is enabled?
Forbidden access issue was solved by having the following Dockerfile structure - I figured out in the end the directories need to be mapped explicitly to their Docker container folders for the admin to work properly - here is my Dockerfile example - I also had to add a few npm install commands in order to get it running.
FROM amazon/aws-lambda-nodejs:16
WORKDIR /var/task
RUN npm i python3
COPY src /var/task/src
COPY build /var/task/build
COPY config /var/task/config
COPY database /var/task/database
COPY public /var/task/public
COPY package*.json /var/task
COPY .env.example /var/task
RUN npm install
RUN npm install --platform=linux --arch=x64 sharp
RUN chmod -R 755 /var/task
CMD [ "/var/task/src/app.strapiHandler" ]
Any questions let me know! There are also some serverless.yml tweaks and strapi security config required in order to get images and media uploading/working properly.

Is it possible to run Selenium like tests in a Windows Docker Container

we have a Windows Electron application that runs e2e Tests via Spectron. The application is platform-dependent and won't run on Linux (Containers). We want to run our Spectron e2e Tests inside a preconfigured Docker container to have them isolated.
To get a grasp of it I have built a minimal nodejs application that does basically nothing and has an e2e test (jest) that opens a browser tab and checks the title, no functionality just a simple spike.
I created a Dockerfile to build a container to run the tests:
FROM mcr.microsoft.com/windows:20H2-amd64
RUN mkdir "C:/app"
WORKDIR "C:/app"
COPY app "C:/app"
RUN powershell -Command \
Set-ExecutionPolicy unrestricted;
ENV chocolateyUseWindowsCompression false
RUN powershell -Command \
iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'));
RUN choco install googlechrome -y --version=91.0.4472.101 --ignore-checksums
RUN choco install chromedriver -y --version=91.0.4472.1010 --ignore-checksums
RUN choco install nodejs-lts -y --version=14.17.1
RUN npm config set strict-ssl false
RUN npm install
ENTRYPOINT npm test
Note this is a Windows container, as our main app will also need a Windows container to run. The container builds and runs the test but crashes with the error: SessionNotCreatedError: session not created thrown by from tab crashed. On my Windows Host, the test runs fine.
Is there anything wrong with my Dockerfile or is this simply not possible in a Windows container?
I don't think it's relevant to the problem but here is also the test file that gets executed when the container does npm test:
describe('test google.com', () => {
const {
Builder,
By,
Key,
until
} = require('selenium-webdriver');
var driver;
beforeEach(() => {
driver = new Builder()
.forBrowser('chrome')
.build();
});
afterEach(() => {
driver.quit();
});
it('should open google search', async () => {
await driver.get('http://www.google.com');
driver
.getTitle()
.then(title => {
expect(title).toEqual('Google');
});
});
});
We had a similar problem, but we are using .net-core with Selenium. For some reason, installing the Chromedriver did not work inside container, so we had to do two things:
manually download the driver based on the chrome version and export the zip into the working directory. (It's been a while though, and we did not really update the image, installing via choco may be working now)
Even stranger thing is that we had to install some fonts for some reason.
Take look at my repo: https://github.com/yamac-kurtulus/Windows-Docker-Images/tree/master/DotnetCore%20Selenium%20With%20Chrome
The relevant part is after line 23 in the Dockerfile.
Note: If you are not very deep into the project, I strongly suggest you to migrate to Linux. Working with Docker on Windows is like a nightmare that you cannot wake up from.

NPM and NodeJS Compatibility: NodeJS works from PM prompt, but not script

I am attempting to get a lighthouse script running in Node.JS (which I am new to). I followed the intial instructions here https://github.com/GoogleChrome/lighthouse/blob/master/docs/readme.md#using-programmatically. I was able to complete the prior steps in the package manager console (Visual Studio 2017):
npm install -g lighthouse
lighthouse https://airhorner.com/
//and
lighthouse https://airhorner.com/ --output=json --output-path=./report/test1.json
However, I do get an initial warning that NPM only supports Node.JS in versions 4 through 8 and recommends a newer version. The problem is I am running Node v12 and NPM v5 - both the latest.
When I create a script version like below (app.js)
const lighthouse = require('lighthouse');
const chromeLauncher = require('chrome-launcher');
const config = {
extends: 'lighthouse:default',
settings: {
emulatedFormFactor: 'desktop',
onlyCategories: 'performance',
output: 'json',
outputPath: './report.json'
}
};
function launchChromeAndRunLighthouse(url, opts = null, config) {
return chromeLauncher.launch().then(chrome => {
opts.port = chrome.port;
return lighthouse(url, opts, config).then(results => {
return chrome.kill().then(() => results.lhr);
});
});
}
// Usage:
launchChromeAndRunLighthouse('https://airhorner.com/', config).then(results => {
// Use results!
});
And run the command
C:\src\project> node app.js
I get the error - Cannot find module 'lighthouse'
don't install lighthouse locally use it inside your working dir .
first start by running npm init that will create the package.json file inside the current working dir
then npm install --save lighthouse will download it and save it to node_modules now you can use it locally inside your working dir
it should look something like this
app.js
package.json
node_modules/
then run node app.js

How to execute non-global npm binary in docker

Suppose I have following Dockerfile
WORKDIR $APP_DIR
ENTRYPOINT ["npm", "run"]
CMD ["start"]
The start is mapped to babel-node bin/server, where babel-node is a nodejs non-global dependency binary file (installed inside $APP_DIR/node_modules by default)
On my mac, without any set up, this works fine. But when I run it inside docker, it shows command not found, which is not surprise.
So the same command npm run start works on my host machine but inside Docker.
sh: 1: babel-node: not found
error Command failed with exit code 127.
you can try in this way: CMD [./node_modules/babel-node/bin/babel-node your app.js];
or try to use another way:
// you can rename this file to bin.js
const fs = require('fs');
const babelrc = fs.readFileSync('.babelrc', 'utf-8');
let config = {};
try {
config = JSON.parse(config);
} catch(err) {
console.error('==> ERROR: Error parsing your .babelrc.');
}
require("babel-polyfill");
require('babel-register')(config);
// then require your app.js
require('./app.js);
then you Dockerfile will be look like as follow:
...other codes
CMD [node bin.js]

Resources