This Meteor code uses "puppeteer 8.0.0", "puppeteer-core 10.0.0", puppeteer-extra 3.1.18" and "puppeteer-extra-plugin-stealth 2.7.8", It gives this error:
Error: Could not find expected browser (chrome) locally. Run npm install to download the correct Chromium revision (884014).
Tried "npm install" for no avail. Reading up online, tried removing "puppeteer-core": "^10.0.0" from package.json dependencies for no avail.
Any help is much appriciated. Thanks
const puppeteer = require('puppeteer-extra');
const nameH = require('./NameH');
const puppeteerOptions = {
headless: true,
ignoreHTTPSErrors: true,
args: ['--no-sandbox', '--single-process', '--no-zygote', '--disable-setuid-sandbox']
}
let browser;
let pageNameH;
const init = async () => {
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
console.log('1') //>>>>>>>>>>>> Prints 1
puppeteer.use(StealthPlugin());
console.log('2') //>>>>>>>>>>>> Prints 2
browser = await puppeteer.launch(puppeteerOptions);
console.log('3') //>>>>>>>>> DID NOT PRINT <<<<<<<<<<<<<<<
pageNameH = await browser.newPage();
console.log('4')
await pageNameH.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36');
await pageNameH.setViewport({ width: 1366, height: 768 });
await pageNameH.setRequestInterception(true);
blockResources(pageNameH);
}
const blockResources = page => {
page.on('request', (req) => {
if (req.resourceType() == 'stylesheet' || req.resourceType() == 'font' || req.resourceType() == 'image') {
req.abort();
}
else {
req.continue();
}
});
}
export const abc = async (nm, loc) => {
try {
console.log('name try') //>>>>>>>>>>>> Prints "name try"
if (!(browser && pageNameH))
await init();
//use "required" nameh here
} catch (error) { // print the error <<<<<<<<<<<<<<<<<<<<<<<<<
console.log("Could not launch Puppeteer or open a new page.\n" + error);
if (browser && browser.close === 'function') await browser.close();
}
}
// included in package.json
"dependencies": {
"#babel/runtime": "^7.11.2",
"axios": "^0.21.1",
"check": "^1.0.0",
"cheerio": "^1.0.0-rc.6",
"jquery": "^3.5.1",
"meteor-node-stubs": "^1.0.1",
"nightmare": "^3.0.2",
"pending-xhr-puppeteer": "^2.3.3",
"puppeteer": "^8.0.0",
"puppeteer-core": "^10.0.0",
"puppeteer-extra": "^3.1.18",
"puppeteer-extra-plugin-adblocker": "^2.11.11",
"puppeteer-extra-plugin-block-resources": "^2.2.9",
"puppeteer-extra-plugin-stealth": "^2.7.8"
},
may be you can try this, it works for me on linux(centOS), puppeteer(10.2.0).
cd ./node_modules/puppeteer
npm run install
If that fails, you can also try running:
cd ./node_modules/puppeteer
npm install
This will download the chromium to ./node_modules/puppeteer/.local-chromium
I had the same problem. I checked my env variables, and even though PUPPETEER_SKIP_CHROMIUM_DOWNLOAD was set to false, it was still not working. After I removed the variable (unset PUPPETEER_SKIP_CHROMIUM_DOWNLOAD for mac), it worked.
related dependancies:
"dependencies": {
"chrome-aws-lambda": "^10.0.0",
"puppeteer-core": "^10.0.0",
},
"devDependencies": {
"puppeteer": "^10.0.0",
}
Launching Chromium:
import chromium from "chrome-aws-lambda";
const browser = await chromium.puppeteer.launch({
executablePath: await chromium.executablePath,
});
Fixed it by running this command to install chromium manually.
node node_modules/puppeteer/install.js
if you are testing locally, make sure you have puppeteer installed as dev dependencies. specifically
npm install puppeteer --save-dev
https://github.com/alixaxel/chrome-aws-lambda/wiki/HOWTO:-Local-Development#workaround
this approach allows us to rely on puppeteer for local development and puppeteer-core for production deployments.
I had the same issue. What worked for me was to specify as the executablePath Puppeteer launch option the fullpath to the local chromium used by Puppeteer.
Something like this:
const launchOptions = {
// other options (headless, args, etc)
executablePath: '/home/jack/repos/my-repo/node_modules/puppeteer/.local-chromium/linux-901912/chrome-linux/chrome'
}
As noted in another answer, it seems that also referencing a chromium local binary would work, but I think it's a worse solution, since Puppeteer is guaranteed to work only with the local bundled version of Chromium.
Throwing my answer in, in hopes that it helps someone not waste their entire evening like I did.
I was writing a Typescript server that used Puppeteer, and I'm using ESBuild to transpile from TS to JS. In the build step, esbuild was trying to bundle everything into one file, but I had to instruct it to preserve the Puppeteer import from node_modules.
I did this by marking puppeteer as external. See docs here.
Since npm i downloads a compatible version of chromium to the node_modules folder, once I preserved this import, it was able to find Chromium in the node_modules folder.
My build file looks like:
require("esbuild").buildSync({
entryPoints: ["src/index.ts"],
outdir: "build",
bundle: true,
platform: "node",
target: "node16",
external: ["puppeteer"],
});
And I run it with node prod-build.js.
Now in my code, I can just call launch!
const browser = await puppeteer.launch()
I used the installed version on my pc (maybe not what you're looking for)
const browser = await puppeteer.launch({headless:false, executablePath:
'C:/Program Files/.../chrome.exe' });
You may need to install some dependencies depending on your OS. Check Puppeteer's Troubleshooting page for more details.
I was having this error too and I noticed that I started getting such an error after updating my node from my Dockerfile to version ^16 (my puppeteer is in a container). How did I solve it? I downgraded my node from Dockerfile to version 12.
Hope this resolves it...
OBS: I use Ubuntu 21.04 in machine.
EDIT ------
In newest versions of node, you need to go in ./node_modules/puppeteer and run command npm install, then the correct packages will be installed.
I'm using that solution.
If you are using puppeteer in AWS SAM and you don't have puppeteer in dependencies, you can install the same in puppeteer-core using
node node_modules/puppeteer/install.js
For this setup to work you will have to add chrome-aws-lambda in the devDependencies.
"devDependencies": {
"chrome-aws-lambda": "^10.1.0"
}
Also before you take it to production don't forget to add the layer in your template.yaml file:
Layers:
- !Sub 'arn:aws:lambda:ap-south-1:764866452798:layer:chrome-aws-lambda:25'
Related
I have written this small typescript hello world example
import axios from 'axios';
import { wrapper } from 'axios-cookiejar-support';
import { CookieJar } from 'tough-cookie';
const jar = new CookieJar();
const client = wrapper(axios.create({ jar }));
client.get('https://example.com');
when I run this from webstorm i get the following error
/usr/bin/node /usr/local/lib/node_modules/ts-node/dist/bin.js /home/nayana/WebstormProjects/hello-world/hello.ts
Error: Cannot find module 'node:url'
anyone have idea on how to resolve this?
I already tried npm install node:url and url
i have isolated the error to this line
const client = wrapper(axios.create({ jar }));
The issue maybe is related to the node version.
The axios-cookiejar-support requires a specific node version ("node": ">=14.18.0 <15.0.0 || >=16.0.0").
Check node --version and package-lock.json.
Sample:
"node_modules/axios-cookiejar-support": {
"version": "4.0.3",
"resolved": "https://registry.npmjs.org/axios-cookiejar-support/-/axios-cookiejar-support-4.0.3.tgz",
"integrity": "sha512-fMQc0mPR1CikWZEwVC6Av+sD4cJuV2eo06HFA+DfhY54uRcO43ILGxaq7YAMTiM0V0SdJCV4NhE1bOsQYlfSkg==",
"dependencies": {
"http-cookie-agent": "^4.0.2"
},
"engines": {
"node": ">=14.18.0 <15.0.0 || >=16.0.0"
},
"peerDependencies": {
"axios": ">=0.20.0",
"tough-cookie": ">=4.0.0"
}
},
You might need to install a later version of node.js.
I was running 14.17.6 and after installing 16.17.0 with nvm then I was able to run the project.
If you have nvm installed you can install a specific version of node e.g.
nvm install 16.17.0
make sure the types array in your tsconfig.json file contains "node"
{
"compilerOptions": {
"types": [
// ... your other types
"node"
],
// ... your other settings
},
}
The only thing you need to do, if you didn't install typescript is to change in the vite.config.js file, the import line like this:
import { fileURLToPath, URL } from 'node:url'
To:
import { fileURLToPath, URL } from 'url'
I am using Cypress (version:10+) + Cucumber+ Typescript. I need to run the test using tags. Also, I tried cypress-tag but it's not working. Is there a way I can run the cypress test using tags without skipping the test?
You can refer to this sample repository for your setup check it here:
https://github.com/badeball/cypress-cucumber-preprocessor/tree/master/examples/browserify-ts
in your cypress.config.ts
import { defineConfig } from "cypress";
import { addCucumberPreprocessorPlugin } from "#badeball/cypress-cucumber-preprocessor";
import browserify from "#badeball/cypress-cucumber-preprocessor/browserify";
async function setupNodeEvents(
on: Cypress.PluginEvents,
config: Cypress.PluginConfigOptions
): Promise<Cypress.PluginConfigOptions> {
await addCucumberPreprocessorPlugin(on, config);
on(
"file:preprocessor",
browserify(config, {
typescript: require.resolve("typescript"),
})
);
// Make sure to return the config object as it might have been modified by the plugin.
return config;
}
export default defineConfig({
e2e: {
specPattern: "**/*.feature",
supportFile: false,
setupNodeEvents,
},
});
in your package.json should contain the following dependencies and important to set cypress-cucumber-preprocessor settings "filterSpecs: true" and "omitFiltered: true" to run successfully through tags
{
"dependencies": {
"#badeball/cypress-cucumber-preprocessor": "latest",
"#cypress/browserify-preprocessor": "latest",
"cypress": "latest",
"typescript": "latest"
},
"cypress-cucumber-preprocessor": {
"filterSpecs": true,
"omitFiltered": true
}
}
then you can run your feature files like this:
cypress run --env tags=#foo
Best solution to it is the Cucumber Cypress preprocessor. I was able run my test using tags without any issue. The problem I faced in Cypress version 10 was the Itegration folder in the Cypress folder structure was renamed to e2e folder. And in Cucumber-Cypress-preprocessor will always look for files in integration folder (which was there in Cypress version less than 10) for searching tags.
I think the better solution is cypress-grep you can check about cypress-grep in the follow link https://github.com/cypress-io/cypress-grep
I'm trying to run some simple tests on a NodeJS app, using Selenium and the Brave browser.
I'm running Big Sur (11.5.2), using Brave version:
Version 1.28.105 Chromium: 92.0.4515.131 (Official Build) (x86_64)
I've downloaded the ChromeDriver, and I've installed in my path:
> which chromedriver
/Users/dd/bin/scripts/chromedriver
> chromedriver --version
ChromeDriver 92.0.4515.107 (87a818b10553a07434ea9e2b6dccf3cbe7895134-refs/branch-heads/4515#{#1634})
I've created a folder, yarn inited, added the selenium-webdriver as a dev dependency,
{
"name": "seleniumtest",
"version": "1.0.0",
"description": "seleniumtest",
"main": "index.js",
"license": "MIT",
"dependencies": {
"cors": "^2.8.5",
"express": "^4.17.1"
},
"devDependencies": {
"eslint": "^7.32.0",
"jest": "^27.0.6",
"nodemon": "^2.0.12",
"selenium-webdriver": "^4.0.0-beta.4"
},
"scripts": {
"start": "node ./index.js"
}
}
run up the following Javascript file, index.js:
const { Builder, By, Key, until } = require('selenium-webdriver');
(async function example() {
let driver = await new Builder().forBrowser('brave').build();
try {
await driver.get('http://www.google.com/');
await driver.findElement(By.name('q')).sendKeys('webdriver', Key.RETURN);
await driver.wait(until.titleIs('webdriver - Google Search'), 1000);
} finally {
await driver.quit();
}
})();
I've fired up the chromedriver:
> chromedriver
Starting ChromeDriver 92.0.4515.107 (87a818b10553a07434ea9e2b6dccf3cbe7895134-refs/branch-heads/4515#{#1634}) on port 9515
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
but when I run yarn start I get :
UnhandledPromiseRejectionWarning: Error: Do not know how to build driver: browser; did you forget to call usingServer(url)?
If I replace Builder().forBrowser('brave').build(); with Builder().forBrowser('chrome').build(); then Chrome fires up, but the window just sits there trying to load data:, in the url bar. I tried chromium - same result as for brave.
What am I doing wrong?
You can use setChromeBinaryPath function and point the brave binary path to it:
const chrome = require('selenium-webdriver/chrome');
(async function helloSelenium(){
let options = await new chrome.Options();
options.setChromeBinaryPath('/opt/brave.com/brave/brave-browser');
in the forBrowser('brave') replace it with 'chrome', as selenium will do a switch case with that and only the values in Browser enum will be valid (chrome, edge, firefox, internet_explorer, safari, opera);
I made a nodeJS app using puppeteer-extra and some puppeteer-extra-plugins, debugging the source code works just fine. It also runs OK when I pack it into an .exe file using https://www.npmjs.com/package/pkg.
However, when the .exe is moved to a different directory than the 'output' directory, I'm getting the error
Cannot find module 'debug'
1) If you want to compile the package/file into executable, please pay attention to compilation warnings
and specify a literal in 'require' call. 2) If you don't want to compile the package/file into executable
and want to 'require' it from filesystem (likely plugin), specify an absolute path in 'require' call
using process.cwd() or process.execPath.
I never use the debug module, it must be required internally by some of my dependencies... but how can I handle this error?
I've got these requires
const puppeteer = require(path.join(process.cwd(), 'dependencies', 'puppeteer-extra'));
const StealthPlugin = require(path.join(process.cwd(), 'dependencies', 'puppeteer-extra-plugin-stealth'));
const RecaptchaPlugin = require(path.join(process.cwd(), 'dependencies', 'puppeteer-extra-plugin-recaptcha'));
const UserPrefsPlugin = require(path.join(process.cwd(), 'dependencies', 'puppeteer-extra-plugin-user-preferences'));
and my package.json (config for pkg)
"pkg": {
"scripts": [
"node_modules/puppeteer/lib/*.js"
],
"assets": [
"./node_modules/#types",
"./node_modules/typescript/lib/*.d.ts",
"src/**/*.ts",
"./tsconfig.json"
],
"targets": [
"node8-win32"
]
}
I solved it by changing my requires to
const puppeteer = require('puppeteer-extra');
const RecaptchaPlugin = require('puppeteer-extra-plugin-recaptcha');
const UserPrefsPlugin = require('puppeteer-extra-plugin-user-preferences');
and later, launching the browser setting explicitly the executablePath in puppeteer's options.
const options = {
headless: false,
devtools: false,
args: ['--start-maximized'],
timeout: 300000,
downloadPath: path.join(process.env.USERPROFILE, 'Downloads'),
askDownload: false,
executablePath: path.join(process.cwd(), "puppeteer-extra",
"node_modules\\puppeteer\\.local-chromium\\win64-722234\\chrome-win\\chrome.exe")
}
await puppeteer.launch(this.options)
Note however, that I had to leave out the 'puppeteer-extra-plugin-stealth', because it complained about
"A plugin listed
'puppeteer-extra-plugin-stealth/evasions/chrome.runtime' as
dependency, which is currently missing. Please install it:"
and adding a pkg/asset as suggested here, did not help
I am getting the below error:
bundle.js:1 Uncaught Error: Cannot find module '/node_modules\mongodb-ore\lib\topologies/../../package.json'
Here is my version detial:
OS: Windows10
MongoDB: 2.2.16
MongoDB-core: 2.1.2
Node: 6.9.2
I have used npm install bson-ext and changed \node_modules\mongodb-core\node_modules\bson-ext\ext\index.js to
try {
// Load the precompiled win32 binary
if(process.platform == "win32" && process.arch == "x64") {
bson = require('bson');
} else if(process.platform == "win32" && process.arch == "ia32") {
bson = require('bson');
} else {
bson = require('bson');
}
} catch(err) {
console.log(err)
// Attempt to load the release bson version
try {
bson = require('bindings')('bson.node');
} catch (err) {
throw new Error("js-bson: Failed to load c++ bson extension, using pure JS version");
}
}
while the original is:
bson = require('./win32/x64/bson');
because when I try browserify range.js > bundle.js, it cannot find bson-ext module in mongoDB-core.
I am not sure whether this kind of operation may cause the above error.
Here is my package.json file :
"dependencies": {
"browserify": "^13.1.1",
"bson": "^1.0.1",
"d3": "^4.4.0",
"express": "^4.14.0",
"hbs": "^4.0.1",
"jsdom": "^9.9.1",
"mongodb": "^2.2.16",
"mongodb-core": "^2.1.2"
}
I haven't been able to confirm this yet, but I think the problem is that MongoDB's JavaScript (Node.js) driver is not intended for use in a browser, for security reasons. Not clear if the problem in the OP is due to Browserify incorrectly resolving relative paths or something else, but regardless the preferred technique is to proxy requests to your MongoDB instance via a Node server.
Mongo lists off-the-shelf solutions for this here:
https://docs.mongodb.com/ecosystem/tools/http-interfaces/
Note also the --rest option, which allows an application to read directly from the DB via a URL schema:
https://docs.mongodb.com/ecosystem/tools/http-interfaces/#simple-rest-api
As the docs mention, this is not good practice for security concerns, but may help with prototyping.