I'm working with puppeteer. I want to get the node value of a selected element in a textbox . Using dev tools I have copied the selector:
var mySelector = "div.chosen-container.chosen-container-multi.filter-main-values.fmd-values.chosen-container-active.chosen-with-drop > ul > li.search-choice > span";
I can use this to find it in devtools:
But after running my puppeteer code:
var selectedCountry = await page.evaluate((mySelector) => {return Array.from(document.querySelectorAll(mySelector))});
console.log(selectedCountry);
In the vs code debug window, I see:
(0) []
What am I doing wrong?
You are not passing the selector to the evaluate function properly.
var selectedCounty = await page.evaluate((selectedCountySelector) => {
return Array.from(document.querySelectorAll(selectedCountySelector))
}, selectedCountySelector); // <-- pass it here
console.log(selectedCounty);
Related
I have a Question ?
I have a File Which contains one keyword on each line (5000),
I'm developing a Scraper Using Puppeteer in Nodes Which will go to a website which has a Search Bar and in that Search bar I want to search Using the Keywords from that File, So Someone Please Guide me How to accomplish this ? am I Using the right tools ? I would be grateful .
You can use:
// yourVariable is the text data.
if(yourVariable.contains("What you are looking for.")){
// the code
}
If that is what you mean.
Otherwise, if you mean going to each 5000 lines in the text file, you can use nthline in an async function by running:
npm i nthline --save
in your console, then:
;(async () => {
const nthline = require('nthline'),
filePath = '/path/to/100-million-rows-file',
rowIndex = 42
console.log(await nthline(rowIndex, filePath))
})()
Or if you want to check if the line contains data:
;(async () => {
const nthline = require('nthline'),
filePath = '/path/to/100-million-rows-file',
rowIndex = 42
console.log(await nthline(rowIndex, filePath).contains("Data"))
})()
In my new React Native app, I want to add some Jest tests.
One component renders a background image, which is located directly in the project in assets folder.
Now I stumbled about how to test if this image is actually taken from this path, therefore present in the component, and rendered correctly.
I tried using toHaveStyle from #testing-library/jest-native with a container, which returned the error toHaveStyleis not a function. Then I tried the same with queryByTestId, same error. When I do expect(getByTestId('background').toBeInTheDocument); then I feel this is useless, because it only checks if an element with this testId is present, but not the image source.
Please, how can I test this? Does it actually make sense to test an image source after all?
Here is my code:
1.) The component that should be tested (Background):
const Background: React.FC<Props> = () => {
const image = require('../../../../assets/images/image.jpg');
return (
<View>
<ImageBackground testID="background" source={image} style={styles.image}></ImageBackground>
</View>
);
};
2.) The test:
import React from 'react';
import {render, container} from 'react-native-testing-library';
import {toHaveStyle} from '#testing-library/jest-native';
import '#testing-library/jest-native/extend-expect';
import Background from '../Background';
describe('Background', () => {
test('renders Background image', () => {
const {getByTestId} = render(<Background></Background>);
expect(getByTestId('background').toBeInTheDocument);
/* const container = render(<Background background={background}></Background>);
expect(container).toHaveStyle(
`background-image: url('../../../../assets/images/image.jpg')`,
); */
/* expect(getByTestId('background')).toHaveStyle(
`background-image: url('../../../../assets/images/image.jpg')`,
); */
});
});
If you're using #testing-library/react rather than #testing-library/react-native, and you have an alt attribute on your image, you can avoid using getByDataTestId and instead use getByAltText.
it('uses correct src', async () => {
const { getByAltText } = await render(<MyComponent />);
const image = getByAltText('the_alt_text');
expect(image.src).toContain('the_url');
// or
expect(image).toHaveAttribute('src', 'the_url')
});
Documentation.
Unfortunately, it appears that React Native Testing Library does not include getByAltText. (Thank you, #P.Lorand!)
It's a little hard to say because we can't see <ImageBackground> component or what it does... But if it works like an <img> component we can do something like this.
Use a selector on the image component through its role / alt text / data-testid:
const { getByDataTestId } = render(<Background background={background}>
</Background>);
Then look for an attribute on that component:
expect(getByDataTestId('background')).toHaveAttribute('src', '../../../../assets/images/image.jpg')
When I used getByAltText and getByDataTestId I got is not a function error.
So what worked for me was:
const imgSource = require('../../../../assets/images/image.jpg');
const { queryByTestId } = render(<MyComponent testID='icon' source={imgSource}/>);
expect(queryByTestId('icon').props.source).toBe(imgSource);
I use #testing-library/react-native": "^7.1.0
I ran into this issue today and found that if your URI is a URL and not a required file, stitching the source uri onto the testID works nicely.
export const imageID = 'image_id';
...
<Image testID={`${imageID}_${props.uri}`} ... />
Test
import {
imageID
}, from '.';
...
const testURI = 'https://picsum.photos/200';
const { getByTestId } = render(<Component uri={testURI} />);
expect(getByTestId()).toBeTruthy();
I think that you are looking for:
const uri = 'http://example.com';
const accessibilityLabel = 'Describe the image here';
const { getByA11yLabel } = render (
<Image
source={{ uri }}
accessibilityLabel={accessibilityLabel}
/>
);
const imageEl = getByA11yLabel(accessibilityLabel);
expect(imageEl.props.src.uri).toBe(uri);
I am using Puppeteer to scrape the web from a file template that contains the data of an order.
For this, I am using a puppeteer evaluation function, which works correctly while the file is in .js
However, when the "pkg" package is used to compile the .exe file or evaluate to execute and initiate a return or error: "The passed function is not quite serializable!"
Below is the code:
const dados = {name: 'foo', year: 1}
await page.evaluate(dados => {
let dom = document.querySelector('body');
const tags = Object.keys(dados);
for (let i = 0; i < tags.length; i++) {
const tag = tags[i];
dom.innerHTML = dom.innerHTML.split(`{{${tag}}}`).join(dados[tag]);
}
}, dados);
I try to add --public argument with pkg.
Like: pkg start.js -t node14-win-x64 --public
Then I can freely use ElementHandle.evaluate( (elem)=> elem.textContent );
With the manual of pkg, the --public means that : speed up and disclose the sources of top-level project.
BTW
To fixup cannot find chrome binary file
browser = await puppeteer.launch({
executablePath: "node_modules/puppeteer/.local-chromium/win64-782078/chrome-win/chrome.exe"
});
(The path above can set anywhere as you like.)
To fixup start.exe cannot run
Sometime the output binary exe cannot executable. It
always popup a new cmd prompt window when we enter start.exe. (or just double click.)
Just delete the output exe then rerun pkg
Check the code whether is runnable with node start.js or not
The easiest solution for me is to wrap it with eval() :
async getText(selector: string) {
await this.page.waitForSelector(selector);
let text = await eval(`this.page.$eval('${selector}', el => el.textContent)`)
return text;
}
or this:
await eval(`this.page.evaluate(
(selector) => { (document.querySelector(selector).value = ''); },
selector);`);
I had this exact issue relating to puppeteer and pkg. For some reason pkg doesn't correctly interpret the contents of the callback. The solution that worked for me was to pass a string to evaluate rather than a function:
Change:
const dados = {name: 'foo', year: 1}
await page.evaluate(dados => {
let dom = document.querySelector('body');
const tags = Object.keys(dados);
for (let i = 0; i < tags.length; i++) {
const tag = tags[i];
dom.innerHTML = dom.innerHTML.split(`{{${tag}}}`).join(dados[tag]);
}
}, dados);
to
await page.evaluate(`
(() => {
const dados = {name: 'foo', year: 1};
let dom = document.querySelector('body');
const tags = Object.keys(dados);
for (let i = 0; i < tags.length; i++) {
const tag = tags[i];
dom.innerHTML = dom.innerHTML.split(`{{${tag}}}`).join(dados[tag]);
}
// return dom to do something with the data in node
return dom.innerHTML
})()`);
This answer on github suggests an alternative solution - using the pkg api to inject the callback at compile time, however it didn't work for me.
I would like to add support to async/await to node repl
Following this issue: https://github.com/nodejs/node/issues/8382
I've tried to use this one https://github.com/paulserraino/babel-repl but it is missing async await suppport
I would like to use this snippet
const awaitMatcher = /^(?:\s*(?:(?:let|var|const)\s)?\s*([^=]+)=\s*|^\s*)(await\s[\s\S]*)/;
const asyncWrapper = (code, binder) => {
let assign = binder ? `root.${binder} = ` : '';
return `(function(){ async function _wrap() { return ${assign}${code} } return _wrap();})()`;
};
// match & transform
const match = input.match(awaitMatcher);
if(match) {
input = `${asyncWrapper(match[2], match[1])}`;
}
How can I add this snippet to a custom eval on node repl?
Example in node repl:
> const user = await User.findOne();
As of node ^10, you can use the following flag when starting the repl:
node --experimental-repl-await
$ await myPromise()
There is the project https://github.com/ef4/async-repl:
$ async-repl
async> 1 + 2
3
async> 1 + await new Promise(r => setTimeout(() => r(2), 1000))
3
async> let x = 1 + await new Promise(r => setTimeout(() => r(2), 1000))
undefined
async> x
3
async>
Another option, slightly onerous to start but with a great UI, is to use the Chrome Devtools:
$ node --inspect -r esm
Debugger listening on ws://127.0.0.1:9229/b4fb341e-da9d-4276-986a-46bb81bdd989
For help see https://nodejs.org/en/docs/inspector
> Debugger attached.
(I am using the esm package here to allow Node to parse import statements.)
Then you go to chrome://inspect in Chrome and you will be able to connect to the node instance. Chrome Devtools has top-level await, great tab-completion etc.
The idea is to preprocess the command and wrap it in a async function if
there is an await syntax outside async function
this https://gist.github.com/princejwesley/a66d514d86ea174270210561c44b71ba is the final solution
I'm using the BrowserWindow to display an app and I would like to force the external links to be opened in the default browser. Is that even possible or I have to approach this differently?
I came up with this, after checking the solution from the previous answer.
mainWindow.webContents.on('new-window', function(e, url) {
e.preventDefault();
require('electron').shell.openExternal(url);
});
According to the electron spec, new-window is fired when external links are clicked.
NOTE: Requires that you use target="_blank" on your anchor tags.
new-window is now deprecated in favor of setWindowOpenHandler in Electron 12 (see https://github.com/electron/electron/pull/24517).
So a more up to date answer would be:
mainWindow.webContents.setWindowOpenHandler(({ url }) => {
shell.openExternal(url);
return { action: 'deny' };
});
Improved from the accepted answer ;
the link must be target="_blank" ;
add in background.js(or anywhere you created your window) :
window.webContents.on('new-window', function(e, url) {
// make sure local urls stay in electron perimeter
if('file://' === url.substr(0, 'file://'.length)) {
return;
}
// and open every other protocols on the browser
e.preventDefault();
shell.openExternal(url);
});
Note : To ensure this behavior across all application windows, this code should be run after each window creation.
If you're not using target="_blank" in your anchor elements, this might work for you:
const shell = require('electron').shell;
$(document).on('click', 'a[href^="http"]', function(event) {
event.preventDefault();
shell.openExternal(this.href);
});
I haven't tested this but I assume this is should work:
1) Get WebContents of the your BrowserWindow
var wc = browserWindow.webContents;
2) Register for will-navigate of WebContent and intercept navigation/link clicks:
wc.on('will-navigate', function(e, url) {
/* If url isn't the actual page */
if(url != wc.getURL()) {
e.preventDefault();
openBrowser(url);
}
}
3) Implement openBrowser using child_process. An example for Linux desktops:
var openBrowser(url) {
require('child_process').exec('xdg-open ' + url);
}
let me know if this works for you!
For anybody coming by.
My use case:
I was using SimpleMDE in my app and it's preview mode was opening links in the same window. I wanted all links to open in the default OS browser. I put this snippet, based on the other answers, inside my main.js file. It calls it after it creates the new BrowserWindow instance. My instance is called mainWindow
let wc = mainWindow.webContents
wc.on('will-navigate', function (e, url) {
if (url != wc.getURL()) {
e.preventDefault()
electron.shell.openExternal(url)
}
})
Check whether the requested url is an external link. If yes then use shell.openExternal.
mainWindow.webContents.on('will-navigate', function(e, reqUrl) {
let getHost = url=>require('url').parse(url).host;
let reqHost = getHost(reqUrl);
let isExternal = reqHost && reqHost != getHost(wc.getURL());
if(isExternal) {
e.preventDefault();
electron.shell.openExternal(reqUrl);
}
}
Put this in renderer side js file. It'll open http, https links in user's default browser.
No JQuery attached! no target="_blank" required!
let shell = require('electron').shell
document.addEventListener('click', function (event) {
if (event.target.tagName === 'A' && event.target.href.startsWith('http')) {
event.preventDefault()
shell.openExternal(event.target.href)
}
})
For Electron 5, this is what worked for me:
In main.js (where you create your browser window), include 'shell' in your main require statement (usually at the top of the file), e.g.:
// Modules to control application life and create native browser window
const {
BrowserWindow,
shell
} = require('electron');
Inside the createWindow() function, after mainWindow = new BrowserWindow({ ... }), add these lines:
mainWindow.webContents.on('new-window', function(e, url) {
e.preventDefault();
shell.openExternal(url);
});
I solved the problem by the following step
Add shell on const {app, BrowserWindow} = require('electron')
const {app, BrowserWindow, shell} = require('electron')
Set nativeWindowOpen is true
function createWindow () {
// Create the browser window.
const mainWindow = new BrowserWindow({
width: 1350,
height: 880,
webPreferences: {
nativeWindowOpen: true,
preload: path.join(__dirname, 'preload.js')
},
icon: path.join(__dirname, './img/icon.icns')
})
Add the following listener code
mainWindow.webContents.on('will-navigate', function(e, reqUrl) {
let getHost = url=>require('url').parse(url).host;
let reqHost = getHost(reqUrl);
let isExternal = reqHost && reqHost !== getHost(wc.getURL());
if(isExternal) {
e.preventDefault();
shell.openExternal(reqUrl, {});
}
})
reference https://stackoverflow.com/a/42570770/7458156 by cuixiping
I tend to use these lines in external .js script:
let ele = document.createElement("a");
let url = "https://google.com";
ele.setAttribute("href", url);
ele.setAttribute("onclick", "require('electron').shell.openExternal('" + url + "')");