I wish to connect to a website and download some pdf files. The website allows us to view the content only after log in. It asks us to log in using OTP and can't be login at more than 3 devices simultaneously.
I wish to download all the pdf listed. So I previously tried the
python playwright open --save-storage websitename.json
to save the login. But it doesn't work for that specific website.
The website.json file was empty whereas it worked for other websites.
Therefore the only solution I could think of know, is to connect to the current browser, open that website and then download those pdfs.
If you have some solution for this or even some other approach please do inform.
I was also thinking about switching over to puppeteer for the same.
But, I don't know the html parsing using node.js, since I feel using css selectors more comfortable, so I can't switch it.
Playwright is basically same as Puppeteer. So it wouldn't be a problem if you switch between the two.
You can use puppeteer-core or playwright to control your existing browser installation, for example Chrome, and then use the existing user data (Profile) folder to load the specified website login info (cookies, webstorage, etc).
const launchOptions = {
headless: false,
executablePath: '/Applications/Google Chrome/Contents/MacOS/Google Chrome', // For MacOS
// executablePath: 'C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe', // For Windows
// executablePath: '/usr/bin/google-chrome' // For Linux
args: [
'--user-data-dir=/Users/username/Library/Application Support/Google/Chrome/', // For MacOS
// '--user-data-dir=%userprofile%\\AppData\\Local\\Chrome\\User Data', // For Windows
// '--profile-directory=Profile 1' // This to select default or specified Profile
]
}
const puppeteer = require('puppeteer-core')
const browser = await puppeteer.launch(launchOptions)
For more details about Playwright's method, you can check this workaround:
https://github.com/microsoft/playwright/issues/1985
To connect to an already running browser (Chrome) session, you can use connect_over_cdp method (added in v1.9 of playwright).
For this, you need to start Chrome in debug mode. Create a desktop shortcut for Chrome and edit Target section of shortcut properties to start it with debug mode. Add --remote-debugging-port=9222 to the target box in shortcut properties so that the target path becomes:
C:\Program Files\Google\Chrome\Application\chrome.exe" --remote-debugging-port=9222
Now start Chrome and check if it is in debug mode. For this open a new tab and paste this url in the address bar: http://localhost:9222/json/version. If you are in debug mode, you should see now a page with a json response, otherwise if you are in "normal" mode, it will say "Page not found" or something similar.
Now in your python script, write following code to connect to chrome instance:
browser = playwright.chromium.connect_over_cdp("http://localhost:9222")
default_context = browser.contexts[0]
page = default_context.pages[0]
Here is the full script code:
# Import the sync_playwright function from the sync_api module of Playwright.
from playwright.sync_api import sync_playwright
# Start a new session with Playwright using the sync_playwright function.
with sync_playwright() as playwright:
# Connect to an existing instance of Chrome using the connect_over_cdp method.
browser = playwright.chromium.connect_over_cdp("http://localhost:9222")
# Retrieve the first context of the browser.
default_context = browser.contexts[0]
# Retrieve the first page in the context.
page = default_context.pages[0]
# Print the title of the page.
print(page.title)
# Print the URL of the page.
print(page.url)
Related
When I run my tests, the Developer Tools panel is automatically open. Can I close it using Playwright? I don't need to see it.
If you are using chromium.launch you can disable DevTools from showing with the following config.
const browser = await chromium.launch({
devtools: false
});
More information in the Playwright documentation: https://playwright.dev/docs/api/class-browsertype#browser-type-launch-option-devtools
I'm using Playwright + Firefox to automate downloading of a CSV file from firebase. The download is initiated with a click on the button:
page.click(".table-download-button")
Problem: There's a prompt to download the file (refer to image below). What can I do to accept the download without the prompt? I can't be clicking on the prompt since I am automating it. The same problem is not found in chromium, only firefox! (I have my reasons why firefox is needed)
I have tried: Click on "Do this automatically for files like this from now on", however it doesn't work since once I restart the script the preference has been cleared
My code for the download portion:
const [ download ] = await Promise.all([
page.waitForEvent('download'),
page.click(".table-download-button")
]);
const path = await download.path();
Appreciate your assistance!
You should use expect_download there:
async with page.expect_download() as download_info:
await page.click("a")
path = await download.path()
I'm working on a CLI with OCLIF. In one of the commands, I need to simulate a couple of clicks on a web page (using the WebdriverIO framework for that). Before you're able to reach the desired page, there is a redirect to a page with a login prompt. When I use WebdriverIO methods related to alerts such as browser.getAlertText(), browser.sendAlertText() or browser.acceptAlert, I always get the error no such alert.
As an alternative, I tried to get the URL when I am on the page that shows the login prompt. With the URL, I wanted to do something like browser.url(https://<username>:<password>#<url>) to circumvent the prompt. However, browser.url() returns chrome-error://chromewebdata/ as URL when I'm on that page. I guess because the focus is on the prompt and that doesn't have an URL. I also don't know the URL before I land on that page. When being redirected, a query string parameter containing a token is added to the URL that I need.
A screenshot of the prompt:
Is it possible to handle this scenario with WebdriverIO? And if so, how?
You are on the right track, probably there are some fine-tunings that you need to address to get it working.
First off, regarding the chrome-error://chromewebdata errors, quoting Chrome DOCs:
If you see errors with a location like chrome-error://chromewebdata/
in the error stack, these errors are not from the extension or from
your app - they are usually a sign that Chrome was not able to load
your app.
When you see these errors, first check whether Chrome was able to load
your app. Does Chrome say "This site can't be reached" or something
similar? You must start your own server to run your app. Double-check
that your server is running, and that the url and port are configured
correctly.
A lot of words that sum up to: Chrome couldn't load the URL you used inside the browser.url() command.
I tried myself on The Internet - Basic Auth page. It worked like a charm.
URL without basic auth credentials:
URL WITH basic auth credentials:
Code used:
it('Bypass HTTP basic auth', () => {
browser.url('https://admin:admin#the-internet.herokuapp.com/basic_auth');
browser.waitForReadyState('complete');
const banner = $('div.example p').getText().trim();
expect(banner).to.equal('Congratulations! You must have the proper credentials.');
});
What I'd do is manually go through each step, trying to emulate the same flow in the script you're using. From history I can tell you, I dealt with some HTTP web-apps that required a refresh after issuing the basic auth browser.url() call.
Another way to tackle this is to make use of some custom browser profiles (Firefox | Chrome) . I know I wrote a tutorial on it somewhere on SO, but I'm too lazy to find it. I reference a similar post here.
Short story, manually complete the basic auth flow (logging in with credentials) in an incognito window (as to isolate the configurations). Open chrome://version/ in another tab of that session and store the contents of the Profile Path. That folder in going to keep all your sessions & preserve cookies and other browser data.
Lastly, in your currentCapabilities, update the browser-specific options to start the sessions with a custom profile, via the '--user-data-dir=/path/to/your/custom/profile. It should look something like this:
'goog:chromeOptions': {
args: [
'--user-data-dir=/Users/iamdanchiv/Desktop/scoped_dir18256_17319',
],
}
Good luck!
steam provides links to inspect items in 3D by opening the game and the specific 3D model. Such a link looks like this:
steam://rungame/730/76561202255233023/+csgo_econ_action_preview%20S76561198808861484A14783070567D17060211998222859457
If this link is clicked in a browser, it asks confirmation to open the "Steam Client Bootstrapper" and then runs the game (or you check a box so it doesn't ask that again).
I would like to make a node script, that would open such a link (probably via chrome) and runs the game.
I tried chrome-launcher:
const chromeLauncher = require('chrome-launcher');
inspect("steam://rungame/730/76561202255233023/+csgo_econ_action_preview%20S76561198808861484A14783070567D17060211998222859457")
function inspect(link){
chromeLauncher.launch({
startingUrl: link
}).then(chrome => {
console.log(`Chrome debugging port running on ${chrome.port}`);
});
}
and also the opn module:
const opn = require('opn');
inspect("steam://rungame/730/76561202255233023/+csgo_econ_action_preview%20S76561198808861484A14783070567D17060211998222859457")
function inspect(link){
opn(link, {app: 'chrome'});
}
Both of these have the same result:
Chrome opens up
Address bar is empty
Nothing happens
Any idea on how I could do this?
Thanks in advance!
Remove the app parameter so it uses the standard browser.
I am trying to develop a chrome extension to automtize some tasks on the web (fill form, go to next page, extract data...)
The idea is to develop a kind of http://casperjs.org/ as chrome extension.
I am injecting some JS to an active tab.
JS is run, and everything works fine: problems raise when I navigate to a new url.
When the page navigates to new url (document.location for instantce), the JS stops running since it is unloaded.
My idea would be to store the JS state and resume it once page has loaded:
ex content of source variable: I inject it in background to the web page.
for (i = 0;i < 5;i++) {
document.location = "http://www.example.com/page"+i;
waitUntilPageIsLoaded();
var source = $("body").html();
extractData(source);
}
chrome.tabs.executeScript(null, { code: source });
Of course this does not work since we change the location of the page.
Or more generally speaking, can casperJS can be developped using google chrome extension ?