Java code in thread function:
System.setProperty("webdriver.chrome.driver", "/usr/bin/chromedriver");
ChromeOptions chromeOptions = new ChromeOptions();
chromeOptions.addArguments("--no-sandbox");
chromeOptions.addArguments("--user-data-dir="+config.chromeUserDir);
chromeOptions.addArguments("--profile-directory="+profile);
chromeOptions.addArguments("--start-maximized");
WebDriver driver = new ChromeDriver(chromeOptions);
driver.get("https://www.google.com");
and create object and start in thread with following code
Driver d1 = new Driver(profile);
d1.start();
Driver d2 = new Driver(profile1);
d1.start();
two different profiles have been created, code works well with single thread but with multiple threads it does not open google website in two separate windows. it says,
Starting ChromeDriver 2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac) on port 25692
Only local connections are allowed.
Starting ChromeDriver 2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac) on port 25954
Only local connections are allowed.
Oct 14, 2018 2:10:46 AM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: OSS
Oct 14, 2018 2:10:46 AM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: OSS
Created new window in existing browser session.
and it opens google in only one window. window opened by anther thread remains idle. Could anyone please help?
Issue Analysis
This issue can be reproduced even if you try to run chrome driver in two profile sequential without quiting the driver.
ChromeOptions chromeOptions1 = new ChromeOptions();
chromeOptions1.addArguments("--user-data-dir=C:/Users/My UserName/AppData/Local/Google/Chrome/User Data/Default");
chromeOptions1.addArguments("--profile-directory=Profile 1");
WebDriver driver1 = new ChromeDriver(chromeOptions1);
driver1.get("https://www.google.com");
ChromeOptions chromeOptions2 = new ChromeOptions();
chromeOptions2.addArguments("--user-data-dir=C:/Users/My UserName/AppData/Local/Google/Chrome/User Data/Default");
chromeOptions2.addArguments("--profile-directory=Profile 2");
WebDriver driver2 = new ChromeDriver(chromeOptions2);
When running first instance browser starts and page will be accesed. While running the second instance browser starts, but the page will not open.
The driver.get() line fails with following exception for the second instance
Exception in thread "main" org.openqa.selenium.NoSuchSessionException: invalid session id
(Driver info: chromedriver=70.0.3538.16 (16ed95b41bb05e565b11fb66ac33c660b721f778),platform=Windows NT 10.0.17134 x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 2.99 seconds
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities [{message=unknown error: Chrome failed to start: crashed
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location C:\Program Files (x86)\Google\Chrome\Application\chrome.exe is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
(Driver info: chromedriver=70.0.3538.16 (16ed95b41bb05e565b11fb66ac33c660b721f778),platform=Windows NT 10.0.17134 x86_64), platform=ANY}]
When the first instance is started the user data directory get locked and we are getting error for the second instance as the user data directory is in use.
We can simulate this issue by opening one chrome instance manually with one profile and trying to open one more chrome instance with other profile using chrome driver.
Solution
We have to use different user-data directory for each profile. We no need to create profile manually in chrome browser and also no need to provide --profile-directory argument in chrome options. But you can maintain the sessions and history by mentioning different user-data-dir path for each chrome driver instance
ChromeOptions chromeOptions = new ChromeOptions();
chromeOptions.addArguments("--user-data-dir=C:/ChromeProfiles/FirstProfile"); // Custom directory path for first profile
WebDriver driver = new ChromeDriver(chromeOptions);
driver.get("https://www.google.com");
ChromeOptions chromeOptions = new ChromeOptions();
chromeOptions.addArguments("--user-data-dir=C:/ChromeProfiles/SecondProfile"); // Custom directory path second profile
WebDriver driver = new ChromeDriver(chromeOptions);
driver.get("https://www.google.com");
This is will maintains the sessions and history in two profile that you are looking for.
Also multithreading will work without any issue.
class Driver extends Thread {
private String profile;
public Driver(String profile){
this.profile=profile;
}
public void run()
{
System.out.println ("Thread " +
Thread.currentThread().getId() +
" is running");
ChromeOptions chromeOptions = new ChromeOptions();
chromeOptions.addArguments("--no-sandbox");
chromeOptions.addArguments("--user-data-dir=C:/ChromeProfiles/"+profile);
// chromeOptions.addArguments("--profile-directory="+profile);
chromeOptions.addArguments("--start-maximized");
WebDriver driver = new ChromeDriver(chromeOptions);
driver.get("https://www.google.com");
}
}
public class MultiThreadDriver
{
public static void main(String[] args)
{
ChromeDriverManager.getInstance().setup();
Driver object = new Driver("First Profile");
object.start();
Driver object1 = new Driver("Second Profile");
object1.start();
}
}
Because user data directory get locked when the first instance run, you can change forlder cache of profile 2.
Just copy your profile directory to another folder, and specify it in your --user-data-dir. Browser instances running under debugging mode.
Refer to my answer under another question for detailed explanation: https://stackoverflow.com/a/75191111/4810608
Related
I have updated Selenium but the error keeps occurring even though the web page loads. However, in some instances, the driver starts but it is stagnant. Is this causing an issue and if so, how do I resolve it?
[11556:9032:0502/152954.314:ERROR:device_event_log_impl.cc(162)] [15:29:54.314] Bluetooth: bluetooth_adapter_winrt.cc:1055 Getting Default Adapter failed.
This error message...
ERROR:device_event_log_impl.cc(162)] [15:29:54.314] Bluetooth: bluetooth_adapter_winrt.cc:1055 Getting Default Adapter failed.
...implies that ScopedClosureRunner on_init failed in BluetoothAdapterWinrt::OnGetDefaultAdapter().
Analysis
This error is defined in bluetooth_adapter_winrt.cc as follows:
void BluetoothAdapterWinrt::OnGetDefaultAdapter(
base::ScopedClosureRunner on_init,
ComPtr<IBluetoothAdapter> adapter) {
DCHECK_CALLED_ON_VALID_THREAD(thread_checker_);
if (!adapter) {
BLUETOOTH_LOG(ERROR) << "Getting Default Adapter failed.";
return;
}
Solution
Ensure that:
Selenium is upgraded to current levels Version 3.141.59.
ChromeDriver is updated to current ChromeDriver v84.0 level.
Chrome is updated to current Chrome Version 84.0 level. (as per ChromeDriver v84.0 release notes)
If your base Web Client version is too old, then uninstall it and install a recent GA and released version of Web Client.
Additional considerations
However it was observed that this error can be supressed by running Chrome as root user (administrator) on Linux. but that would be a deviation from the documentation in ChromeDriver - WebDriver for Chrome where it is mentioned:
A common cause for Chrome to crash during startup is running Chrome as root user (administrator) on Linux. While it is possible to work around this issue by passing '--no-sandbox' flag when creating your WebDriver session, i.e. the ChromeDriver session as such a configuration is unsupported and highly discouraged.
Ideally, you need to configure your environment to run Chrome as a regular user instead.
Suppressing the error
Finally, as per the documentation in Selenium Chrome Driver: Resolve Error Messages Regarding Registry Keys and Experimental Options these error logs can be supressed by adding the argument:
excludeSwitches: ['enable-logging']
So your effective code block will be:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ["enable-logging"])
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get("https://www.google.com/")
I had simmilar problems
ConnectionResetError: [WinError 10054] An existing connection was
forcibly closed by the remote host and
Bluetooth: bluetooth_adapter_winrt.cc:1055 Getting Default Adapter
failed.
Both of them disapeared after running cmd as an administrator. I don't know what is the exact cause of this issue but for me it seems that's a lack of privs while running selenium.
If anyone could explain why it is happening would be great.
Simply switching on my device's Bluetooth solved the problem... Don't know the reason behind it
I was getting the same error. On a code that was working yesterday.
The Code is available at this url at this moment https://youtu.be/0kLoVGLTISg?list=PLUDwpEzHYYLvx6SuogA7Zhb_hZl3sln66&t=4073
https://github.com/Microsoft/vscode-python/issues/3252 Found the Resolution hint over here in the comments section, along with https://docs.python.org/3/library/unittest.html#unittest.TestCase.setUp,
suggesting that, we cannot run on "Pycharm"/VSCode using right click -> run from within the class level, we need to run it from the module level i.e. outside the class level, since setUpClass() method is not executed when running from inside of the class.
I receive one of these errors when attempting to open up a session.
selenium.common.exceptions.SessionNotCreatedException: Message: session not created
from chrome not reachable
(Session info: chrome=80.0.3987.132)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created
from disconnected: Unable to receive message from renderer
(Session info: chrome=80.0.3987.132)
ChromeDriver = 80.0.3987.106
I googled these errors and none of the solutions helped, here's my current code: (one of the solutions suggested adding Chrome options)
from selenium import webdriver
class YoutubeBot():
def __init__(self):
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--no-sandbox')
self.driver = webdriver.Chrome('/usr/local/bin/chromedriver', chrome_options=chrome_options)
EDIT: I have now found out that this issue is fixed by using headless, I'd still like to test my code while writing it, is there another solution perhaps? (I'm using the Xfce4 DE on Arch Linux)
This error message...
selenium.common.exceptions.SessionNotCreatedException: Message: session not created from disconnected: Unable to receive message from renderer (Session info: chrome=80.0.3987.132)
...implies that the ChromeDriver was unable to initiate/spawn a new Chrome Browser session.
You need to consider a couple of things:
The absolute location of the ChromeDriver must be passed through a Key / Value pair as follows:
driver = webdriver.Chrome(executable_path='/Users/qa/Documents/Python/chromedriver')
While initializing the session you need to use the options argument instead of chrome_options.
So effectively, you code block will be:
from selenium import webdriver
class YoutubeBot():
def __init__(self):
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--no-sandbox')
self.driver = webdriver.Chrome(executable_path='/usr/local/bin/chromedriver', options=chrome_options)
The solution was simple, sudo the file.
I feel stupid and should've checked this earlier, probably should've taken this into consideration.
When running, all I had to do was sudo python -i main.py and Chrome magically booted up without an error.
For some unknown reasons ,my browser open test pages of my remote server very slowly. So I am thinking if I can reconnect to the browser after quitting the script but don't execute webdriver.quit() this will leave the browser opened. It is probably kind of HOOK or webdriver handle.
I have looked up the selenium API doc but didn't find any function.
I'm using Chrome 62,x64,windows 7,selenium 3.8.0.
I'll be very appreciated whether the question can be solved or not.
No, you can't reconnect to the previous Web Browsing Session after you quit the script. Even if you are able to extract the Session ID, Cookies and other session attributes from the previous Browsing Context still you won't be able to pass those attributes as a HOOK to the WebDriver.
A cleaner way would be to call webdriver.quit() and then span a new Browsing Context.
Deep Dive
There had been a lot of discussions and attempts around to reconnect WebDriver to an existing running Browsing Context. In the discussion Allow webdriver to attach to a running browser Simon Stewart [Creator WebDriver] clearly mentioned:
Reconnecting to an existing Browsing Context is a browser specific feature, hence can't be implemented in a generic way.
With internet-explorer, it's possible to iterate over the open windows in the OS and find the right IE process to attach to.
firefox and google-chrome needs to be started in a specific mode and configuration, which effectively means that just
attaching to a running instance isn't technically possible.
tl; dr
webdriver.firefox.useExisting not implemented
Yes, that's actually quite easy to do.
A selenium <-> webdriver session is represented by a connection url and session_id, you just reconnect to an existing one.
Disclaimer - the approach is using selenium internal properties ("private", in a way), which may change in new releases; you'd better not use it for production code; it's better not to be used against remote SE (yours hub, or provider like BrowserStack/Sauce Labs), because of a caveat/resource drainage explained at the end.
When a webdriver instance is initiated, you need to get the before-mentioned properties; sample:
from selenium import webdriver
driver = webdriver.Chrome()
driver.get('https://www.google.com/')
# now Google is opened, the browser is fully functional; print the two properties
# command_executor._url (it's "private", not for a direct usage), and session_id
print(f'driver.command_executor._url: {driver.command_executor._url}')
print(f'driver.session_id: {driver.session_id}')
With those two properties now known, another instance can connect; the "trick" is to initiate a Remote driver, and provide the _url above - thus it will connect to that running selenium process:
driver2 = webdriver.Remote(command_executor=the_known_url)
# when the started selenium is a local one, the url is in the form 'http://127.0.0.1:62526'
When that is ran, you'll see a new browser window being opened.
That's because upon initiating the driver, the selenium library automatically starts a new session for it - and now you have 1 webdriver process with 2 sessions (browsers instances).
If you navigate to an url, you'll see it is executed on that new browser instance, not the one that's left from the previous start - which is not the desired behavior.
At this point, two things need to be done - a) close the current SE session ("the new one"), and b) switch this instance to the previous session:
if driver2.session_id != the_known_session_id: # this is pretty much guaranteed to be the case
driver2.close() # this closes the session's window - it is currently the only one, thus the session itself will be auto-killed, yet:
driver2.quit() # for remote connections (like ours), this deletes the session, but does not stop the SE server
# take the session that's already running
driver2.session_id = the_known_session_id
# do something with the now hijacked session:
driver.get('https://www.bing.com/')
And, that is it - you're now connected to the previous/already existing session, with all its properties (cookies, LocalStorage, etc).
By the way, you do not have to provide desired_capabilities when initiating the new remote driver - those are stored and inherited from the existing session you took over.
Caveat - having a SE process running can lead to some resource drainage in the system.
Whenever one is started and then not closed - like in the first piece of the code - it will stay there until you manually kill it. By this I mean - in Windows for example - you'll see a "chromedriver.exe" process, that you have to terminate manually once you are done with it. It cannot be closed by a driver that has connected to it as to a remote selenium process.
The reason - whenever you initiate a local browser instance, and then call its quit() method, it has 2 parts in it - the first one is to delete the session from the Selenium instance (what's done in the second code piece up there), and the other is to stop the local service (the chrome/geckodriver) - which generally works ok.
The thing is, for Remote sessions the second piece is missing - your local machine cannot control a remote process, that's the work of that remote's hub. So that 2nd part is literally a pass python statement - a no-op.
If you start too many selenium services on a remote hub, and don't have a control over it - that'll lead to resource drainage from that server. Cloud providers like BrowserStack take measures against this - they are closing services with no activity for the last 60s, etc, yet - this is something you don't want to do.
And as for local SE services - just don't forget to occasionally clean up the OS from orphaned selenium drivers you forgot about :)
OK after mixing various solutions shared on here and tweaking I have this working now as below. Script will use previously left open chrome window if present - the remote connection is perfectly able to kill the browser if needed and code functions just fine.
I would love a way to automate the getting of session_id and url for previous active session without having to write them out to a file during hte previous session for pick up...
This is my first post on here so apologies for breaking any norms
#Set manually - read/write from a file for automation
session_id = "e0137cd71ab49b111f0151c756625d31"
executor_url = "http://localhost:50491"
def attach_to_session(executor_url, session_id):
original_execute = WebDriver.execute
def new_command_execute(self, command, params=None):
if command == "newSession":
# Mock the response
return {'success': 0, 'value': None, 'sessionId': session_id}
else:
return original_execute(self, command, params)
# Patch the function before creating the driver object
WebDriver.execute = new_command_execute
driver = webdriver.Remote(command_executor=executor_url, desired_capabilities={})
driver.session_id = session_id
# Replace the patched function with original function
WebDriver.execute = original_execute
return driver
remote_session = 0
#Try to connect to the last opened session - if failing open new window
try:
driver = attach_to_session(executor_url,session_id)
driver.current_url
print(" Driver has an active window we have connected to it and running here now : ")
print(" Chrome session ID ",session_id)
print(" executor_url",executor_url)
except:
print("No Driver window open - make a new one")
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()),options=myoptions)
session_id = driver.session_id
executor_url = driver.command_executor._url
Without getting into why do you think that leaving an open browser windows will solve the problem of being slow, you don't really need a handle to do that. Just keep running the tests without closing the session or, in other words, without calling driver.quit() as you have mentioned yourself. The question here though framework that comes with its own runner? Like Cucumber?
In any case, you must have some "setup" and "cleanup" code. So what you need to do is to ensure during the "cleanup" phase that the browser is back to its initial state. That means:
Blank page is displayed
Cookies are erased for the session
I'm using selenium chromedriver in nodejs to write some end-to-end tests.
When loading "myurl", the command await driver.get("myurl") always timeouts.
In network I see that there is a websocket (appcues) which continues (status "pending").
I assume that selenium thinks there is still things to load, but the page is already loaded, it takes 2 seconds to load the whole page, except that the websocket is still running.
Is there a way to use driver.get() and specify to ignore explicit requests to be finished ? (since the websocket will never stop).
You can use the pageLoadStrategy capability Documentation WeDriver-W3C
"none" - none causes command to return immediately
"eager" - eager causes command to return after the DOMContentLoaded event fires.
"normal" - The normal state causes command to return after the load event fires on the new page
let driver = await new webdriver.Builder()
.withCapabilities({ 'pageLoadStrategy': 'none' })
.forBrowser('chrome')
.build()
Note : eager is supported in Chromedriver 77
For some unknown reasons ,my browser open test pages of my remote server very slowly. So I am thinking if I can reconnect to the browser after quitting the script but don't execute webdriver.quit() this will leave the browser opened. It is probably kind of HOOK or webdriver handle.
I have looked up the selenium API doc but didn't find any function.
I'm using Chrome 62,x64,windows 7,selenium 3.8.0.
I'll be very appreciated whether the question can be solved or not.
No, you can't reconnect to the previous Web Browsing Session after you quit the script. Even if you are able to extract the Session ID, Cookies and other session attributes from the previous Browsing Context still you won't be able to pass those attributes as a HOOK to the WebDriver.
A cleaner way would be to call webdriver.quit() and then span a new Browsing Context.
Deep Dive
There had been a lot of discussions and attempts around to reconnect WebDriver to an existing running Browsing Context. In the discussion Allow webdriver to attach to a running browser Simon Stewart [Creator WebDriver] clearly mentioned:
Reconnecting to an existing Browsing Context is a browser specific feature, hence can't be implemented in a generic way.
With internet-explorer, it's possible to iterate over the open windows in the OS and find the right IE process to attach to.
firefox and google-chrome needs to be started in a specific mode and configuration, which effectively means that just
attaching to a running instance isn't technically possible.
tl; dr
webdriver.firefox.useExisting not implemented
Yes, that's actually quite easy to do.
A selenium <-> webdriver session is represented by a connection url and session_id, you just reconnect to an existing one.
Disclaimer - the approach is using selenium internal properties ("private", in a way), which may change in new releases; you'd better not use it for production code; it's better not to be used against remote SE (yours hub, or provider like BrowserStack/Sauce Labs), because of a caveat/resource drainage explained at the end.
When a webdriver instance is initiated, you need to get the before-mentioned properties; sample:
from selenium import webdriver
driver = webdriver.Chrome()
driver.get('https://www.google.com/')
# now Google is opened, the browser is fully functional; print the two properties
# command_executor._url (it's "private", not for a direct usage), and session_id
print(f'driver.command_executor._url: {driver.command_executor._url}')
print(f'driver.session_id: {driver.session_id}')
With those two properties now known, another instance can connect; the "trick" is to initiate a Remote driver, and provide the _url above - thus it will connect to that running selenium process:
driver2 = webdriver.Remote(command_executor=the_known_url)
# when the started selenium is a local one, the url is in the form 'http://127.0.0.1:62526'
When that is ran, you'll see a new browser window being opened.
That's because upon initiating the driver, the selenium library automatically starts a new session for it - and now you have 1 webdriver process with 2 sessions (browsers instances).
If you navigate to an url, you'll see it is executed on that new browser instance, not the one that's left from the previous start - which is not the desired behavior.
At this point, two things need to be done - a) close the current SE session ("the new one"), and b) switch this instance to the previous session:
if driver2.session_id != the_known_session_id: # this is pretty much guaranteed to be the case
driver2.close() # this closes the session's window - it is currently the only one, thus the session itself will be auto-killed, yet:
driver2.quit() # for remote connections (like ours), this deletes the session, but does not stop the SE server
# take the session that's already running
driver2.session_id = the_known_session_id
# do something with the now hijacked session:
driver.get('https://www.bing.com/')
And, that is it - you're now connected to the previous/already existing session, with all its properties (cookies, LocalStorage, etc).
By the way, you do not have to provide desired_capabilities when initiating the new remote driver - those are stored and inherited from the existing session you took over.
Caveat - having a SE process running can lead to some resource drainage in the system.
Whenever one is started and then not closed - like in the first piece of the code - it will stay there until you manually kill it. By this I mean - in Windows for example - you'll see a "chromedriver.exe" process, that you have to terminate manually once you are done with it. It cannot be closed by a driver that has connected to it as to a remote selenium process.
The reason - whenever you initiate a local browser instance, and then call its quit() method, it has 2 parts in it - the first one is to delete the session from the Selenium instance (what's done in the second code piece up there), and the other is to stop the local service (the chrome/geckodriver) - which generally works ok.
The thing is, for Remote sessions the second piece is missing - your local machine cannot control a remote process, that's the work of that remote's hub. So that 2nd part is literally a pass python statement - a no-op.
If you start too many selenium services on a remote hub, and don't have a control over it - that'll lead to resource drainage from that server. Cloud providers like BrowserStack take measures against this - they are closing services with no activity for the last 60s, etc, yet - this is something you don't want to do.
And as for local SE services - just don't forget to occasionally clean up the OS from orphaned selenium drivers you forgot about :)
OK after mixing various solutions shared on here and tweaking I have this working now as below. Script will use previously left open chrome window if present - the remote connection is perfectly able to kill the browser if needed and code functions just fine.
I would love a way to automate the getting of session_id and url for previous active session without having to write them out to a file during hte previous session for pick up...
This is my first post on here so apologies for breaking any norms
#Set manually - read/write from a file for automation
session_id = "e0137cd71ab49b111f0151c756625d31"
executor_url = "http://localhost:50491"
def attach_to_session(executor_url, session_id):
original_execute = WebDriver.execute
def new_command_execute(self, command, params=None):
if command == "newSession":
# Mock the response
return {'success': 0, 'value': None, 'sessionId': session_id}
else:
return original_execute(self, command, params)
# Patch the function before creating the driver object
WebDriver.execute = new_command_execute
driver = webdriver.Remote(command_executor=executor_url, desired_capabilities={})
driver.session_id = session_id
# Replace the patched function with original function
WebDriver.execute = original_execute
return driver
remote_session = 0
#Try to connect to the last opened session - if failing open new window
try:
driver = attach_to_session(executor_url,session_id)
driver.current_url
print(" Driver has an active window we have connected to it and running here now : ")
print(" Chrome session ID ",session_id)
print(" executor_url",executor_url)
except:
print("No Driver window open - make a new one")
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()),options=myoptions)
session_id = driver.session_id
executor_url = driver.command_executor._url
Without getting into why do you think that leaving an open browser windows will solve the problem of being slow, you don't really need a handle to do that. Just keep running the tests without closing the session or, in other words, without calling driver.quit() as you have mentioned yourself. The question here though framework that comes with its own runner? Like Cucumber?
In any case, you must have some "setup" and "cleanup" code. So what you need to do is to ensure during the "cleanup" phase that the browser is back to its initial state. That means:
Blank page is displayed
Cookies are erased for the session