Opening Firefox via Selenium in Jenkins - python-3.x

When trying to run a Python Selenium script via Jenkins, it crashes, with a very unhelpful exception - "selenium.common.exceptions.WebDriverException: Message: Process unexpectedly closed with status: 0"
The same Python script runs perfectly fine from outside Python.
I've reduced the script to a very, very basic one, simply launching Firefox -
from selenium import webdriver
if __name__ == "__main__":
print("Hello World")
driver = webdriver.Firefox()
driver.get("http://www.google.com")
driver.maximize_window()
driver.quit()
print("Goodbye World")
Output from cmd prompt:
>C:\Users\kipod>C:\python_projects\Ranger\_just_open_browser.py
>Hello World
>Goodbye World
Works fine.
Running as a Jenkins job:
>##Started by user anonymous
>Building in workspace C:\Program Files (x86)\Jenkins\workspace\Second Test
>[Second Test] $ cmd /c call >C:\Users\kipod\AppData\Local\Temp\jenkins3557827225974274191.bat
>
>C:\Program Files (x86)\Jenkins\workspace\Second Test>python >C:\python_projects\Ranger\_just_open_browser.py
>Hello World
>Traceback (most recent call last):
> File "C:\python_projects\Ranger\_just_open_browser.py", line 8, in <module>
> driver = webdriver.Firefox()
> File "C:\Python\Python36-32\lib\site->packages\selenium\webdriver\firefox\webdriver.py", line 154, in __init__
> keep_alive=True)
> File "C:\Python\Python36-32\lib\site->packages\selenium\webdriver\remote\webdriver.py", line 151, in __init__
> self.start_session(desired_capabilities, browser_profile)
> File "C:\Python\Python36-32\lib\site->packages\selenium\webdriver\remote\webdriver.py", line 240, in start_session
> response = self.execute(Command.NEW_SESSION, parameters)
> File "C:\Python\Python36-32\lib\site->packages\selenium\webdriver\remote\webdriver.py", line 308, in execute
> self.error_handler.check_response(response)
> File "C:\Python\Python36-32\lib\site->packages\selenium\webdriver\remote\errorhandler.py", line 194, in check_response
> **raise exception_class(message, screen, stacktrace)
>selenium.common.exceptions.WebDriverException: Message: Process unexpectedly >closed with status: 0**
>C:\Program Files (x86)\Jenkins\workspace\Second Test>exit 1
>Build step 'Execute Windows batch command' marked build as failure
>Finished: FAILURE
I suspect something is off with Jenkins's environment variables, but I have no idea what to check. And the exception is so unspecific ("I crashed. lols"), That I found nothing helpful on Google. And I've been searching for a while.
Versions:
Jenkins: 2.73.1
Python: 3.6.2
Geckodriver: 0.19.0
Selenium: 3.6.0
(Tried opening chrome instead of firefox - same behaviour.)

I think you need to run your python script using a jenkins slave configured to log on as a standard windows user account (for example "kipod") ? Make sure that auto login is set for the slave machine for the user account, in order to allow selenium to launch Firefox in a desktop.
Some documenation here: Step by step guide to set up master and slave machines on Windows

Related

Python selenium script connecting to Microsoft edge without debug option?

For those who might not know selenium is a library in many languages to work with browsers. In this specific case, I'm trying to connect in python script on the browser that the user lunches and can save all the running tabs. There are many things that I'm facing so I separated them into 3 problems. I hope they will be understandable. The main goal is to be able to connect to the browser without debug options.
Example
This is a script that should be able to connect to the browser that was launched with debug options in the command prompt. As I provide later I'm no longer able to connect to the remote browser at 127.0.0.1:9222." I've tried searching for a solution, but I haven't found one that works.
Launching browser in with debug port
"c:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe" --remote-debugging-port=9222
Python script
from selenium import webdriver
from selenium.webdriver.edge.options import Options
edge_options = Options()
edge_options.add_experimental_option('debuggerAddress', '127.0.0.1:9222')
driver = webdriver.Edge(options=edge_options)
driver.get(youtube.com)
printf(driver.title)
1. problem
For some reason, my code can no longer connect to the browser launched by the command mentioned. This is an error message Im facing.
part of the error message
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to microsoft edge at 127.0.0.1:9222
**from chrome not reachable
**
full error message
Traceback (most recent call last):
File "C:\Users\user\OneDrive\Plocha\záložky\msedge - running\Try 18.py", line 7, in <module>
driver = webdriver.Edge(options=edge_options)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\selenium\webdriver\edge\webdriver.py", line 73, in __init__
super().__init__(
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\selenium\webdriver\chromium\webdriver.py", line 104, in __init__
super().__init__(
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\selenium\webdriver\remote\webdriver.py", line 286, in __init__
self.start_session(capabilities, browser_profile)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\selenium\webdriver\remote\webdriver.py", line 378, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\selenium\webdriver\remote\webdriver.py", line 440, in execute
self.error_handler.check_response(response)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\selenium\webdriver\remote\errorhandler.py", line 245, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to microsoft edge at 127.0.0.1:9222
from chrome not reachable
Stacktrace:
Backtrace:
Microsoft::Applications::Events::EventProperties::SetProperty [0x00007FF72EE616C2+15186]
Microsoft::Applications::Events::EventProperty::EventProperty [0x00007FF72EDFA212+827554]
(No symbol) [0x00007FF72EABEBF3]
(No symbol) [0x00007FF72EAAEC8A]
(No symbol) [0x00007FF72EAF013D]
(No symbol) [0x00007FF72EAE5D4A]
(No symbol) [0x00007FF72EB29C56]
(No symbol) [0x00007FF72EB21D23]
(No symbol) [0x00007FF72EAF3B80]
(No symbol) [0x00007FF72EAF2B0E]
(No symbol) [0x00007FF72EAF4344]
Microsoft::Applications::Events::EventProperties::SetProperty [0x00007FF72ECDC3B0+182752]
(No symbol) [0x00007FF72EBB0095]
Microsoft::Applications::Events::EventProperty::EventProperty [0x00007FF72ED3A6EA+42362]
Microsoft::Applications::Events::EventProperty::EventProperty [0x00007FF72ED3D425+53941]
Microsoft::Applications::Events::ILogManager::DispatchEventBroadcast [0x00007FF72F058AB3+1456595]
Microsoft::Applications::Events::EventProperty::EventProperty [0x00007FF72EE0276A+861690]
Microsoft::Applications::Events::EventProperty::EventProperty [0x00007FF72EE07854+882404]
Microsoft::Applications::Events::EventProperty::EventProperty [0x00007FF72EE079AC+882748]
Microsoft::Applications::Events::EventProperty::EventProperty [0x00007FF72EE1097E+919566]
BaseThreadInitThunk [0x00007FF8D3C726BD+29]
RtlUserThreadStart [0x00007FF8D5A6DFB8+40]
What have I tried? There are many tutorials on using python and chrome, but not many are for an edge. Here are some samples of things I've tried:
https://www.youtube.com/watch?v=Zrx8FSEo9lk
https://www.youtube.com/watch?v=FGaU0OgtbNk&t=181s
https://www.youtube.com/watch?v=pZdMsLKAjs4&t=271s
https://bugs.chromium.org/p/chromedriver/issues/detail?id=710#c3
https://www.geeksforgeeks.org/automated-browser-testing-with-edge-and-selenium-in-python/
https://stackoverflow.com/questions/8344776/can-selenium-interact-with-an-existing-browser-session
2. problem
My goal is to be able to connect to the browser that didn't start with debug port. Is there a way around the debug options?
What have I tried? Switching the localhost address with the IP address of processes named "msedge.exe". But it got the same error as mentioned.
3. problem
I'm wondering if there's a way to switch between these windows without having to repeatedly reconnect to each one. Is there a method or tool available that would allow me to easily navigate between these separate windows within the application, or do I have no choice but to reconnect each time? Any advice or suggestions would be greatly appreciated. Multiple windows of the browser
Please be patient, thanks for your help.

Index out of error in while deploying flask app using ngrok

I have created a server using Flask for a demonstration of my ML model. I was running it on Google Colab and using flask-ngrok for tunneling. It was working properly, but suddenly it stopped working today and is showing this error:
=> loading checkpoint './semi_supervised_model_3/ckpt_epoch_10.00.pth'
=> loaded checkpoint './semi_supervised_model_3/ckpt_epoch_10.00.pth' (epoch 10.0)
* Serving Flask app "demo_Server" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 1182, in run
self.function(*self.args, **self.kwargs)
File "/usr/local/lib/python3.6/dist-packages/flask_ngrok.py", line 70, in start_ngrok
ngrok_address = _run_ngrok()
File "/usr/local/lib/python3.6/dist-packages/flask_ngrok.py", line 38, in _run_ngrok
tunnel_url = j['tunnels'][0]['public_url'] # Do the parsing of the get
IndexError: list index out of range
What is the meaning of this error? Why is it coming even when I didn't change anything?
My problem got solved when I terminated the current session. In the new session, I reinstalled flask-ngrok and it worked fine.

For the IPython run magic the option to ignore SystemExist doesn't seem to work with modules

The docs for the %run magic -e option state:
ignore sys.exit() calls or SystemExit exceptions in the script being
run. This is particularly useful if IPython is being used to run
unittests, which always exit with a sys.exit() call. In such cases you
are interested in the output of the test results, not in seeing a
traceback of the unittest module.
This works when running scripts but doesn't seem to work when running modules.
So when I type %run -e -m pytest I still get a traceback when a test fails due the SystemExit thrown by pytest which is case mentioned in the docs above that -e is meant to address. I know I can type !pytest but I don't want to wait until pytest completed before I start to see results, and I also want to add the current directory to to module search path.
I am running IPython within Spyder but the behaviour is the same if I run IPython from the Windows command prompt. I there any way of doing what I want and avoiding the distracting traceback?
I ran the following test with %run -m pytest from the spyder ipython console
import pytest
def test_fail():
assert 0
The output was:
============================= test session starts =============================
platform win32 -- Python 3.7.7, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: D:\home\shane\temp\pytest
collected 1 item
test_dummy.py F [100%]
================================== FAILURES ===================================
__________________________________ test_fail __________________________________
def test_fail():
> assert 0
E AssertionError
test_dummy.py:21: AssertionError
=========================== short test summary info ===========================
FAILED test_dummy.py::test_fail - AssertionError
============================== 1 failed in 0.03s ==============================
Traceback (most recent call last):
File "c:\opt\python37\lib\runpy.py", line 205, in run_module
return _run_module_code(code, init_globals, run_name, mod_spec)
File "c:\opt\python37\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "c:\opt\python37\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "c:\opt\python37\lib\site-packages\pytest\__main__.py", line 7, in <module>
raise SystemExit(pytest.main())
SystemExit: ExitCode.TESTS_FAILED

Could not find 'adb.exe' in ......?

How to fix Could not find 'adb.exe'?
I think my question is one of the most asked and answered question of the Android world. I read and tried them all.
I checked and update my environment variables.
I desinstalled and reinstalled ANdroid Studio
I copied pasted adb in all possible folders:
C:\Users\Nino\AppData\Local\Android\Sdk
C:\Program Files\Android
C:\Program Files\Android\tools
C:\Program Files\Android\platform-tools
In fact I spent all the day by searching, reading and testing all the solution possible.
And nothing fixed my problem. So I really need your help.
I am newbie and I want to share my experience. Here is my code:
from appium import webdriver
import unittest
from selenium.webdriver.common.by import By
class Instagram(unittest.TestCase):
def setUp(self):
desired_caps = {}
desired_caps['platformName']='Android'
desired_caps['platformVersion']='6.0'
desired_caps['deviceName']='S6S5IN3G'
desired_caps['noReset']='true'
desired_caps['appPackage']='com.instagram'
desired_caps['appActivity']='com.instagram.android.activity.MainTabActivity'
self.driver = webdriver.Remote('http://localhost:4723/wd/hub',desired_caps)
#self.driver = webdriver.Remote('http://0.0.0.0:4723/wd/hub',desired_caps)
def tearDown(self):
self.driver.quit()
def test_scrap_email(self):
search_button = self.driver.findElement(By.xpath ("//android.widget.ImageView[#bounds='[126,800][162,836]']"))
search_button.click()
if __name__ == '__main__':
suite = unittest.Testloader().loadTestsFromTestCase(Instagram)
unittest.TextTestRunner(verbosity=1).run(suite)
This is the errors:
Testing started at 01:00 ...
C:\Users\Nino\PycharmProjects\Instagram\venv\Scripts\python.exe
"C:\Program Files\JetBrains\PyCharm Community Edition
2019.1.3\helpers\pycharm_jb_unittest_runner.py" --target mybot_instagram.Instagram.test_scrap_email Launching unittests with
arguments python -m unittest
mybot_instagram.Instagram.test_scrap_email in
C:\Users\Nino\PycharmProjects\Instagram
Error Traceback (most recent call last): File
"C:\Users\Nino\AppData\Local\Programs\Python\Python37\lib\unittest\case.py",
line 59, in testPartExecutor
yield File "C:\Users\Nino\AppData\Local\Programs\Python\Python37\lib\unittest\case.py",
line 624, in run
self.setUp() File "C:\Users\Nino\PycharmProjects\Instagram\mybot_instagram.py", line 17,
in setUp
self.driver = webdriver.Remote('http://localhost:4723/wd/hub',desired_caps) File
"C:\Users\Nino\PycharmProjects\Instagram\venv\lib\site-packages\appium\webdriver\webdriver.py",
line 144, in init
proxy File "C:\Users\Nino\PycharmProjects\Instagram\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py",
line 157, in init
self.start_session(capabilities, browser_profile) File "C:\Users\Nino\PycharmProjects\Instagram\venv\lib\site-packages\appium\webdriver\webdriver.py",
line 219, in start_session
response = self.execute(RemoteCommand.NEW_SESSION, parameters) File
"C:\Users\Nino\PycharmProjects\Instagram\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py",
line 321, in execute
self.error_handler.check_response(response) File "C:\Users\Nino\PycharmProjects\Instagram\venv\lib\site-packages\selenium\webdriver\remote\errorhandler.py",
line 242, in check_response
raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: An unknown
server-side error occurred while processing the command. Original
error: Could not find 'adb.exe' in ["C:\Program
Files\Android\platform-tools\platform-tools\adb.exe","C:\Program
Files\Android\platform-tools\emulator\adb.exe","C:\Program
Files\Android\platform-tools\tools\adb.exe","C:\Program
Files\Android\platform-tools\tools\bin\adb.exe"]. Do you have
Android Build Tools installed at 'C:\Program
Files\Android\platform-tools'?
Assertion failed
C:\Users\Nino\AppData\Local\Programs\Python\Python37\lib\unittest\case.py:656:
ResourceWarning: unclosed
outcome.errors.clear() ResourceWarning: Enable tracemalloc to get the
object allocation traceback
Ran 1 test in 2.139s
FAILED (errors=1)
Process finished with exit code 1
Assertion failed
Assertion failed
I am using pycharm. When I run the code I see some error log, but when I run it from cmd, there is nothing.
And here the main part of Appium log:
> Android >= 6, since UIAutomator1 framework is not maintained anymore by the OS vendor.
> [Appium] Appium v1.13.0 creating new AndroidDriver (v4.15.1) session
> [Appium] Capabilities:
> [Appium] platformName: Android
> [Appium] platformVersion: 6.0
> [Appium] deviceName: S6S5IN3G
> [Appium] noReset: true
> [Appium] appPackage: com.instagram
> [Appium] appActivity: com.instagram.android.activity.MainTabActivity
> [BaseDriver] W3C capabilities {"alwaysMatch":{"platformNa... and MJSONWP desired capabilities {"platformName":"Android","... were
> provided
> [BaseDriver] Creating session with W3C capabilities: {"alwaysMatch":{"platformNa...
> [BaseDriver] Capability 'noReset' changed from string to boolean. This may cause unexpected behavior
> [BaseDriver] Session created with session id: 786ec1a0-280c-4573-a917-18df120f0629
> [AndroidDriver] Shutting down Android driver
> [AndroidDriver] Called deleteSession but bootstrap wasn't active
> [BaseDriver] Event 'newSessionStarted' logged at 1566428435932 (01:00:35 GMT+0200 (Paris, Madrid (heure d’été)))
> [W3C] Encountered internal error running command: Error: Could not find 'adb.exe' in ["C:\\Program
> Files\\Android\\platform-tools\\platform-tools\\adb.exe","C:\\Program
> Files\\Android\\platform-tools\\emulator\\adb.exe","C:\\Program
> Files\\Android\\platform-tools\\tools\\adb.exe","C:\\Program
> Files\\Android\\platform-tools\\tools\\bin\\adb.exe"]. Do you have
> Android Build Tools installed at 'C:\Program
> Files\Android\platform-tools'?
> [W3C] at ADB. (C:\Users\Nino\AppData\Local\Programs\Appium\resources\app\node_modules\appium\node_modules\appium-adb\lib\tools\system-calls.js:106:11)
> [W3C] at Generator.next ()
> [W3C] at asyncGeneratorStep (C:\Users\Nino\AppData\Local\Programs\Appium\resources\app\node_modules\appium\node_modules\#babel\runtime\helpers\asyncToGenerator.js:3:24)
> [W3C] at _next (C:\Users\Nino\AppData\Local\Programs\Appium\resources\app\node_modules\appium\node_modules\#babel\runtime\helpers\asyncToGenerator.js:25:9)
> [W3C] at run (C:\Users\Nino\AppData\Local\Programs\Appium\resources\app\node_modules\appium\node_modules\core-js\modules\es6.promise.js:75:22)
> [W3C] at C:\Users\Nino\AppData\Local\Programs\Appium\resources\app\node_modules\appium\node_modules\core-js\modules\es6.promise.js:92:30
> [W3C] at flush (C:\Users\Nino\AppData\Local\Programs\Appium\resources\app\node_modules\appium\node_modules\core-js\modules\_microtask.js:18:9)
> [W3C] at process._tickCallback (internal/process/next_tick.js:61:11)
> [HTTP] <-- POST /wd/hub/session 500 49 ms - 1544
> [HTTP]
Can anyone suggest me some solutions please?
THank you.
You need to have adb.exe in your operating system PATH
I would recommend going for the following setup:
Create ANDROID_HOME environment variable pointing to your Android SDK installation
Create JAVA_HOME environment variable pointing to your JDK installation
Create PATH environment variable like:
%ANDROID_HOME%\tools;%ANDROID_HOME%\platform-tools;%JAVA_HOME%\bin
That should be sufficient to successfully launch your Python Appium test
You can always use appium-doctor tool in order to check the integrity of your Appium pre-requisites installation.
More information:
Appium Getting Started
Appium - Code Examples - Python

Unable to run Firefox in headless mode

I am trying to run a Python + Selenium script in headless mode with Firefox using Xvfb but I am getting errors. There is not much documents or guides available for Xvfb to troubleshoot the issue so looking for assistance here.
Environment info:
OS: CentOS release 6.5 (Minimal installation)
Xvfb: xorg-x11-server-Xvfb-1.15.0
Firefox: 52.8.0
geckodriver: 0.24.0
Python: 3.6.7
Steps followed:
Once done installing the above-mentioned requirements. I started a virtual display with:
$Xvfb :1 -ac &
Also, I tried with:
$Xvfb :1 -screen 0 1024x768x24 -extension RANDR &
And then I set Display variable:
export DISPLAY=:1
When I tried to initiate Selenium WebDriver in Python console I am getting the error Connection refused:
> from selenium import webdriver
> from pyvirtualdisplay import Display
> display = Display(visible=0, size=(800, 600))
> display.start()
> driver = webdriver.Firefox()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.6/site-packages/selenium/webdriver/firefox/webdriver.py", line 174, in __init__
keep_alive=True)
File "/usr/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "/usr/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/usr/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: connection refused
Any help or suggestion will be much appreciated.
There is a wrapper around xvfb called PyVirtualDisplay that seems designed for this exact solution. If you simply do a pip install pyvirtualdisplay the following script should run a headless Firefox window:
from pyvirtualdisplay import Display
from selenium import webdriver
display = Display(visible=0, size=(800, 600))
display.start()
# now Firefox will run in a virtual display.
# you will not see the browser.
browser = webdriver.Firefox(executable_path="/Users/username/Location/geckodriver")
browser.get('http://www.google.com')
print browser.title
browser.quit()
display.stop()

Resources