Python Selenium : How to hide geckodriver? - python-3.x

I am writing an program for a web automation in python. Is here a ways to hide the geckodriver? So that the console (see picture) won't show up when I start the program.
console of geckodriver
here is a fraction of my code:
from selenium import webdriver
from selenium import *
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC`
driver=webdriver.Firefox()
wait=WebDriverWait(driver,120)
url = r"http://google.com"
driver.get(url) #This line starts the console (see picture)

To prevent geckodriver from displaying any windows, you need to pass -headless argument:
from selenium import webdriver
options = webdriver.FirefoxOptions()
options.add_argument('-headless')
driver = webdriver.Firefox(options=options)
driver.get('https://www.example.com')
...
driver.quit()

This worked for me in C#. It blocks both geckodriver and firefox window
FirefoxOptions f = new FirefoxOptions();
f.AddArgument("-headless");
var ffds = FirefoxDriverService.CreateDefaultService();
ffds.HideCommandPromptWindow = true;
driver = new FirefoxDriver(ffds,f);

I was able to do that after implementing PyVirtualDisplay
sudo pip install pyvirtualdisplay # Install it into your Virtual Environment
Then just import Display as follows:
from pyvirtualdisplay import Display
Then, before fetching, start the virtual display as follows:
# initiate virtual display with 'visible=0' activated
# this way you will hide the browser
display = Display(visible=0, size=(800, 600))
# Start Display
display.start()
...
# Do your fetching/scrapping
...
# Stop Display
display.stop()
I hope it helps

Here the approach which has solved it in my case. I used #thekingofravens suggestion but realized that it is just enough to create the Run.bat file which he mentions in his post. Previously I ran my programms in IDLE with F5. Because of this the Geckodriver popped up every few seconds.
My Solution: I just made the Run.bat file with the same code:
cd C:\PathToYourFileWhichIsCausingTheGeckodriverToPopUp
python FileWhichIsCausingTheGeckodriverToPopUp.py
And thats it. Just start this file when you want to run your code and the Geckodriver won't pop up. (That it works without the whole path to you program, Python has to be in PATH.)
Also, of course you can run your program from the command line with the same commands like above and without creating an extra bat file.

You have to take care of a couple of stuffs here:
Keep the useful imports. Unusual imports like from selenium import * (in your code) must be avoided.
Keep your code minimal. wait=WebDriverWait(driver,120) have no usage in your code block.
When you use the raw r switch remember to use single quotes '...'
If you initialize the webdriver instance remember to call the quit() method.
Be careful about indentation while using Python.
Here is the minimal code which uses geckodriver to open the url http://www.google.com, print the Page Title and quits the driver instance.
from selenium import webdriver
driver=webdriver.Firefox(executable_path=r'C:\Utility\BrowserDrivers\geckodriver.exe')
url = r'http://www.google.com'
driver.get(url)
print("Page Title is : %s" %driver.title)
driver.quit()
Console Output:
Page Title is : Google

So I found a very generic workaround to solve this on Windows (in Linux it doesn't seem to be an issue in the first place, can't speak for OSX).
So you need to make three files and its very awkward.
First, you make a file. call it start.bat (or anything) and put the following code in it:
wscript.exe "C:\Wherever\invisible.vbs" "C:\Some Other Place\Run.bat"
This will be the top level batch script. It will be visible for a split second while it launches a visual basic script and passes a batch script as an argument. The purpose of this is to make the next console invisible. Next we make the VBscript, invisible.vbs:
CreateObject("Wscript.Shell").Run """" & WScript.Arguments(0) & """", 0, False
Finally, we make the script that invisible.vbs is supposed to hide, we could call it Run.bat
cd C:\Wherever
python script_using_geckodriver.py
What happens is, as follows:
The first .bat file launches invisible.vbs
invisible.vbs launches the second .bat file without showing it on screen.
The second .bat file then launches the python program. Python (and geckodriver) output to the invisible cmd therefore hiding the geckodriver console window.
P.S. all of this works with PyInstaller to produce a single redistributable package the user can just click on.
Credit harrymc # superuser.com for this solution, which I found when trying to solve an unrelated problem. I tested and realized it was cross applicable to this.
https://superuser.com/questions/62525/run-a-batch-file-in-a-completely-hidden-way

Related

edge browser closing automatically

When I am trying to automate my edge browser using selenium, in code I didn't mentioned about the close keyword but my browser is running very fast and closing automatically.
What could be the solution for this ?
This is my simple code
from selenium import webdriver
from selenium.webdriver.edge.service import Service
from selenium.webdriver.common.by import By
service_object = Service("C:\\Users\\piyus\\OneDrive\\Documents\\drivers\\edge\\msedgedriver.exe")
driver = webdriver.Edge(service= service_object)
driver.maximize_window()
driver.get("https://rahulshettyacademy.com/angularpractice/")
driver.find_element(By.NAME, "email").send_keys("hello#gmail.com")
driver.find_element(By.ID, "exampleInputPassword1").send_keys("1234#Vivo")
driver.find_element(By.ID, "exampleCheck1").click()
I tried with wait keyword but that is not working for me
I want the browser should not close automatically or should not close without my instructions
Add the below statement and try:
from selenium.webdriver.edge.options import Options
options = Options()
options.add_experimental_option("detach", True)
driver = webdriver.Edge(service= service_object, options=options)
Are you familiar with Python breakpoints? You could drop one into your code so that your program pauses execution after reaching it.
There's a lot of info about that in this Stack Overflow post: Simpler way to put PDB breakpoints in Python code?
Eg:
import pdb; pdb.set_trace()
Once you are in a breakpoint, type c in the console and hit Enter to continue from where you left off. Or you can go line-by-line from there by using n and hitting Enter. Or to step into a method, use s and hit Enter.
If using breakpoints for debugging selenium tests that use pytest, there's also this post for reference: https://seleniumbase.com/the-ultimate-pytest-debugging-guide-2021/

How to check if Open File dialog has been open after pressing a button in a Chrome Browser tab on Python?

I'm trying to automate a process within the OpenSea Create page after having logged in with Metamask, and so far, I have managed to develop a simple program that chooses a particular image file using a path which passes to the Open File dialog "implicitly", here's the code:
import pyautogui
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
def wait_xpath(code): #function to wait for the xpath of an element to be located
WebDriverWait(driver, 60).until(EC.presence_of_element_located((By.XPATH, code)))
opt = Options() #the variable that will store the selenium options
opt.add_experimental_option("debuggerAddress", "localhost:9222") #this allows bulk-dozer to take control of your Chrome Browser in DevTools mode.
s = Service(r'C:\Users\ResetStoreX\AppData\Local\Programs\Python\Python39\Scripts\chromedriver.exe') #Use the chrome driver located at the corresponding path
driver = webdriver.Chrome(service=s, options=opt) #execute the chromedriver.exe with the previous conditions
nft_folder_path = r'C:\Users\ResetStoreX\Pictures\Cryptobote\Cryptobote NFTs\Crypto Cangrejos\SANDwich\Crabs'
start_number = 3
if driver.current_url == 'https://opensea.io/asset/create':
print('all right')
print('')
print(driver.current_window_handle)
print(driver.window_handles)
print(driver.title)
print('')
nft_to_be_selected = nft_folder_path+"\\"+str(start_number)+".png"
wait_xpath('//*[#id="main"]/div/div/section/div/form/div[1]/div/div[2]')
imageUpload = driver.find_element(By.XPATH, '//*[#id="main"]/div/div/section/div/form/div[1]/div/div[2]').click() #click on the upload image button
print(driver.current_window_handle)
print(driver.window_handles)
time.sleep(2)
pyautogui.write(nft_to_be_selected)
pyautogui.press('enter', presses = 2)
Output:
After checking the URL, the program clicks on the corresponding button to upload a file
Then it waits 2 seconds before pasting the image path into the Name textbox, for then pressing Enter
So the file ends up being correctly uploaded to this page.
The thing is, the program above works because the following conditions are met before execution:
The current window open is the Chrome Browser tab (instead of the Python program itself, i.e. Spyder environment in my case)
After clicking the button to upload a file, the Name textbox is selected by default, regardless the current path it opens with.
So, I'm kind of perfectionist, and I would like to know if there's a method (using Selenium or other Python module) to check if there's an Open File dialog open before doing the rest of the work.
I tried print(driver.window_handles) right after clicking that button, but Selenium did not recognize the Open File dialog as another Chrome Window, it just printed the tab ID of this page, so it seems to me that Selenium can't do what I want, but I'm not sure, so I would like to hear what other methods could be used in this case.
PS: I had to do this process this way because send_keys() method did not work in this page
The dialog you are trying to interact with is a native OS dialog, it's not a kind of browser handler / dialog / tab etc. So Selenium can not indicate it and can not handle it. There are several approaches to work with such OS native dialogs. I do not want to copy - paste existing solutions. You can try for example this solution. It is highly detailed and looks good.

How to load default profile in Chrome using Python and the Selenium Webdriver?

I hope you are all well =)
this is my first post/question on stackoverflow =)
I'm writing this after trying all the answers in this thread. (the last answer I tried was from Youssof H.)
I'm a python newbie and trying to write a script that can help me upload products to a website.
Since I need to be logged in to be able to add products, I figured why not use a browser profile where I'm already logged in, instead of writing code to get this done (I figured using a browser profile would be easier)
I've been trying to get this working for many hours now and I don't seem to be able to solve this by myself.
When I run the code, instead of opening Chromium it keeps opening google-chrome. Prior to trying to use chromium I tried it with chrome, if I open chrome it opens google-chrome-stable but when I run the python file it runs google-chrome.
My operating system is Mint 20.1 (Cinnamon) and I use Visual Studio Code
If anyone can help me with this it would be highly appreciated =)
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
# Do not use this path that is extracted from "chrome://version/"
exec_path_chrome = "/usr/bin/chromium"
exec_path_driver = "/home/equinix/drivers/chromedriver"
ch_options = Options() # Chrome Options
# Extract this path from "chrome://version/"
ch_options.add_argument(
"user-data-dir = /home/equinix/.config/chromium/Default")
# Chrome_Options is deprecated. So we use options instead.
driver = webdriver.Chrome(executable_path=exec_path_driver, options=ch_options)
driver.get("https://duckduckgo.com/")
I dont have the same paths but, you can pinpoint the exact binary that you want to use, either chromium or chrome
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
# Do not use this path that is extracted from "chrome://version/"
exec_path_chrome = "/usr/lib/bin/chromium"
exec_path_driver = "/home/art/drivers/chromedriver"
ch_options = Options() # Chrome Options
# Extract this path from "chrome://version/"
ch_options.add_argument(
"user-data-dir = /home/art/.config/chromium/Default")
# in your case its "/usr/lib/bin/chromium/chromium"
ch_options.binary_location = '/usr/lib/chromium/chromium'
#to point to google-chrome-stable
#ch_options.binary_location = '/opt/google/chrome/google-chrome'
# Chrome_Options is deprecated. So we use options instead.
driver = webdriver.Chrome(executable_path=exec_path_driver, options=ch_options)
driver.get("https://duckduckgo.com/")
I use such a simple construction, but it only works when you have one google chrome profile.
from selenium.webdriver.chrome.options import Options
from selenium import webdriver
import time
from selenium.webdriver.common.by import By
chrome_options = Options()
chrome_options.add_argument("user-data-dir=C:/Users/yourusername/AppData/Local/Google/Chrome/User Data/")
browser = webdriver.Chrome(chrome_options=chrome_options)
browser.get("https://google.com/")

Selenium webdriver python element screenshot not working properly

I looked up Selenium python documentation and it allows one to take screenshots of an element. I tried the following code and it worked for small pages (around 3-4 actual A4 pages when you print them):
from selenium.webdriver import FirefoxOptions
firefox_profile = webdriver.FirefoxProfile()
firefox_profile.set_preference("browser.privatebrowsing.autostart", True)
# Configure options for Firefox webdriver
options = FirefoxOptions()
options.add_argument('--headless')
# Initialise Firefox webdriver
driver = webdriver.Firefox(firefox_profile=firefox_profile, options=options)
driver.maximize_window()
driver.get(url)
driver.find_element_by_tag_name("body").screenshot("career.png")
driver.close()
When I try it with url="https://waitbutwhy.com/2020/03/my-morning.html", it gives the screenshot of the entire page, as expected. But when I try it with url="https://waitbutwhy.com/2018/04/picking-career.html", almost half of the page is not rendered in the screenshot (the image is too large to upload here), even though the "body" tag does extend all the way down in the original HTML.
I have tried using both implicit and explicit waits (set to 10s, which is more than enough for a browser to load all contents, comments and discussion section included), but that has not improved the screenshot capability. Just to be sure that selenium was in fact loading the web page properly, I tried loading without the headless flag, and once the webpage was completely loaded, I ran driver.find_element_by_tag_name("body").screenshot("career.png"). The screenshot was again half-blank.
It seems that there might be some memory constraints put on the screenshot method (although I couldn't find any), or the logic behind the screenshot method itself is flawed. I can't figure it out though. I simply want to take the screenshot of the entire "body" element (preferably in a headless environment).
You may try this code, just that you need to install a package from command prompt using the command pip install Selenium-Screenshot
import time
from selenium import webdriver
from Screenshot import Screenshot_Clipping
driver = webdriver.Chrome()
driver.maximize_window()
driver.implicitly_wait(10)
driver.get("https://waitbutwhy.com/2020/03/my-morning.html")
obj=Screenshot_Clipping.Screenshot()
img_loc=obj.full_Screenshot(driver, save_path=r'.', image_name='capture.png')
print(img_loc)
time.sleep(5)
driver.close()
Outcome/Result comes out to be like, you just need to zoom the screenshot saved
Hope this works for you!

get active url on chrome and firefox using python

I work with python 3.7 on windows and I want to write a script that print the actual active urls on browsers (chrome and firefox), I found the script:
import webbrowser
webbrowser.open(the url)
but this script allows to open the url not to find the active urls.
can someone help me
Here you need to download and install pywin32 and import these modules in your script like this -
import win32gui
import win32con
#to get currently active windows
window = win32gui.GetForegroundWindow
Or to get the Google Chrome window handle
win32gui.FindWindow
You can use selenium module and loop through all open tabs.
This code prints all open tabs of chrome and firefox:
from selenium import webdriver
chromeDriver = webdriver.Chrome()
firefoxDriver = webdriver.Firefox()
for handle in chromeDriver.window_handles[0]:
chromeDriver.switch_to.window(handle)
print(chromeDriver.current_url)
for handle in firefoxDriver.window_handles[0]:
firefoxDriver.switch_to.window(handle)
print(firefoxDriver.current_url)
Note:
This code is inefficient and should change to use only one loop.

Resources