Dealing with reCAPTCHA in Python Selenium - python-3.x

I need to automate a web page using python selenium, but it encounters a reCaptcha, which is in another frame. I want to solve the captcha, and continue the script by clicking the login button, when the reCaptcha has been solved; However, this gets tricky, since a frame is involved, and the frame needs to switch back to the default content. Can anyone help me in this regard?
from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select
import time
browser = webdriver.Chrome()
browser.delete_all_cookies()
browser.maximize_window()
browser.get("https://developer.syntecx.org/ptcl_ebills/signin.php")
browser.switch_to.frame(browser.find_element_by_tag_name("iframe"))
browser.find_element_by_xpath("//*[#id='recaptcha-anchor']/div[1]").click()
time.sleep(20)
browser.switch_to_default_content()
browser.find_element_by_xpath("//*[#id='login']/button").click()

Once you fill in the Email and Password field to click on the recaptcha you can use the following Locator Strategies:
Code Block:
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get("https://developer.syntecx.org/ptcl_ebills/signin.php")
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input#email"))).send_keys("Asad_Ullah#stackoverflow.com")
driver.find_element_by_css_selector("input#password").send_keys("Asad_Ullah")
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[src^='https://www.google.com/recaptcha/api2/anchor?']")))
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "span.recaptcha-checkbox.goog-inline-block.recaptcha-checkbox-unchecked.rc-anchor-checkbox"))).click()
driver.switch_to_default_content()
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button.btn.btn-primary.block.full-width.m-b"))).click()
Browser Snapshot:

Try do it:
Create a variable that get the main window:
mainWin = driver.current_window_handle
When u need to switch to main window:
driver.switch_to.window(mainWin)

Don't switch to the iframe.
Everything you need is in #g-recaptcha-response and [data-sitekey] which are both in main context.

You can simply wait for the checkbox to show the done icon and then the wait will be over
Like in the code below
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox()
driver.get("http://somedomain/url_that_delays_loading")
try:
element = WebDriverWait(driver, 100).until(
EC.presence_of_element_located((By.ID, "myDynamicElement"))
)
finally:
driver.quit() here

Here I found a solution when you encounter a reCaptcha in a login page.
Simple approach that works very good for me. Check out for login button in a login page. If it lost it's existance, than you can proceed to the next stage. So, my solution does not require any timeout. Happy automating your stuff :)
def check_exists_by_xpath(xpath):
try:
self.driver.find_element_by_xpath(xpath)
except NoSuchElementException:
return False
return True
login_btn_xpath = 'your login button xpath'
while True:
print('You can solve recaptcha during this period and hit the login button.')
if check_exists_by_xpath(login_btn_xpath) is False:
break
# TODO: you can go along from here!

Related

selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference error while sending text to Search box in YouTube

I would like to create a code snippet for opening YouTube, accepting the cookies, finding the search bar, typing some string into it and finally clicking on the search button. Its not too hard but something is not working. I have tried using the WebDriverWait as well but still not working.
If I open the Google (of course from code) and doing the same procedure then everything works well. I have tried finding elements not only XPATH but also ID, and CSS_SELECTOR.
After the send_keys() function the error message is:
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
My code is:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_experimental_option('detach', True)
options.add_experimental_option('excludeSwitches', ['enable-logging'])
driver = webdriver.Chrome(options=options)
driver.get("https://www.youtube.com/")
driver.maximize_window()
cookie_accept = driver.find_element(By.XPATH, '//*[#id="content"]/div[2]/div[6]/div[1]/ytd-button-renderer[2]/yt-button-shape/button/yt-touch-feedback-shape/div/div[2]')
cookie_accept.click()
yt_searchbox = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="search"]')))
yt_searchbox.send_keys('Python Selenium')
I've tried without the WebDriverWait as well:
yt_search = driver.find_element(By.XPATH, '//*[#id="search"]')
yt_search.send_keys("Pyhton Selenium")
And it is not working either. I don't know what's going on.
The Search box within Youtube homepage is a dynamic element, so ideally to send a character sequence to the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
driver.get('https://www.youtube.com/')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input#search"))).send_keys("Python Selenium")
Using XPATH:
driver.get('https://www.youtube.com/')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#id='search']"))).send_keys("Python Selenium")
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser snapshot:

Can't locate this input element selenium

I am trying to locate the input “to” field on this webpage: Link here
(sorry about having to login to see it)
My code:
name_field = driver.find_element_by_xpath("//form[#id='compose-message']/div[6]/div/div/div[2]/div/div/textarea")
But that gives the following error:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//form[#id='compose-message']/div[6]/div/div/div[2]/div/div/textarea"}
(Session info: chrome=93.0.4577.63)
For some reason this page is the only page that has given me any issue.
Most of the textarea web elements are wrapped inside iframes.
Using explicit waits, does give your script a stability that it need.
Also prefer to use css selector over xpath.
In Selenium, to access elements which are inside of iframe, we need to switch the focus of webdriver to iframe. Like below
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "iframe[src^='/message/compose/']")))
and once you are inside this iframe you can interact with to input web element. Like below
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.NAME, "to"))).send_keys('something here')
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//span[text()='message']/following-sibling::div/descendant::textarea"))).send_keys('Some message')
Imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Also, remember to switch to default content when you are done with iframe.
driver.switch_to.default_content()
In order to work with elements which are outside of this iframe.
The Element you are trying to find is in an iframe. Was able to find and send text to the elements with below code.
from selenium import webdriver
from selenium.webdriver import ChromeOptions
import time
opt = ChromeOptions()
opt.add_argument("--user-data-dir=C:\\Users\\*****\\AppData\\Local\\Google\\Chrome\\User Data")
driver = webdriver.Chrome(executable_path="path to chromedriver.exe",options=opt)
driver.maximize_window()
driver.implicitly_wait(10)
driver.get("https://www.reddit.com/message/compose/")
driver.switch_to.frame(driver.find_element_by_xpath("//iframe[#class='saPujbGMyXRwqISHcmJH9']"))
driver.find_element_by_xpath("//input[#name='to']").send_keys("Sample Text")
driver.find_element_by_xpath("//div[#class='usertext']//textarea").send_keys("Sample Text")
time.sleep(5)
driver.quit()
First of all that element is inside an iframe. To access it you need to switch to that iframe. Also, your locator are not looking good.
Please try the following code:
wait = WebDriverWait(driver, 20)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[contains(#src,'message')]")))
name_field = wait.until(EC.visibility_of_element_located((By.XPATH, "//input[#name='to']")))
To use the above expected conditions you will need the following imports:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
use command CSS selector instead of XPath such as use below code.
yourwebsriver.find_elements_by_css_selector("textarea[name=text]").send_keys('something here')

Why does firefox keep getting hungup after the fourth run through

I am trying to create a bot that clicks on a link on my site, opens a tab, closes, and then continues the cycle.
This keeps getting hungup after the 4-5 cycle however.
I also cannot figure out how to simply close both of the tabs, and get the url again as to not have to quit.
Sorry if this is very basic, I am new to this. :)
I am using selenium, firefox webdriver
from time import sleep
from selenium import webdriver
from keyboard import press
while True:
browser = webdriver.Firefox()
browser.implicitly_wait(10)
browser.get('https://pridedogecoin.com/')
sleep(15)
login_link = browser.find_element_by_xpath('//*[#id="slider-6-slide-6-layer-13"]')
login_link.click()
sleep(5)
browser.quit()
sleep(15)
okay, you need to switch to first tab like below, also you should use close not quit, since close is meant to kill current window instance, where as quit will kill all of the browser instance :
import time
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
driver = webdriver.Firefox(executable_path = "D:\geckodriver.exe")
driver.maximize_window()
while True:
driver.refresh()
time.sleep(10)
driver.get("https://pridedogecoin.com/")
driver.implicitly_wait(60)
wait = WebDriverWait(driver, 30)
current_windows_handle = driver.current_window_handle
wait.until(EC.element_to_be_clickable((By.ID, "slider-6-slide-6-layer-13"))).click()
driver.switch_to.window(driver.window_handles[1])
time.sleep(5)
driver.close()
driver.switch_to.window(current_windows_handle)

Cannot scroll all the way down the page, because it keeps refreshing Selenium Python

I am trying to automate saving website's description and url. I loop the program and come to the function get_info(). Basically it need to add the first website on the google page I load and scroll down so when it executes again it can add other websites. The problem is that the program refresh the page everytime it executes get_info() and brings you back at the top.
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
import time
browser=webdriver.Firefox()
def get_info():
browser.switch_to.window(browser.window_handles[2])
description = WebDriverWait(browser, 10).until(
EC.presence_of_element_located((By.TAG_NAME, "h3"))
).text
site = WebDriverWait(browser, 10).until(
EC.presence_of_element_located((By.TAG_NAME, "cite"))
)
site.click()
url=browser.current_url
browser.back()
browser.execute_script("window.scrollBy(0,400)","")
To stop Chrome from auto-reloading , you can do this
driver.execute_cdp_cmd('Emulation.setScriptExecutionDisabled', {'value': True})
Link to read about this Chrome Devtools flag - here
The question already has been asked here

How to deal with nested iframes with Selenium and Python

I'm trying to automate a daily task, but I got stuck with some nested iframes. The script is working very well till the part of unselecting all document types, it fails.
I tried waiting until the element is clickable. Also, I tried accessing the element with id, css_selector and XPath but nothing helped.
I think the element is located within nested iframes, but can't get the correct sequence.
Here's my script:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import time
options = webdriver.ChromeOptions()
options.add_argument("disable-infobars")
#options.add_argument("--headless")
options.add_argument("--no-sandbox")
options.add_argument('--disable-dev-shm-usage')
#options.add_argument('--disable-gpu')
options.add_argument("--start-maximized")
options.add_experimental_option("excludeSwitches",["ignore-certificate-errors"])
options.add_argument("--disable-blink-features=AutomationControlled")
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
link = r"https://countyfusion10.kofiletech.us/countyweb/loginDisplay.action?countyname=TylerWV"
options.add_experimental_option("prefs", {
"download.directory_upgrade":True,
"safebrowsing.enabled":True,
"download.prompt_for_download":False,
"plugins.always_open_pdf_externally": True,
})
driver = webdriver.Chrome(executable_path='./chromedriver', options=options)
driver.get(link)
#Login as Guest
driver.find_element_by_xpath(r'//*[#id="maindiv"]/table[2]/tbody/tr[1]/td[2]/table/tbody/tr/td/center/input').click()
time.sleep(5)
#Accept Disclaimer
driver.switch_to.frame("bodyframe")
driver.find_element_by_id(r'accept').click()
time.sleep(30)
#Deselect All types
driver.switch_to.default_content()
boxes = WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,r'//*[#id="elemAllNamesJSP_20"]/table/tbody/tr/td[2]/span/input[1]')))
print(boxes)
I'm not sure you had to switch to the default content, anyway since you got out of the bodyframe iframe to access the "boxes" element you have switch to the first parent iframe first, then to the inner parent and then to the inner child. Like this:
driver = webdriver.Chrome(executable_path='./chromedriver', options=options)
driver.get(link)
#Login as Guest
driver.find_element_by_xpath(r'//*[#id="maindiv"]/table[2]/tbody/tr[1]/td[2]/table/tbody/tr/td/center/input').click()
#Accept Disclaimer
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it(By.NAME,"bodyframe"))
driver.find_element_by_id(r'accept').click()
#Deselect All types
driver.switch_to.default_content()
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it(By.NAME,"bodyframe"))
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it(By.NAME,"dynSearchFrame"))
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it(By.NAME,"criteriaframe"))
boxes = WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,r'//*[#id="elemAllNamesJSP_20"]/table/tbody/tr/td[2]/span/input[1]')))
print(boxes)
I'm not sure what you trying to achieve by printing the web element "boxes" with print(boxes)...
Also I removed the hardcoded delays. Especially the 30 seconds sleep, you don't need it. Use webdriver explicit wait instead.
It will work much faster and still stable.

Resources