This question already has answers here:
How can I make sure if some HTML elements are loaded for Selenium + Python?
(2 answers)
Do we have any generic function to check if page has completely loaded in Selenium
(7 answers)
Closed 4 years ago.
I'm try to write process on UI with selenium and python3.x. In some case, I would like to wait particular query finish before go to next step. since during the query finish, there is no element I can pick up to use and return document.readyState as complete does not work for me. Is it possible to use selenium wait.until to do it? like
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
WebDriverWait(driver, 120).until(lambda x: "query return 0")
WebDriverWait(driver, 120).until(EC."something to wait for")
Shorter solution:
WebDriverWait(driver, 10).until(lambda d: d.execute_script("return jQuery.active == 0"))
If the query is being run with Ajax and you are using JQuery, you could try using the javascript executor in order to wait for all ajax requests in flight to finish after intitiating the request:
driver.execute_script("return jQuery.active == 0")
You could wrap this into a custom ExpectedCondition for re-use with code like this:
class ajax_requests_to_finish(object):
def __call__(self, driver):
return driver.execute_script("return jQuery.active == 0")
This can be used like this:
# First perform an action that fires an ajax request
wait = WebDriverWait(driver, 10)
wait.until(ajax_requests_to_finish())
Disclaimer: I'm a java programmer, I don't really know any python. I drew upon this page as an example for how to create custom wait conditions in Python, since I only know how to do this in Java. Sorry if my code is incorrect, this sourced page describes a more generic working example. https://selenium-python.readthedocs.io/waits.html
Edit: As sers pointed out in another answer, this potential solution is possible as a one-liner
Related
Using selenium find element (using xpath) method to close the popup but it is not able to detect it.
time.sleep(10)
driver.fin_element(By.XPATH,"XPATH").close()
I have also use time.sleep and webdriver wait methods but it not working
Website: www.multcloud.com
time.sleep(10)
webdriver wait
ec. Element traceable method
Also try Find elements but it is showing empty list.
Tried find element using class_name,xpath, full xpath,cs locator,link text
Try the below code
driver.get("https://www.multcloud.com/")
sleep(4)
driver.switch_to.frame("layui-layer-iframe1")
You could use explicit wait in the place of sleep
sleep(3)
button = driver.find_element(By.XPATH,"((//div[#class='sale-close-img'])[2])")
button.click()
Imports
from time import sleep
from selenium import webdriver
from selenium.webdriver.common.by import By
In python I wrote:
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=chrome_options)
driver.get(url)
try:
WebDriverWait(driver, 10).until(
lambda driver: driver.execute_script('return document.readyState') == 'complete')
except se.TimeoutException:
return False
# Start Parsing
Even though I have waited for readyState for some websites when I parse it I see that there is no checkbox. But, If I add time.sleep(5) Before parsing for the same website I get that there is a checkbox.
My question is, how can I have a general solution that works with the majority of websites? I can't just write time.sleep(5) as some websites might need much more and some might finish within 0.001 seconds (which will have bad impact on performance...)
I just want to stimulate a real browser and not to handle anything before the refresh button appears again (which means everything was loaded).
Ideally web applications when accessed through get(), returns the control to the WebDriver only when document.readyState equals to complete. So unless the AUT(Application under Test) behaves otherwise, the following line of code is typically an overhead:
WebDriverWait(driver, 10).until(lambda driver: driver.execute_script('return document.readyState') == 'complete')
However, as per your test requirements you can configure the pageLoadStrategy either as:
none
eager
normal
You can find a detailed discussion in What is the correct syntax checking the .readyState of a website in Selenium Python
At this point, it is to be noted that using time.sleep(secs) without any specific condition to achieve defeats the purpose of Automation and should be avoided at any cost.
Solution
The generic approach that would work with all the websites is to induce WebDriverWait as per the prevailing test scenario. As an example:
To wait for the presence of an element you need to invoke the expected_conditions of presence_of_element_located()
To wait for the visibility of an element you need to invoke the expected_conditions of visibility_of_element_located()
To wait for the element to be visible, enabled and interactable such that you can click it you need to invoke the expected_conditions of element_to_be_clickable()
This question already has answers here:
Unable to locate element of credit card number using selenium python
(2 answers)
NoSuchElementException: Message: Unable to locate element while trying to click on the button VISA through Selenium and Python
(3 answers)
org.openqa.selenium.NoSuchElementException: Returned node (null) was not a DOM element when trying to locate card-fields-iframe by CssSelector
(3 answers)
Closed 3 years ago.
i'm using selenium with python to register into a website.
My problem is that when i need to insert the credit card number and the security number, though i copy the xpath of the field, wait for the field to be visible and clickable, python raise a timeout exception becouse he didn't find any field with that xpath. The html code of the field is that:
<input id="cardnumber" autocomplete="cc-number" type="tel" pattern="[0-9]*" placeholder="1111 2222 3333 4444" data-encrypted-name="number" class="form-control">
UPDATE: Maybe I understand.
When I print out driver.page_source I notice that it's not loaded properly, and now I understand why Selenium can't find nothing, neither by xpath, neither by name or everything. I notice that what is loaded can be clicked and all, but some fields are not loaded.
So why Selenium behave like that?
UPDATE AGAIN: Solved, the solution is that i need to switch iframe every time.
Try This By Adding Some Wait:
from selenium.webdriver.common.by import By
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait as Wait
driver = webdriver.Chrome()
driver.maximize_window()
driver.get("Your Website")
try:
cardnumber = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.ID, "#cardnumber")))
except:
print('Sorry!')
cardnumber.send_keys('XXXXX')
You are passing wrong ID. You dont need to pass #when you are trying to identify element with its ID.
cardnumber = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.ID, "cardnumber")))
Please check locator strategies :https://selenium-python.readthedocs.io/locating-elements.html
This question already has answers here:
How to get rid of the hardcoded sleep()?
(3 answers)
Closed 3 years ago.
Selenium has a lot of actions, e.g., driver.find_element_by_xpath('XXX').click(), driver.find_element_by_xpath('XXX').send_keys(). I would like to add some features (e.g. random pause) to make them real (like humans).
Adding time.sleep() before each action is not Pythonic and takes too many lines (one line before an action). In addition, we want the pause time for different types of actions are different (send_keys(input_text) should be longer than click()).
I tried the following but failed:
from selenium.webdriver.common.action_chains import ActionChains
def pause_wrapper(func, min_time=0, max_time=None):
"""Before execute func, we pause some time [min_time, max_time]."""
def wrapper(*args, **kwargs):
pause_time = random.uniform(min_time, max_time)
print(f'We are going to pause {pause_time} s.')
time.sleep(pause_time)
return func(*args, **kwargs)
return wrapper
ActionChains.send_keys = pause_wrapper(ActionChains.send_keys, min_time=5, max_time=10)
ActionChains.click = pause_wrapper(ActionChains.click, min_time=1, max_time=3)
driver.find_element_by_xpath('XXX').send_keys(Keys.ENTER)
My questions are:
1. Why my method fails (no pause and no print result)?
2. How to approach it?
To clarify: I want to force hard pause, not the other around (as wrongly flagged as similar).
you could define a wait like:
from selenium.webdriver.support.ui import WebDriverWait
wait = WebDriverWait(driver_instance, 1000)
and then use it to perform certain waits like:
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support import expected_conditions as EC
wait.until(
EC.url_contains("http://example.com")
)
do keep in mind that the wait will throw a TimeoutException after a while.
In some cases time.sleep is an acceptable solution.
More about the expected conditions can be consulted in the selenium documentation
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
driver = webdriver.Firefox()
driver.get("https://google.com")
#driver.implicitly_wait(10)
WebDriverWait(driver,10)
print("waiting 10 sec")
driver.quit()
It just quits after page loading. the waits have no effect at all!
demo : https://www.youtube.com/watch?v=GocfsDZFqk8&feature=youtu.be
any help would be highly appreciated.
If you want a pause 10 seconds, use time.sleep():
import time
time.sleep(10) # take a pause 10 seconds
Note: WebDriverWait(driver,10) doesn't work like that. Instead you can use it like this:
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait
# this will wait at least 10 seconds until url will contain "your_url"
WebDriverWait(driver, 10).until(EC.url_contains(("your_url")))
but it will not wait exactly 10 seconds, only until expected_conditions will be satisfied.
Also: as source code us tells:
def implicitly_wait(self, time_to_wait):
"""
Sets a sticky timeout to implicitly wait for an element to be found,
or a command to complete. This method only needs to be called one
time per session. To set the timeout for calls to
execute_async_script, see set_script_timeout.
:Args:
- time_to_wait: Amount of time to wait (in seconds)
:Usage:
driver.implicitly_wait(30)
"""
...
driver.implicitly_wait(10) also is used for waiting elements, not to pause script.
PS: it is always a good practice to use WebDriverWait instead of hard pause, because with WebDriverWait your test will be more quickly, since you don't have to wait the whole amount of time, but only until expected_conditions will be satisfied. As I understood, you are just playing arround at the moment, but for the future WebDriverWait is better to use.
At least with Python and Chrome driver, my experience is that even when using
WebDriverWait you STILL need to use time.sleep for things to work reliably. using implicitly_wait doesnt work. I need to put time.sleep(1) after each operation, or sometimes things don't fire off.
So we had this same problem, what we did was modify the driver class in selenium with a decorator to sleep for .44 seconds on functions that we modified in the get_driver() function. In this case we wanted to wait to find elements by class, name and id before selenium inputted our desired content. Worked like a charm.
def sleep_decorator(func):
def wrapper(*args, **kwargs):
time.sleep(.44) # Added side-effect
return func(*args, **kwargs) # Modified return
return wrapper
def get_driver():
driver = webdriver.Chrome()
driver.find_element_by_id = sleep_decorator(driver.find_element_by_id)
driver.find_element_by_name = sleep_decorator(driver.find_element_by_name)
driver.find_element_by_class_name = sleep_decorator(driver.find_element_by_class_name)
driver.find_elements_by_id = sleep_decorator(driver.find_elements_by_id)
driver.find_elements_by_name = sleep_decorator(driver.find_elements_by_name)
driver.find_elements_by_class_name = sleep_decorator(driver.find_elements_by_class_name)
return driver
By using this WebDriverWait(driver,10) , you have declared the Explicit wait. This is just the declaration , you are not using explicit wait at all.
For make use of Explicit wait, you will have to bind the above code with EC which is Expected condition.
Something like :
wait = WebDriverWait(driver,10)
element = wait.until(EC.element_to_be_clickable((By.NAME, 'q')))
element.send_keys("Hi Google")
You can refer this link for explicit wait : Explicit wait
Note that time.sleep(10) is worst/extreme type of explicit wait which sets the condition to an exact time period to wait. There are some convenience methods provided that help you write code that will wait only as long as required. WebDriverWait in combination with ExpectedCondition is one way this can be accomplished.