Selenium Scroll down on Newly Opened Body - python-3.x

I am working with Selenium python, and would like to scroll down on a page after clicking an element. Clicking the element gives a pop-up menu, and then when I try some scrolling methods they scroll on the original page.
I have selected the body of the new element that has appeared and tried this:
from selenium import webdriver
from selenium import Keys
driver = webdriver.get(url)
button = driver.find_element_by_xpath(pathto)
button.click()
body = driver.find_element_by_css('body')
body.send_keys(Keys.PAGE_DOWN)
This does not actually do anything as is. However, if I first manually click on the new popup element before doing body.send_keys(Keys.PAGE_DOWN), it does scroll.
One way I have resolved this is using ActionChains to right click on the body before scrolling down. Directly using .click() on the body does not work as it clicks a link in the body.
Are there any better ways to do this? In particular, I am somewhat afraid that the right click menu that opens up when I use my current method may mess with the rest of the scraping. Is there at least a method to guarantee that it does not interfere? Another solution could be somehow "selecting" or clicking on the body element within selenium, so are there any suggestions for this?

You could try using END key instead of DOWN key. It's worth a shot:
from selenium.webdriver.common.keys import Keys
driver = webdriver.get(url)
button = driver.find_element_by_xpath(pathto)
button.click()
driver.find_element_by_tag_name('body').send_keys(Keys.END)
You also may need to click the pop-up in order to focus it first:
from selenium.webdriver.common.keys import Keys
driver = webdriver.get(url)
button = driver.find_element_by_xpath(pathto)
button.click()
# click popup menu to get it into focus
popup_menu = driver.find_element_by_xpath(pathToPopupMenu)
popup_menu.click()
driver.find_element_by_tag_name('body').send_keys(Keys.END)

Related

Python- selenium webdriver is not able to find these buttons on modal window

Not able to click x or continue to flights
Here is the url: https://www.kayak.com/flights/ORD-JFK/2020-09-13/2020-09-20?sort=bestflight_a
using css = '.Button-No-Standard-Style.close-button'
I was using the following to find and click it:
driver.find_element_by_css_selector('.Button-No-Standard-Style.close-button').click()
What I found out was the button had dynamic ID. I had to click the button based on partial ID comparison.
I used:
cl_bt = driver.find_element_by_xpath("//*[contains(#id, 'covid-loading-dialog-close')]")
where 'covid-loading-dialog-close' was the part of the ID that was not changing.

how do I identify if a popup comes up

I'm making a sneaker bot that simulates a checkout on shoe palace (using air force 1s as an example in this program). Basically what I'm having trouble with is after about 5 seconds after opening the website (it's not a consistent amount of time) I get a popup basically just advertising signing up for their club or whatever and it screws up my code because the button doesn't get pressed so the script terminates. I've tried using time.sleep(5) to hopefully wait for it to popup and then simulating pressing the escape key to close it but it just doesn't work. Is there a way that I can recognize when the popup comes up so I can run a piece of code that closes it immediately? Below I have the code I'm using currently which in an instance where the popup doesn't come up works perfectly. Any help is GREATLY appreciated as I've been stumped with this problem.
from selenium import webdriver
import time
# from pynput.keyboard import Key, Controller
# keyboard = Controller()
PATH = "C:\Program Files (x86)\chromedriver.exe"
driver = webdriver.Chrome(PATH)
#driver.delete_all_cookies()
#open product page (CHANGE THIS)
#driver.get('https://www.shoepalace.com/product/nike/ck7214-100/air-force-1-07-lv8-2-mens-lifestyle-shoe-white-white/')
driver.get('https://www.shoepalace.com/product/nike/ck7214-100/air-force-1-07-lv8-2-mens-lifestyle-shoe-white-white/')
time.sleep(.025)
#secondary size
button = driver.find_element_by_xpath("/html/body/div[2]/div[4]/form/div[1]/button[5]")
button.click()
#primary size
button = driver.find_element_by_xpath("/html/body/div[2]/div[4]/form/div[1]/button[7]")
button.click()
#add to cart
button = driver.find_element_by_xpath("/html/body/div[2]/div[4]/form/div[2]/input")
button.click()
#checkout
button = driver.find_element_by_xpath("/html/body/div[2]/div/div[2]/div/div/div/ul[2]/li[1]")
button.click()
You can create a method where you should:
Identify if the pop-up is displayed
Close it if so
Wait until pop-up is not displayed
Keep in mind that your code runs line by line, so if you know when the pop up can appear, you can add your method there. If not, based on your code structure is a tricky one because you will need to add it before each action.

How to close popover/popup by <a> element using Selenium

I'm working on a webscraper for https://www.grailed.com/designers/jordan-brand/hi-top-sneakers. When the page has opened a popup for login comes up. Searching through the web design I can locate the X element to close the browser like so: temp = WebDriverWait(driver, 10).until(ec.visibility_of_element_located((By.CLASS_NAME, 'close'))). If I go more into this there is an and element. I have tried using .click() on the element (with class 'close'), as well as the SVG and path elements. None of these close the box, and there is no button or other element of this kind for the X. What can I do to close this popover? I'm not sure if I need to find a button-ish element to click, but I can't find one like that. I've looked at a couple of questions and articles (https://stackoverflow.com/questions/61923909/trying-to-close-popover-python-selenium-glassdoor, https://sqa.stackexchange.com/questions/5310/how-to-close-pop-up-window-in-selenium-webdriver, https://saucelabs.com/resources/articles/the-selenium-click-command) but can't find a solution.
You can do double click with actions for resolving this problem
WebDriverWait(driver, 20).until(ec.visibility_of_element_located((By.XPATH,
'//a[#class = 'close']/*[name()='svg']')))
close = driver.find_element_by_xpath("//a[#class = 'close']/*[name()='svg']")
actionChains = ActionChains(driver)
actionChains.double_click(close).perform()
And the Java code for this:
new WebDriverWait(driver, 20)
.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//a[#class = 'close']/*[name()='svg']")));
WebElement close = driver.findElement(By.xpath("//a[#class = 'close']/*[name()='svg']"));
Actions action = new Actions(driver);
action.doubleClick(close).build().perform();
I actually just ran into the same issue myself. The above solution didn't work for me, so I'll post what I did here:
Essentially, I clicked by the coordinates instead of by trying to find the button element.
Note: some webpages don't display the popup until you try to click something else, so I had to first attempt to click another element. If you need to do that, make sure to wait a second or so for the popup to load.
Then, you can do the rest via ActionChain:
elem = driver.find_element_by_class_name("CLASSNAME")
ac = ActionChains(driver)
ac.move_to_element(elem).click().perform()
You'll want to encase this in a try-except block for extra safety.
Credit to Dirk Bergstrom for providing part of the solution here: Clicking at coordinates without identifying element

Scrolling down with Python Selenium on part of the web page

So I have a web page where a part of the page is a user list. A user can scroll down this list, but scrolling only works if the mouse pointer is located above this list. Keys like page down or end do not work.
I can locate the element using Selenium and Xpath without any issue, but I don't know how to perform the scrolling in selenium as it only works when the focus is on that specific web element.
I've tried sending keys like page down and end or code like
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
Anybody got any ideas?
Because you mentioned you can locate the element, you could try scrolling to the specific element, instead of using document.body.scrollHeight as your scroll target.
element = driver.find_element(someLocatorHere)
driver.execute_script("arguments[0].scrollIntoView(true);", element)
Hope this helps a bit.
Scrolling can be achieved in two different ways.
Javascript
elementId = driver.find_element_by_id("elementId")
driver.execute_script("arguments[0].scrollIntoView();", elementId)
Action class
actions = ActionChains(driver)
actions.move_to_element(elementId).perform()

not able to click on a button for scraping using selenium and python

THIS IS THE BUTTON CODE WHICH I WANTED TO CLICK
I used this code.
login_btn1 = driver.find_elements_by_tag_name('button')
login_btn1.submit()
but they are showing the error:
AttributeError: 'list' object has no attribute 'submit'
can someone help me with this please.Where am i going wrong?
By using find_elements_by_tag_name you are looking for all 'button' elements which will return a list.
You need to replace this with find_element_by_tag_name to retrieve a single 'button' element. (this will be the first button element found)
Alternatively, in the likely situation that there is more than one button on the page, you can select by class name using find_element_by_class_name('cdp-view-all-button') or another method from the selenium docs.
When you write this :
login_btn1 = driver.find_elements_by_tag_name('button')
Now login_btn1 is a list of Web Element. you can have method which are applicable for list.
Since you want to click on button and button is a web element. you can't do login_btn1.click() .
If you want to click on a single button, first you have to find the locator then there are multiple ways to click on it such as :
button = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#data-track-app='discovery']")))
button.click()
If the provided xpath is Unique in DOM then the above code would work.

Resources