Actually, i have a script with python 3.10, selenium, undetected-chromedriver and i want to click on two buttons on this webpage:
https://keepa.com/#!
Buttons are french flag, and ".fr" here:
And html code for these two buttons are here:
-first highlighted is flag fr
-second is country ".fr"
I've tested somes part of script, but not work actually:
For language flag fr:
driver.find_element(By.XPATH, "(//span[#id='lang_fr'])").click()
driver.find_element(By.ID,"lang_fr").click()
For country .fr:
driver.find_element(By.XPATH, "(//span[#setting='4'])").click()
driver.find_element(By.XPATH, "(.//span[contains(text(), '.fr')])").click()
I have somes errors on results, "element not found", or this:
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
Anyone can help me or have an idea where is the problem please?
Thanks for help, bye!
You have to click on default flag first and then there will be 2 button which you can click like below:
Code:
driver_path = r'C:\\Users\\****\\***\\Desktop\\Automation\\chromedriver.exe'
driver = webdriver.Chrome(driver_path)
driver.maximize_window()
driver.get("https://keepa.com/#!")
wait = WebDriverWait(driver, 30)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div#currentLanguage .languageMenuText"))).click()
flag = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "img#lang_fr")))
flag.click()
lang = wait.until(EC.visibility_of_element_located((By.XPATH, "//span[text()='.fr']")))
lang.click()
Imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You can try the below code it will help you (this is using java though, ).
Actions actions = new Actions(driver);
WebElement ctr;
ctr= driver.findElement(By.xpath("//*[contains(text(),'YOUR DESIRED TEXT')]"));
Thread.sleep(1000);
actions.click(spatial).perform();
Related
I am trying to scrape data from this website using Selenium.
There are three features in data, "Value", "Net change" and "percent change", including values for net and percentage changes for 1, 3, 6, and 12 months. I want to fetch 1 month's net change and percent change. For that, I need to click on the check boxes and click on the update button.
Now, I performed these actions using selenium's find element by XPath method but for percent change, I needed to use the ActionChains command, as I was getting "Element not clickable error".
When I execute the code, all three features should occur in the downloaded csv. But that's not happening. I am just able to fetch "Value" and "1 Month Net change". If anyone knows, may I know, why the is not getting updated and or how to fix it? Thanks
My code:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome('chromedriver',chrome_options=chrome_options)
driver.get("https://beta.bls.gov/dataViewer/view/timeseries/CUUR0000SA0")
soup = BeautifulSoup(driver.page_source, "html.parser", from_encoding='utf-8')
driver.find_element(By.XPATH, '/html/body/div[2]/div/div/div[4]/div/div[1]/form/div[2]/fieldset/div[1]/table/tbody/tr[1]/td[1]/label/input').click() //1 month net change
element = WebDriverWait(driver, 60).until(EC.element_to_be_clickable((By.XPATH, '// [#id="percent_monthly_changes_div"]/table/tbody/tr[1]/td[1]/label/input')))
ActionChains(driver).move_to_element(element).click().perform() //1 month percent change
driver.find_element(By.XPATH, '/html/body/div[2]/div/div/div[4]/div/div[1]/form/div[4]/input').click() //update button
driver.find_element(By.XPATH, '//*[#id="csvclickCU"]').click() //download csv button
The website is showing N/A in the column of 1 Month Net change.
if you still not getting 1 month % change value you can do
driver.execute_script('document.querySelector("#percent_monthly_changes_div > table > tbody > tr:nth-child(1) > td:nth-child(1) > label > input").click()')
instead of:
element = WebDriverWait(driver, 60).until(EC.element_to_be_clickable((By.XPATH, '// [#id="percent_monthly_changes_div"]/table/tbody/tr[1]/td[1]/label/input')))
ActionChains(driver).move_to_element(element).click().perform()
this might not be the optimal solution, but it works fine.
and 1 month net change value is not given from the website itself.
To click on the elements with text as 1-Month Net Change and 1-Month % Change using ActionChains will be an overhead and you can avoid it easily.
Ideally, you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
driver.get("https://beta.bls.gov/dataViewer/view/timeseries/CUUR0000SA0")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[value='1N']"))).click()
driver.find_element(By.CSS_SELECTOR, "input[value='1P']").click()
Using XPATH:
driver.get("https://beta.bls.gov/dataViewer/view/timeseries/CUUR0000SA0")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#value='1N']"))).click()
driver.find_element(By.XPATH, "//input[#value='1P']").click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser Snapshot:
using imports:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait
using this def to navigate to the exact iFrame that the button is under :
def changeIframetoManagement():
wait2 = WebDriverWait(driver, 10)
wait2.until(EC.frame_to_be_available_and_switch_to_it((By.ID, "moduleManagement")))
my attempt to achieve that task :
changeIframetoManagement()
driver.find_element_by_css_selector("body > div.bootbox.modal.fade.show-order-info-bootbox-
container.in > div > div > div.modal-body > button").click()
im also tried to reach by xPath like this
driver.find_element_by_xpath("//html/body/div[5]/div/div/div[1]/button").click()
that is the last i frame i was in
def changeIframetoOrders():
wait3 = WebDriverWait(driver, 10)
wait3.until(EC.frame_to_be_available_and_switch_to_it((By.NAME,
"orderInfo")))
After switching into the iframe try this:
driver.find_element_by_css_selector("div.modal-dialog button.bootbox-close-button").click()
UPD
According to your last comments, since you previously switched into iframe located by orderInfo name, before switching into iframe located by moduleManagement id you have yo switch to the default content, like this:
changeIframetoOrders()
driver.switch_to.default_content()
changeIframetoManagement()
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.modal-dialog button.bootbox-close-button"))).click()
See if this xpath works:-
driver.find_element_by_xpath(".//button[contains(#class,'close-button')]").click()
The x button is in moduleManagement iframe.
and the goal is to click on it using Selenium.
as per OP
wait3.until(EC.frame_to_be_available_and_switch_to_it((By.NAME,
"orderInfo")))
looks like he was in a different iframe orderInfo.
So I would suggest whenever you are done with any iframe stuff, always switch the control back to default content like below :
driver.switch_to.default_content()
Now again you will have to switch to moduleManagement iframe like below :
wait = WebDriverWait(driver, 10)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.ID, "moduleManagement")))
and click on x button like this :
wait.until(EC.element_to_be_clickable((By.XPATH, "//button[contains(#class, ''bootbox-close-button) and text()='x']"))).click()
I want to select an option that displays only after you've clicked on the dropdown (see attached image). I have been able to click on the dropdown to get the list, but have not been able to figure out how to click on an option, say option 1, 'Last Day' after the list comes into picture.
Here's what I have so far:
from selenium import webdriver
binary = FirefoxBinary('C:\\Program Files\\Firefox Developer Edition\\firefox.exe')
cap = DesiredCapabilities().FIREFOX
cap["marionette"] = True
url='https://www.glassdoor.com/Job/jobs.htm?suggestCount=0&suggestChosen=false&clickSource=searchBtn&typedKeyword=data+sc&sc.keyword=data+scientist&locT=C&locId=1154532&jobType='
driver = webdriver.Firefox(firefox_binary=binary, capabilities=cap, executable_path=GeckoDriverManager().install())
driver.get(url=url)
driver.implicitly_wait(10)
driver.maximize_window()
# clicking on dropdown
d = driver.find_element_by_id('filter_fromAge')
d.click()
I also tried using the following code (found on another SO answer) but it did not work either:
element = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR,"ul#css-1dv4b0s ew8xong0")))
I'm new to web scraping and not really familiar with XPATHs and how to deal with actions. Help appreciated!
You can use Java script to click on your element. As element is present in DOM but its only appears when click on drop down, so normal click method mau work or may not. But with JS it will always click. Can use below code:
day = driver.find_element_by_xpath("//span[contains(text(),'Last Day')]") #Identify your element
driver.execute_script("arguments[0].click();", day) # CLick it with help of JS
out Put:
You can try the following approach:
driver.get('https://www.glassdoor.com/Job/boston-data-scientist-jobs-SRCH_IL.0,6_IC1154532_KO7,21.htm')
driver.maximize_window()
expand_element = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.ID, 'filter_fromAge')))
expand_element.click()
target_text = 'Last 3 Days'
target_element = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//ul[#class="css-1dv4b0s ew8xong0"]/li/span[text()="{}"]'.format(target_text))))
target_element.click()
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, '//div[#id="filter_fromAge"]/span[text()="{}"]'.format(target_text))))
To select the option with text as Last Day that displays only after clicking on the dropdown you need to induce WebDriverWait for the element_to_be_clickable() and you can use the following Locator Strategy:
Using XPATH:
driver.get('https://www.glassdoor.com/Job/jobs.htm?suggestCount=0&suggestChosen=false&clickSource=searchBtn&typedKeyword=data+sc&sc.keyword=data+scientist&locT=C&locId=1154532&jobType=')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div#filter_fromAge>span"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[#id='PrimaryDropdown']/ul//li//span[#class='label' and contains(., 'Last Day')]"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser Snapshot:
i tried so many xpath in selenium but all were unable to click the element and always give me a error element not found or element not interactable
how to solve it any help would be appreciated
Here is xpath of the Element is give Below:
(//a[#href='javascript:void(0)' and #class='select2-choice select2-default'])[1]
Try with ".//*[#id='s2id_search_input']/a"
Wait for element to be clickable before click on it:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# ...
wait = WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#s2id_search_input a.select2-choice'))).click()
With scroll:
element = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#s2id_search_input a.select2-choice')))
driver.execute_script('arguments[0].scrollIntoView()', element)
element.click()
Click using JavaScript:
element = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#s2id_search_input a.select2-choice')))
driver.execute_script('arguments[0].click()', element)
I am trying to click on the "Next" button at the bottom of a BusinessWire web page. I have some code that goes from the BusinessWire homepage to my desired search results page. I want to be able to click on that page's "Next" button, but I get the error message telling me that the "Next" element is not clickable at point(X,Y). The next button is at the bottom of the window. For some reason the
browser.execute_script("window.scrollTo(0, document.body.scrollHeight);")
command is not scrolling as I would expect. Because the window is not scrolling as expected the element is not visible to be clicked (at least I believe that is the problem). I use this same command twice earlier in the code and it works fine in those two instances. Any help on getting the window to scroll would be greatly appreciated. Here is my code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
def BusinessWire():
browser = webdriver.Chrome()
browser.get('http://www.businesswire.com/portal/site/home/')
search_box_element = browser.find_element_by_id('bw-search-
input')
search_box_element.clear()
search_box_element.send_keys('biotechnology')
search_box_element.send_keys(Keys.ENTER)
browser.execute_script("window.scrollTo(0,
document.body.scrollHeight);")
search_box_element_two = browser.find_element_by_id('more-news-
results')
search_box_element_two.click()
browser.execute_script("window.scrollTo(0,
document.body.scrollHeight);")
time.sleep(5)
next_page_click_element = WebDriverWait(browser, 10).until(
EC.presence_of_element_located((By.XPATH, '//*[#id="more-news-pagination"]/div/div[1]/div/a'))
)
next_page_click_element.click()
Try using scrollIntoView of the element you want to scroll to, sorry this is in Java, but should work the same in python:
driver.manage().window().maximize();
driver.get("http://www.businesswire.com/portal/site/home/");
wait = new WebDriverWait(driver, 10);
driver.findElement(By.id("bw-search-input")).clear();
driver.findElement(By.id("bw-search-input")).sendKeys("biotechnology");
driver.findElement(By.id("bw-search-input")).sendKeys(Keys.ENTER);
WebElement clicklink = driver.findElement(By.id("more-news-results"));
((JavascriptExecutor) driver).executeScript("arguments[0].scrollIntoView(true);", clicklink);
clicklink.click();
Thread.sleep(1000);
WebElement clicknext = wait.until(ExpectedConditions.presenceOfElementLocated(By.xpath("//*[#id=\"more-news-pagination\"]/div/div[1]/div/a")));
((JavascriptExecutor) driver).executeScript("arguments[0].scrollIntoView(true);", clicknext);
clicknext.click();
StackOverFlow won't let me post any more comments? The url is in the code. But the homepage url is not the problem page. The problem page is the search page after the home page. The search page can only be gotten to through a search on the homepage. My code does all this.