I am trying to capture an Error which would restart my program and change proxy but I am unable to catch the error as its stored like this and classes are dynamically named :
<p class="g4Vm4">By signing up, you agree to our <a target="_blank" href="https://help.instagram.com/581066165581870">Terms</a> . Learn how we collect, use and share your data in our <a target="_blank" href="https://help.instagram.com/519522125107875">Data Policy</a> and how we use cookies and similar technology in our <a target="_blank" href="/legal/cookies/">Cookies Policy</a> .</p>
so I am trying to catch the xpath by this function but I am un able to do so.
def has_error(browser):
try: #/*[contains(text(), 'technology')]/html/body/span/section/main/div/article/div/div[1]/div/form/p"
browser.find_element_by_xpath("/html/body//*[contains(text(),'technology')]")
return False
except: return True
if not has_error(browser):
print('Error found! , aborted!')
browser.quit()
os.execv(sys.executable, ['python'] + sys.argv)
To Handle dynamic element use WebDriverwait and following Xpath Startegy.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions
element=WebDriverWait(driver,30).until(expected_conditions.element_to_be_clickable((By.XPATH,'//p[contains(.,"technology")]')))
print(element.text)
You can check if the source of the web-page contains special text.
if 'By signing up, you agree to our ' in browser.page_source:
pass
# TODO Exception
Related
I'm trying to automate facebook marketplace posts. But i'm struggling to upload pictures to it.
I already locate the element. When i click the element it will show the 'box' showing the file manager so that i can click on the folders and then the desired image.
ele = wait.until(EC.element_to_be_clickable((By.XPATH,'//*[#id="rc.js_c"]/div/div[1]/div[5]/div[2]/div/div/div/div/div[1]/div/div/span/div/a/div[2]')))
ele.click()
But when i try this:
ele.send_keys('/file_path/rasp.jpeg')
It raises this exception:
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
I also tried using the os library:
ele.send_keys(os.getcwd() + '/home/br1/Downloads/rasp.jpeg')
Getting the same exception error.
The html code where the element is visible (element used in code):
<div class="_3jk">
which is the parent of (where the element is not visible):
<input accept="image/*" multiple="" name="composer_photo" title="Elige un archivo para subir" data-testid="add-more-photos" display="inline-block" type="file" class="_n _5f0v" id="js_wg">
Here is all the code if you want to try it:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By 10
# driver protocols
options = Options()
options.add_argument('disable-notifications')
options.add_argument('start-maximized')
driver = webdriver.Chrome(options=options, executable_path='/chromedriver')
wait = WebDriverWait(driver,10)
# url
driver.get('http://facebook.com/marketplace')
driver.implicitly_wait(10)
# logging
driver.find_element_by_id('email').send_keys('username')
driver.find_element_by_id('pass').send_keys('password')
driver.find_element_by_id('u_0_2').click()
# entering marketplace
driver.find_element_by_xpath('//*[contains(text(), "Vender algo")]').click()
driver.find_element_by_xpath('//*[contains(text(), "Artículo en venta")]').click()
ele = wait.until(EC.element_to_be_clickable((By.XPATH,'//*[#id="rc.js_c"]/div/div[1]/div[5]/div[2]/div/div/div/div/div[1]/div/div/span/div/a/div[2]')))
ele.send_keys('/file_path/rasp.jpeg')
Any ideas and suggestions will be aprecciate it.
I'm a Linux user.
You should try using the input to send the file path rather the div.
Try the below.
ele = wait.until(EC.presence_of_element_located((By.XPATH,'//input[#name="composer_photo" and #type="file"]')))
ele.send_keys("file_to_be_uploaded")
I am trying to automate an extraction of stock prices in my broker website because yahoo and google finance have delays. But i need the code to wait for the 'home-broker' to be online so it can continue with scraping...
Here is my code:
expected = 'online'
while True:
try:
driver.find_element_by_xpath('//*[#id="spnStatusConexao"]').text == expected
except NoSuchElementException:
print('offline')
else:
print('online')
But, while testing it, it prints 'online' even when the homebroker displays 'offline' message.
I need to print 'offline' when the xpath text is equal to: offline . And to print 'online' when xpath text is equal to: online.
EDIT:
outter HTML:
<span id="spnStatusConexao" hover="DV_bgConexao" class="StatusConexao online">online</span>
XPath:
//*[#id="spnStatusConexao"]
Full XPath:
/html/body/form/div[9]/div/div/p[2]/span
expected_conditions in Python has a built in operation for this called text_to_be_present_in_element. The below code snippet will wait for the span element to contain the text online:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
WebDriverWait(driver, 10).until(EC.text_to_be_present_in_element((By.ID, "spnStatusConexao"), 'online'))
If this does not work, you can try invoking WebDriverWait on the presence_of_element_located and include the text in your XPath query:
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//span[#id='spnStatusConexao' and contains(text(),'online')]")))
I am still quite amateur at python, I am trying to scrape data from a website using selenium
<small class="fxs_price_ohl"> <span>Open 1.29814</span> <span>High 1.29828</span> <span>Low 1.29775</span> </small> </div> </div> </li> <script type="application/ld+json">
trying to obtain the data Open 1.29814, High 1.29828 and Low 1.29775 from the html code above^
count_element = browser.find_element_by_xpath("//small[#class='fxs_price_ohl']//span")
print(count_element.text)
I'm using selenium with python, this is my code ^
But count_element.text prints empty, how to get the data Open 1.29814, High 1.29828 and Low 1.29775
Use
"find_elements_by_xpath"
if you want to retrieve multiple elements.
count_elements = browser.find_elements_by_xpath("//small[#class='fxs_price_ohl']//span")
for ele in count_elements:
print(ele.text)
You can also use a css selector of class for the parents with descendant combinator and type selector for the child spans but you also need a wait condition as page is slow loading
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
browser = webdriver.Chrome()
browser.get('https://www.fxstreet.com/rates-charts/gbpusd')
before_text = ''
while True: #this could be improved with a timeout
elements = [i for i in WebDriverWait(browser,20).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, ".fxs_chart_cag_cont .fxs_price_ohl span")))]
elem = elements[-1]
if elem.text != before_text:
break
print([elem.text for elem in elements])
Before asking, I find answer through google for 2hours. But there's no answer for me.
I use selenium with python
I apply below q/a answer to my code but nothing text printed.
XPath query to get nth instance of an element
What I want to get is "Can't select"
<li data-index="5" date="20190328" class="day dimmed">
<a href="#" onclick="return false;">
<span class="dayweek">Tuesday</span>
<span class="day">28</span>
<span class="sreader">Can't select</span>
</a>
</li>
I use xpath because I need to repeat
I should do this.
The HTML code above is a simple change
day_lists = driver.find_elements_by_xpath('//li')
Nothing printed and there's no error
for day_list in day_lists:
print(day_list.find_element_by_xpath('//span[#class="sreader"]').text)
++++ 2019/3/24/16:45(+09:00)
When I test with below code
print(day_list.find_element_by_xpath('.//span[#class="sreader"]/text()'))
Error comes out. Why there's no such element?
selenium.common.exceptions.NoSuchElementException:
Message: no such element: Unable to locate element:
{"method":"xpath","selector":".//span[#class="sreader"]/text()"}
If nothing printed and there's no error then required text might be hidden or not generated yet.
For first case you might need to use get_attribute('textContent'):
day_lists = driver.find_elements_by_tag_name('li')
for day_list in day_lists:
print(day_list.find_element_by_xpath('.//span[#class="sreader"]').get_attribute('textContent'))
For second case:
from selenium.webdriver.support.ui import WebDriverWait as wait
day_lists = driver.find_elements_by_tag_name('li')
for day_list in day_lists:
print(wait(driver, 10).until(lambda driver: day_list.find_element_by_xpath('.//span[#class="sreader"]').text)
Note that in both cases you need to add leading dot in XPath:
'//span' --> './/span'
To print the desired text e.g. 선택불가 from the <span> tag you can write a function as follows:
def print_text(my_date):
print(driver.find_element_by_xpath("//li[#class='day dimmed'][#date='" +my_date+ "']//span[#class='sreader']").get_attribute("innerHTML"))
Now you can call the function with any date as follows:
print_text("20190328")
Here are the solutions. If this does not work then I predict the element might present either in separate window or frame.
CSS:
li[class='day dimmed'][date='20190328'] .sreader
xpath:
To check if there are multiple windows use below
print(len(driver.window_handles))
To check if there are multiple frames use below
print(len(driver.find_elements_by_css_selector('iframe')))
Try the following options:
day_lists=WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, '//li[#class="day dimmed"]')))
for day_list in day_lists:
print(day_list.find_element_by_xpath('//span[#class="sreader"]').text)
OR
day_lists=WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, '//li[#class="day dimmed"]')))
for day_list in day_lists:
print(driver.execute_script("return arguments[0].innerHTML;", day_list.find_element_by_xpath('//span[#class="sreader"]')))
Please note that you need following imports as well.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
I’m trying to click an Upload from my Computer button on a page that has the source below.
I’m using selenium and tried several different approaches. The past failed approaches are commented out below, along with the current failed approach. The error that’s returned with the current approach is below.
Can anyone see what the issue might be and suggest how to solve it? I’m new to selenium so if someone can provide some explanation of what the html is doing and how their code solves the issue as well it would be really helpful for my understanding.
HTML code of the button:
<div class="hidden-xs">
<label for="fuUploadFromMyComputer" class="hidden">
Upload from my Computer
</label>
<input id="fuUploadFromMyComputer" type="file" name="upload">
<button id="btnUploadFromMyComputer"
class="center-block btn btn-white-fill btn-block "
data-resume-type="COMPUTER" type="submit">
<i class="zmdi zmdi-desktop-mac"></i>
Upload from my Computer
</button>
</div>
attempts:
# clicking upload button
# upload_btn = driver.find_element_by_id("fuUploadFromMyComputer")
# upload_btn = driver.find_element_by_css_selector(
# '.center-block.btn.btn-white-fill.btn-block')
# upload_btn = driver.find_element_by_link_text('Upload from my Computer')
# upload_btn.click()
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
WebDriverWait(driver, 20).until(EC.element_to_be_clickable(
(By.CSS_SELECTOR, "div.center-block btn.btn-white-fill.btn-block"))).click()
error:
---------------------------------------------------------------------------
TimeoutException Traceback (most recent call last)
<ipython-input-43-8fd80ff3c690> in <module>()
14 from selenium.webdriver.support import expected_conditions as EC
15
---> 16 WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div.center-block btn.btn-white-fill.btn-block"))).click()
17
18 time.sleep(3)
~/anaconda/envs/py36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py in until(self, method, message)
78 if time.time() > end_time:
79 break
---> 80 raise TimeoutException(message, screen, stacktrace)
81
82 def until_not(self, method, message=''):
TimeoutException: Message:
To click on the element with text as Upload from my Computer you need to induce WebDriverwait for the element to be clickable and you can use either of the following solutions:
CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button.center-block.btn.btn-white-fill.btn-block#btnUploadFromMyComputer"))).click()
XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#class='center-block btn btn-white-fill btn-block ' and #id='btnUploadFromMyComputer']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Selenium's click() does not support to operate on invisible element. Thus please double confirm the button is visible or not when your code intend to click it.
If the button is not visible, how do you click it hands-on? Thus change your script to following the human steps to make the button visible before you can click it.
Back to your failure on below code
WebDriverWait(driver, 20).until(EC.element_to_be_clickable(
(By.CSS_SELECTOR, "div.center-block btn.btn-white-fill.btn-block"))).click()
The reason is you give a wrong css selector which can't find any element from the page util reach the waiting timeout.
The correct css selector of the button can be any one of following:
button.center-block.btn.btn-white-fill.btn-block
button#btnUploadFromMyComputer
For C#, I used IJavaScriptExecutor to click on element. You may search this solution for Python syntax
public static void scrollElementToClick(IWebDriver driver, IWebElement element)
{
IJavaScriptExecutor ex = (IJavaScriptExecutor)driver;
ex.ExecuteScript("arguments[0].click();", element);
}