Have a look at this website- https://mops.twse.com.tw/mops/web/t146sb05 .
Enter the value 6150 in the text box and press Enter.
See that the url does not change, but the HTML changes. I want to scrape the values 117,129 and -13.57% from this page. I have entered the value and pressed Enter using Selenium, but don't know how to proceed further.
from selenium.webdriver.common.keys import Keys
import requests
from lxml import html
from selenium.webdriver.common.by import By
DRIVER_PATH = 'E:/Anaconda3/chromedriver'
driver = webdriver.Chrome(executable_path=DRIVER_PATH)
driver.get('https://mops.twse.com.tw/mops/web/t146sb05')
input_entering = driver.find_element_by_xpath('//*[#id="co_id"]').click()
new_driver = driver.find_element_by_xpath('//*[#id="co_id"]').send_keys(6150, Keys.RETURN)
In your code you are trying to perform action on object which not valid click & send_keys attribute.
Please try below solution ::
driver.get('https://mops.twse.com.tw/mops/web/t146sb05')
element=driver.find_element_by_xpath('//*[#id="co_id"]')
element.send_keys("6150",Keys.RETURN)
Related
Im using Selenium with Python3 on a Service Now Website.
So the process is as follows: selenium loads up the ServiceNow URL and then I use sendKeys to automate typing in of username and password, then the page is loaded which has a table of incidents I need to extract. Unfortunately I have to login in every single time because of the group policy I have.
This works up until I have to find the dynamically rendered Javascript table with data and I can't for the life of me seem to find it. I even tried to put a sleep in there for 15 seconds to allow it to load.
I also double checked the XPaths and Id / Class names and they match up. When I print query.page_source I don't see anything rendered by JS.
I've used beautiful soup too but that also doesn't work.
Any ideas?
from time import sleep
from collections import deque
from selenium import webdriver
from selenium.webdriver.support.ui import Select # for <SELECT> HTML form
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
query = webdriver.Firefox()
get_query = query.get("SERVICENOW URL")
query.implicitly_wait(10)
login_username = query.find_element_by_id('username')
login_password = query.find_element_by_id('password')
login_button = query.find_element_by_id('signOnButton')
username = "myUsername"
password = "myPassword"
login_username.send_keys(username)
login_password.send_keys(password)
login_button.click()
sleep(10)
incidentTableData = []
print(query.page_source)
// *** THESE ALL FAIL AND RETURN NONE ***
print(query.find_elements())
tableById = query.find_element_by_id('service-now-table-id')
tableByXPath = query.find_element_by_xpath('service-now-xpath')
tableByClass = query.find_element_by_id('service-now-table-class')
Since it's a dynamically rendered Javascript table, I would suggest you to implement explicit wait in your code.
so instead of this :
tableById = query.find_element_by_id('service-now-table-id')
tableByXPath = query.find_element_by_xpath('service-now-xpath')
tableByClass = query.find_element_by_id('service-now-table-class')
re-write these lines like this :
wait = WebDriverWait(query, 10)
service_now_with_id = wait.until(EC.element_to_be_clickable((By.ID, "service-now-table-id")))
service_now_with_xpath = wait.until(EC.element_to_be_clickable((By.XPATH, "service-now-xpath")))
service_now_with_class = wait.until(EC.element_to_be_clickable((By.ID, "service-now-table-class")))
You are gonna need to use the below imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as E
PS :- service_now_with_id, service_now_with_xpath, service_now_with_class, these are web elements returned by explicit waits. you may wanna have to interact with them as per your requirement meaning, clicking on it or sending keys or whatever.
Hey I am trying to make a simple Insta bot using selenium.I reached up to my following list by automating but now I don't know how I can scroll down the list of my following/followers. I want to grab the account from my following and followers list and compare it and make a list of the accounts who havenot followed me back.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from time import sleep
from secrets import password,username
class Instabot:
def __init__(self,username,password):
self.driver = webdriver.Chrome(executable_path="C:\\Users\\user\\PycharmProjects\\chromedriver_win32\\chromedriver.exe")
self.driver.get("https://www.instagram.com")
sleep(5)
self.driver.find_element_by_xpath('//*[#id="react-root"]/section/main/article/div[2]/div[1]/div/form/div[2]/div/label/input').send_keys(username) #searching the username box and giving it username
self.driver.find_element_by_xpath('//*[#id="react-root"]/section/main/article/div[2]/div[1]/div/form/div[3]/div/label/input').send_keys(password)#searching the password box and giving it password
self.driver.find_element_by_xpath('//*[#id="react-root"]/section/main/article/div[2]/div[1]/div/form/div[4]/button').click() #clicking login button
wait=WebDriverWait(self.driver,10)
notNowButton = wait.until(
lambda d: d.find_element_by_xpath('//button[text()="Not Now"]'))#it will click on first notnow button
notNowButton.click()
next_not_now=wait.until(lambda notnow:notnow.find_element_by_xpath('//button[text()="Not Now"]'))
next_not_now.click() #it will click on second not now button
def get_unfollowers(self):
wait = WebDriverWait(self.driver, 10)
clickprofile= wait.until(lambda a: a.find_element_by_xpath('//*[#id="react-root"]/section/nav/div[2]/div/div/div[3]/div/div[5]/a'))
clickprofile.click()
following_list = wait.until(EC.element_to_be_clickable((By.XPATH, "//a[contains(#href,'/following')]")))
following_list.click()#IT clicks the following and gives window of following listy
print("CLicked following list")
my_bot=Instabot(username,password)
my_bot.get_unfollowers()
I saw about execute_script() But I dont know what to put under those brackets.
Well I figure it out. This code works for me now!
fBody = self.driver.find_element_by_css_selector("div[class='isgrP']")
scrolling_times=(numoffollowers/4)
scroll=0
scroll_count = scrolling_times+5 # You can use your own logic to scroll down till the bottom
while scroll < scroll_count:
self.driver.execute_script(
'arguments[0].scrollTop = arguments[0].scrollTop + arguments[0].offsetHeight;',
fBody)
sleep(2)
scroll += 1
You can use something like that :
self.driver.execute_script("window.scrollTo(0, Y)")
with Y : how much you want to scroll in pixel.
It's up to you to see how far down you have to go down.
I cannot use send keys correctly to input values.
I would like to be able to insert text into the text box.
Tried 2 different methods
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument("--test-type")
driver = webdriver.Chrome('/Users/.../Documents/chromedriver')
driver.get('http://codepad.org/')
text_area = driver.find_element_by_id('textarea')
text_area.send_keys("This text is send using Python code.")
from selenium import webdriver
driver = webdriver.Chrome('/Users/.../Documents/chromedriver')
driver.get( 'https://www.google.com/maps/dir///#36.0667234,-115.1059052,15z')
driver.find_element_by_xpath("//*[#placeholder='Choose starting point, or click on the map...']").click()
driver.find_element_by_xpath("//*[#placeholder='Choose starting point, or click on the map...']").clear()
driver.find_element_by_xpath("//*[#placeholder='Choose starting point, or click on the map...']").send_keys("New York")
Put a value into the fields i am trying to put the values in
Here is the code that you can use, which will wait for the element to present and then set the value in the input box.
WebDriverWait(driver,30).until(EC.visibility_of_element_located((By.XPATH, "(//input[#class='tactile-searchbox-input'])[1]"))).send_keys("new york")
BTW you need below imports in order to work with explicit wait used in the above code.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I am trying to write a script to automate the search for multiple entries. Here is the input:
<mirna-1>
<mirna-2>
<mirna-3>
:
:
The website address is https://www.mirnet.ca/miRNet/faces/upload/MirUploadView.xhtml
How can I fill mirna-1,.... into miRNA list box. I have made the following attempts but failed:
gecko = os.path.normpath('~/geckodriver')
nirnet_Results = []`
browser = Browser(executable_path=gecko)
browser.visit('https://www.mirnet.ca/miRNet/faces/home.xhtml')
# click miRNA_list
xpath = '//*[#id="j_idt39:j_idt55"]'
browser.find_by_xpath(xpath).click()
time.sleep(1)
#Select Human
xpath = '//*[#id="form:j_idt34"]/div[3]/span'
browser.find_by_xpath(xpath).click()
time.sleep(1)
xpath = '//*[#id="form:j_idt34_1"]'
browser.find_by_xpath(xpath).click()
time.sleep(1)
# select miRBase ID type
xpath = '//*[#id="form:j_idt38"]/div[3]/span'
browser.find_by_xpath(xpath).click()
time.sleep(1)
xpath = '//*[#id="form:j_idt38_1"]'
browser.find_by_xpath(xpath).click()
time.sleep(1)
# select Tissue type
xpath = '//*[#id="form:sourceOpt"]/div[3]/span'
browser.find_by_xpath(xpath).click()
time.sleep(1)
xpath = '//*[#id="form:sourceOpt_1"]'
browser.find_by_xpath(xpath).click()
time.sleep(1)
# select target type
xpath = '//*[#id="form:targetOpts"]/div[3]/span'
browser.find_by_xpath(xpath).click()
time.sleep(1)
xpath = '//*[#id="form:targetOpts_1"]'
browser.find_by_xpath(xpath).click()
time.sleep(1)
# Fill miRNA list
xpath = '//*[#id="form:listData"]'
browser.find_by_xpath(xpath).click()
time.sleep(1)
for mir in known_mir:
browser.fill('form:listData',mir)
browser.fill('form:listData','\n')
I am getting success in selecting options in starting drop-down list and when I try to paste a list of names in the miRNA list, I am getting all the names like this:
<mirna-1><mirna-2><mirna-3>....
while the right format is each name should in the next line. How to do this. Kindly suggest. Thanks.
You can fill this form Using Selenium Chrome Driver and python.If you haven't use selenium before and would like use selenium.However here is the official website of selenium.
To install selenium use command line
pip install -U selenium
You need download chrome driver from following website website Chromedriver based on compatibility,such as which chrome browser supports by the chrome driver.Always prefer the latest versions of chrome driver and chrome browser.
Here is the selenium code which fills the form
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
from selenium import webdriver
import time
driver=webdriver.Chrome('path to chrome driver exe') #example : 'C:\chromedriver\chromedriver.exe'
driver.get("https://www.mirnet.ca/miRNet/faces/upload/MirUploadView.xhtml")
#select Organism here
WebDriverWait(driver,5).until(EC.element_to_be_clickable((By.ID,"form:j_idt34_label"))).click()
time.sleep(2)
element=WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.XPATH,"//div[#class='ui-selectonemenu-items-wrapper']//ul[#id='form:j_idt34_items']//li[contains(.,'H. sapiens (human)')]")))
ActionChains(driver).move_to_element(element).send_keys(Keys.UP).send_keys(Keys.DOWN).send_keys(Keys.ENTER).perform()
#select ID type here
WebDriverWait(driver,5).until(EC.element_to_be_clickable((By.ID,"form:j_idt38_label"))).click()
time.sleep(2)
element=WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.XPATH,"//div[#class='ui-selectonemenu-items-wrapper']//ul[#id='form:j_idt38_items']//li[contains(.,'miRBase ID')]")))
ActionChains(driver).move_to_element(element).send_keys(Keys.UP).send_keys(Keys.DOWN).send_keys(Keys.ENTER).perform()
#select Tissue here
WebDriverWait(driver,5).until(EC.element_to_be_clickable((By.ID,"form:sourceOpt_label"))).click()
time.sleep(2)
element=WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.XPATH,"//div[#class='ui-selectonemenu-items-wrapper']//ul[#id='form:sourceOpt_items']//li[contains(.,'Exosomes [1250]')]")))
ActionChains(driver).move_to_element(element).send_keys(Keys.UP).send_keys(Keys.DOWN).send_keys(Keys.ENTER).perform()
#select Target type here
WebDriverWait(driver,5).until(EC.element_to_be_clickable((By.ID,"form:targetOpts_label"))).click()
time.sleep(2)
element=WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.XPATH,"//div[#class='ui-selectonemenu-items-wrapper']//ul[#id='form:targetOpts_items']//li[contains(.,'Genes')]")))
ActionChains(driver).move_to_element(element).send_keys(Keys.UP).send_keys(Keys.DOWN).send_keys(Keys.ENTER).perform()
items=['mirna-1','mirna-2','mirna-3','mirna-4']
for item in items:
element=WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//textarea[#id='form:listData']")))
element.send_keys(item)
element.send_keys(Keys.RETURN)
Output:
Although I would recommend selenium for something like this, it seems you want to stick with splinter. The following fix using splinter. When reproducing this issue, each time browser.fill() is run it replaces the text already written. Since you didn't include what known_mir was in your code, I am assuming it is a list (correct me if I'm wrong). I tried combining what you wanted to put into the form and it worked.
Replace:
for mir in known_mir:
browser.fill('form:listData',mir)
browser.fill('form:listData','\n')
With:
browser.fill('form:listData', '\n'.join(mir for mir in known_mir))
All my code does is combine each of the strings in known_mir.
I want to enter text into a textarea. The problem is the textarea is only created and shows up in the html after clicking twice on a visual element in the browser.
For this visual element Dubbelklik om je tekst te typen, I cannot find an element in the html that refers to it and perform a double_click(). There are only containers, that either are unclickable elements, or allow clicks, but don't perform an action.
Up until here it works and the visual cue to double click appears slightly below the center:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait as wait
driver = webdriver.Chrome()
driver.get("https://www.hallmark.nl/kaarten/verjaardag-man/")
#wait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[href*='https://www.hallmark.nl:443/kaarten/verjaardag-man/grappig-m/make-that-the-cat-wise/happy-bursdeej-to-jou-3415094.aspx']"))).click()
wait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[href*='https://www.hallmark.nl:443/kaarten/verjaardag-man/grappig-m/hallmark/een-jaguar-voor-je-verjaardag-3346861.aspx']"))).click()
wait(driver, 10).until(EC.element_to_be_clickable((By.ID, "btnShowSizepicker"))).click()
wait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//span[text()='Standaard']"))).click()
wait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//div[contains(#class, 'showDesktop')]//button[contains(text(),'Binnenkant')]"))).click()
But from then on I can't seem to find anything to make selenium click on the element.
I have tried clicking the things I can locate in the html:
# elem = driver.find_element_by_xpath("//div[contains(#class, 'canvasAnchor')]").click()
This gives an element not interactable error. I have also tried using an offset for a findable element:
elem = driver.find_element_by_class_name("canvas-container")
print(elem.location)
print(elem.size)
action = webdriver.common.action_chains.ActionChains(driver)
action.move_to_element_with_offset(elem, 0.5*elem.location.get('x'), (0.5*elem.location.get('y'))) #should click in the middle of this container
action.double_click()
action.perform()
But for different elements and values of x and y, I can only get it to do either nothing, or click on the contact sidebar (which highlights the word 'Neem').
If I click twice manually, a textarea appears in the html and I can execute send_keys('hello world')
The element where you have to click is actually canvas, so what you have to do is that move cursor to the position where double click is needed and then click there.
I tried this over your code and worked for me:
from selenium.webdriver.common.action_chains import ActionChains
elem = driver.find_element_by_xpath('//*[#class="canvasWrapper active"]//*[#class="canvas-container"]')
ActionChains(driver).move_to_element_with_offset(elem, 70, 60).double_click().perform()
Note:You may have to wait for the canvas element to load properly. I just tried from prompt.
You can read about this mouse action here.
UPDATE:
I noticed that somehow the double click was behaving as single click for me, so changed to this.
from selenium.webdriver.common.action_chains import ActionChains
import time
elem = driver.find_element_by_xpath('//*[#class="canvasWrapper active"]//*[#class="canvas-container"]')
ActionChains(driver).move_to_element_with_offset(elem, 70, 60).click().perform()
time.sleep(1)
ActionChains(driver).move_to_element_with_offset(elem, 70, 60).click().perform()