How can I click button "View profile" on this page.
Problem: when I want to copy XPath of "View profile" button, the button disappeares.
https://www.linkedin.com/sales/search/people?savedSearchId=515913166
[]
Another image
This is my code. Reads email and password from config.txt file.
Goes to linkedin site, then goes to Sales Navigator
My code
import os, random, sys, time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup
browser = webdriver.Chrome('driver/chromedriver')
browser.get('https://www.linkedin.com/uas/login')
file = open('config.txt')
lines = file.readlines()
username = lines[0]
password = lines[1]
elementID = browser.find_element_by_id('username')
elementID.send_keys(username)
elementID = browser.find_element_by_id('password')
elementID.send_keys(password)
visitingSalesNavID = '/sales/homepage/'
fullLink = 'https://www.linkedin.com' + visitingSalesNavID
browser.get(fullLink)
time.sleep(4)
SavedSearchesID = '/sales/search/saved-searches/people'
fullLink = 'https://www.linkedin.com' + SavedSearchesID
browser.get(fullLink)
time.sleep(4)
SavedSearchID = '/sales/search/people?savedSearchId=515913166'
fullLink = 'https://www.linkedin.com' + SavedSearchID
browser.get(fullLink)
time.sleep(4)
browser.find_element_by_xpath('/html[1]/body[1]/div[5]/main[1]/div[1]/div[1]/section[1]/div[1]/ol[1]/li[1]/div[2]/div[1]/div[1]/div[1]/article[1]/section[1]/div[2]/ul[1]/li[1]/div[1]/div[2]/div[1]/div[1]/button[1]/li-icon[1]/*').click()
You have to freeze the DOM:
1) Open the dev console (F12 key)
2) Select sources tab
3) click F8 to freeze the DOM
Then do whatever you do with that
You don’t need to look element’s detail. You can click with the text. Try below code after open menu.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
element = WebDriverWait(driver, 50).until(
EC.presence_of_element_located((By.XPATH, "//*[contains(., 'View profile')]")))
element.click()
Related
I was happily extracting information from a page, which requires clicking on a link to see the email. Everything was working fine until I turned off my laptop for a few days, I'm trying to do this task again but for some reason, it isn't working anymore :( (I never changed my code or anything). Please HELP!
Before anything, I tried to replace that click with Actions, tried to use Java, changed the webdriver, changed the waiting time, tried with CSS_SELECTOR, XPATH, etc (previously worked only with LINK_TEXT) but nothing I do seems to work. Here's my code, I censored my information:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
DRIVER_PATH = '/Users/andre/Desktop/geckodriver'
driver = webdriver.Firefox(executable_path=DRIVER_PATH)
driver.get('https://membership.icsc.com/login')
USERNAME = 'XXXXXXXXXXXX'
PASSWORD = 'XXXXXX'
#login first
login = driver.find_element(By.NAME, '_username').send_keys(USERNAME)
password = driver.find_element(By.NAME, '_password').send_keys(PASSWORD)
submit = driver.find_element(By.CSS_SELECTOR, "body > div > div > div > div > div > div > form > div.text-center > p:nth-child(1) > button").click()
main = driver.get("THEURLISTOOLONGTOPUTITHERE")
#thescraper666
def get_scarping(link):
driver.get(link)
try:
email = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.LINK_TEXT, "Click here to display email")))
email.click()
driver.implicitly_wait(5)
email2 = driver.find_element(By.XPATH, '/html/body/div[5]/div/div/div/div/div/div[2]/div[2]/div/div/div[1]/div[2]/a').text
print(email2)
except NoSuchElementException:
print("NO EMAIL")
return driver.current_url
links = ['A LIST WITH A BUNCH OF URLS']
scrapings = []
for link in links:
scrapings.append(get_scarping(link))
I am using Python Selenium to open https://www.walmart.com/ and then want to click "Create an account" tab under "Sign In Account" as shown in this Walmart website pic. The source code of button from Walmart website along with the image is as follows:
<a link-identifier="Create an account" class="no-underline" href="/account/signup?vid=oaoh">
<button class="w_C8 w_DB w_DE db mb3 w-100" type="button" tabindex="-1">Create an account</button>
</a>
Walmart Website Source code-Pic
My python code for opening https://www.walmart.com/ for accessing Create an account button and click on it is as follows:
import time
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
url = "https://www.walmart.com/"
s=Service('C:/Users/Samiullah/.wdm/drivers/chromedriver/win32/96.0.4664.45/chromedriver.exe')
driver = webdriver.Chrome(service=s)
driver.get(url)
driver.maximize_window()
elems = driver.find_element(By.XPATH, '//button[normalize-space()="Create an account"]')
time.sleep(3)
elems.click()
However I am getting this error of ElementNotInteractableException as shown by this Error Pic.
Can anyone guide me what is wrong with my code/approach?.
Thanks in advance
You can not click this element directly, you have to hover over "Sign In" element to make this element appeared.
It is also highly recommended to use expected conditions.
This should work:
import time
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
url = "https://www.walmart.com/"
s=Service('C:/Users/Samiullah/.wdm/drivers/chromedriver/win32/96.0.4664.45/chromedriver.exe')
driver = webdriver.Chrome(service=s)
wait = WebDriverWait(driver, 20)
actions = ActionChains(driver)
driver.get(url)
driver.maximize_window()
sign_in_btn = wait.until(EC.visibility_of_element_located((By.XPATH, "//div[text()='Sign In']")))
actions.move_to_element(sign_in_btn).perform()
time.sleep(0.5)
wait.until(EC.visibility_of_element_located((By.XPATH, '//button[normalize-space()="Create an account"]'))).click()
I have compiled a code that would Create an Account on https://www.walmart.com/ using selenium python. The code opens Walmart website goes to Create an account tab, fills the required details and click on Create Account button. However, the problem is Walmart's Human verification challenge which appears randomly at any stage. Following are the snapshots that shows the Human verification challenge appearing just after opening the URL or after clicking on create account button:
I have found a code on stackoverflow to bypass this challenge (shown below) but it didn;t work fro me.
element = driver.find_element(By.CSS_SELECTOR, '#px-captcha')
action = ActionChains(driver)
action.click_and_hold(element)
action.perform()
time.sleep(100)
action.release(element)
action.perform()
time.sleep(0.2)
action.release(element)
For reference my python code is as follows:
import time
import requests
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.action_chains import ActionChains
url = "https://www.walmart.com/"
first_name = "chuza"
last_name = "789"
email_id = "chuza789#gmail.com"
password = "Eureka1#"
options = Options()
s=Service('C:/Users/Samiullah/.wdm/drivers/chromedriver/win32/96.0.4664.45/chromedriver.exe')
driver = webdriver.Chrome(service=s, options=options)
driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")
driver.execute_cdp_cmd("Page.addScriptToEvaluateOnNewDocument", {
"source":
"const newProto = navigator.__proto__;"
"delete newProto.webdriver;"
"navigator.__proto__ = newProto;"
})
wait = WebDriverWait(driver, 20)
actions = ActionChains(driver)
driver.get(url)
sign_in_btn = wait.until(EC.visibility_of_element_located((By.XPATH, "//div[text()='Sign In']")))
actions.move_to_element(sign_in_btn).perform()
time.sleep(0.5)
wait.until(EC.visibility_of_element_located((By.XPATH, '//button[normalize-space()="Create an account"]'))).click()
f_name = driver.find_element(By.ID, 'first-name-su')
l_name = driver.find_element(By.ID, 'last-name-su')
email = driver.find_element(By.ID, 'email-su')
pswd = driver.find_element(By.ID, 'password-su')
f_name.send_keys(first_name)
driver.implicitly_wait(2)
l_name.send_keys(last_name)
driver.implicitly_wait(2.5)
email.send_keys(email_id)
driver.implicitly_wait(3)
pswd.send_keys(password)
driver.implicitly_wait(2.8)
###
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[data-automation-id='signup-submit-btn']"))).click()
Can anyone please guide me how to solve Walmart's Robot or Human? Challenge and how to integrate it with my existing code? Thanks in advance.
The simplest way to overcome this is to check for this element presence.
In case this element appears - perform the page reloading until this element is no more present.
Something like this:
human_dialog = driver.find_elements(By.XPATH, "//div[#aria-labelledby='ld_modalTitle_0']")
while human_dialog:
driver.refresh()
time.sleep(1)
human_dialog = driver.find_elements(By.XPATH, "//div[#aria-labelledby='ld_modalTitle_0']")
I want to click on the check box, but my code is not working. Steps which I want to follow is
Open website
Select Profession e.g:- Dental
License Type e.g :- Dentist
Enter a alphabet with * to get all the records
Click on the check box
Click on search button
Script also ask for image verification for which I am unable to process, any help would be appriciated
home_page = 'https://forms.nh.gov/licenseverification/'
import requests
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
import csv
from shutil import copyfile
import datetime
import subprocess
import time
import multiprocessing
import sys
import subprocess
current_date=datetime.date.today()
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import WebDriverException
from selenium.webdriver.chrome.options import Options
driver = webdriver.Chrome(r"C:\Users\psingh\AppData\Local\Programs\Python\Python38-32\chromedriver.exe")
driver.get(home_page)
select_prof = WebDriverWait(driver, 30).until(
EC.presence_of_element_located((By.ID, "t_web_lookup__profession_name")))
Select(select_prof).select_by_value('Dental')
select_lic_type = WebDriverWait(driver, 30).until(
EC.presence_of_element_located((By.ID, "t_web_lookup__license_type_name")))
# select profession criteria value
Select(select_lic_type).select_by_value('Dentist')
time.sleep(1)
send = WebDriverWait(driver, 30).until(
EC.presence_of_element_located((By.ID, "t_web_lookup__last_name"))
)
send.send_keys('A*')
time.sleep(1)
# click on check box
driver.find_element_by_xpath("/html/body/div[1]").click()
# click search button
driver.find_element_by_id("sch_button").click()
# wait to get the result
time.sleep(1)
The check box which you are trying to click is inside the iframe. First switch to that iframe and then try to click on that checkbox.
Below code works fine in pycharm and windows 10.
driver = webdriver.Chrome(PATH)
url = 'https://forms.nh.gov/licenseverification/'
driver.get(url)
driver.maximize_window()
sleep(5)
select_prof = driver.find_element(By.ID, 't_web_lookup__profession_name')
Select(select_prof).select_by_value('Dental')
select_lic_type = driver.find_element(By.ID, "t_web_lookup__license_type_name")
# select profession criteria value
Select(select_lic_type).select_by_value('Dentist')
sleep(1)
send = driver.find_element(By.ID, "t_web_lookup__last_name")
send.send_keys('A*')
sleep(1)
# click on check box
driver.switch_to.frame(0)
captcha = driver.find_element(By.CSS_SELECTOR, '.recaptcha-checkbox-border')
captcha.click()
# click search button
driver.find_element(By.XPATH, "sch_button").click()
sleep(1)
Working in python, selenium, Win 7, I want to click on sowcomment button after it is able to clickable that is located on this webPage, handled by wait then I want to click on showmore Comments button to see more comments in order to scrape more comments. After first button, i am able to extract comments.
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
import selenium.common.exceptions
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
import selenium.webdriver.support.ui as UI
driver = webdriver.Firefox()
driver.get("http://www.aljazeera.com/news/2016/07/erdogan-west-mind-business-160729205425215.html")
wait = UI.WebDriverWait(driver, 10)
next_page_link = wait.until(
EC.element_to_be_clickable((By.ID, 'showcomment')))
next_page_link.click()
wait = UI.WebDriverWait(driver, 20)
next_page_link2 = wait.until(
EC.element_to_be_clickable((By.LINK_TEXT, 'Show more comments')))
next_page_link2.click()
v = driver.find_elements_by_class_name('gig-comment-body')
print(v)
for h in v:
print(h.text)
but second button is unable to click rather giving the exception:
selenium.common.exceptions.TimeoutException:
What is the problem?
I think you should try using execute_script() to perform click as below :
next_page_link2 = wait.until(
EC.element_to_be_clickable((By.XPATH, '//div[contains(text(),"Show more comments")]')))
#now click this using execute_script
driver.execute_script("arguments[0].click()", next_page_link2)
Hope it helps...:)