I'm trying to run a bot which likes every post on the instagram feed automatically.
I got the bot to the point, where it:
--> opens Google Chrome
--> Logs-in and
--> Likes the first post.
However the liking of the first post only works when I code the bot directly in the terminal without classes and creating a bot as an object. Or it works with Solution 2 - see my code snippet - but then after the like the browser closes immediately.
I already tried a lot of things:
--> Make sure that there is enough time between each step
--> Make sure that all the xpath are named the right way
--> Avoid syntax mistakes like "find__elements__by__(...)
But now I ran out of ideas..
Here is my Code:
# Import Selenium and Webdriver to connect and interact with the Browser
from selenium import webdriver
# Import time package and sleep function in order to let the website load when it opens
from time import sleep
# Import username and password
from secrets import username, password
# Create the LikingBot Class
class LikingBot():
def __init__(self):
# Past the Directory where the chromediver is in order to run it
self.driver = webdriver.Chrome('/Users/anwender/Coding/Python_Projects/Automation/Chrome_Webdriver/chromedriver')
# --> Type "python3 -i main.py" in the Terminal and start developing
def login(self):
# 1) Go to Instagram main website
self.driver.get('https://www.instagram.com/')
# 1.1) Give the Website time to load; otherwise the following steps wont work
sleep(2)
# 2) Select the Email Login Box
email_login = self.driver.find_element_by_xpath('//*[#id="react-root"]/section/main/article/div[2]/div[1]/div/form/div[2]/div/label/input')
# 3) Interact with the Email Login Box and type in the Email address or the Username
email_login.send_keys(username)
# 4) Repeat step 2) & 3) with the Password Box
password_login = self.driver.find_element_by_xpath('//*[#id="react-root"]/section/main/article/div[2]/div[1]/div/form/div[3]/div/label/input')
password_login.send_keys(password)
# 5) Click Log In Button
login_btn = self.driver.find_element_by_xpath('//*[#id="react-root"]/section/main/article/div[2]/div[1]/div/form/div[4]/button')
login_btn.click()
sleep(5)
# 6) Click Activate Messages Button if it pops up
activate_btn = self.driver.find_element_by_xpath('/html/body/div[4]/div/div/div[3]/button[1]')
activate_btn.click()
# Writing the automatic liking function
def auto_like(self):
"""
Here are the elements for the automation liking. It seems like every article has the same xpath of the
button. So the goal is to iterate through the articles with a for Loop, while the other attributes stay
the same!
//*[#id="react-root"]/section/main/section/div[2]/div[1]/div/article[1]/div[2]/section[1]/span[1]/button
//*[#id="react-root"]/section/main/section/div[2]/div[1]/div/article[2]/div[2]/section[1]/span[1]/button
//*[#id="react-root"]/section/main/section/div[2]/div[1]/div/article[3]/div[2]/section[1]/span[1]/button
//*[#id="react-root"]/section/main/section/div[2]/div[1]/div/article[..]/div[2]/section[1]/span[1]/button
"""
sleep(2)
# Start iteration
article_iteration = 1
# First Solution
'''
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
like_buttons stores the xpath of the first like button. However it is not working even the xpath is absolutely right.
like_buttons = self.driver.find_element_by_xpath('//*[#id="react-root"]/section/main/section/div[2]/div[1]/div/article['+str(article_iteration)+']/div[2]/section[1]/span[1]/button')
like_buttons.click()
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
'''
# Second Solution
'''
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
This like_buttons variable stores the same button element but takes the class name. It is working but after the like the browser closes.
like_buttons = self.driver.find_element_by_class_name('wpO6b ')
like_buttons.click()
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
'''
# Write the liking loop
'''
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
while True:
like_buttons.click()
article_iteration += 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
'''
# Initialize Bot
bot = LikingBot()
bot.login()
bot.auto_like()
Any help is appreciated & hopefully someone can help!
Related
So I was trying to make an automation script for a website called Quia. Although the problem is, whenever you go to a new page in the website and such, it generates a random new URL. I think this is making my code not be able to function. Is there any way to get around this? Here is my code:
'''
This is a program that will complete any quia quizzes for you.
How it works:
1. First it will complete many quizzes entering random answers.
2. Then it will complete a quiz, and use the answers it is given from the other quizzes.
How it is powered:
1. It uses the Selenium webdriver.
2. It uses the BeautifulSoup library.
Steps to build:
1. Get users information.
2. Access Chrome using Selenium and BeautifulSoup.
3. Go to the selected quiz
4. Login with user credentials
5. Open the quiz and complete it with random answers, leave tab open.
6. Repeat steps 4, 5, and 6 for as many times as the user wants.
7. Open a final quiz.
8. Use the answers from other quizzes for the final quiz.
9. Show user their score and close any tabs that the program opened.
'''
# Import libraries, selenium, and beautifulsoup
from selenium import webdriver
from bs4 import BeautifulSoup
from selenium.webdriver.common.keys import Keys
# Complete step 1:
# Get users information
creds_username = input("Enter your username: ")
creds_password = input("Enter your password: ")
info_quiz_count = int(input("How many practice quizzes do you want to complete? "))
info_quiz_url = input("Enter the url of the quiz you want to complete: ")
# Complete step 2:
# Access Chrome using Selenium and BeautifulSoup.
browser = webdriver.Chrome()
# Complete step 3:
# Go to the selected quiz in a new tab
def open_quiz(browser, info_quiz_url):
browser.find_element_by_tag_name('body').send_keys(Keys.CONTROL + 't')
browser.get(info_quiz_url)
# Complete step 4:
# Login with users credentials
def login(browser, creds_username, creds_password):
username = browser.find_element_by_css_selector('[name="tag_quiz_username"]')
username.send_keys(creds_username)
password = browser.find_element_by_css_selector('[name="tag_quiz_password"]')
password.send_keys(creds_password)
login_button = browser.find_element_by_css_selector('[name="tagStartQuiz"]')
login_button.click()
# Complete step 5:
# Answer with random answers. If it is multiple choice, choose the first one. If it is text, enter "This is a test."
def practice(browser):
# If it is a text question:
try:
# Find the text input box
text_input = browser.find_element_by_xpath("//*[starts-with(#id, 'tq_g_')]")
# Enter "This is a test"
text_input.send_keys("This is a test.")
except:
print("Can't find text input box, trying multiple choice.")
else:
# If it is a multiple choice question:
# Find the first multiple choice input box
multiple_choice_input = browser.find_element_by_css_selector("[id$='guess_0']")
# Click it
multiple_choice_input.click()
# Click submit
submit_button = browser.find_element_by_css_selector('[name="tag_submit_one_at_a_time"]')
# Complete step 6:
# Repeat steps 4, 5, and 6 for as many times as the user wants.
for i in range(info_quiz_count):
open_quiz(browser, info_quiz_url)
login(browser, creds_username, creds_password)
practice(browser)
Sorry for bad code, I am new to this stuff and just trying to find my way around.
So I get the messages from this line:
<pre class="_3Gy8WZD53wWAE41lr57by3 ">Sleep</pre>
My code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
PATH = 'C:\\Users\\User\\Desktop\\chromedriver.exe'
driver = webdriver.Chrome(PATH)
driver.get('https://www.reddit.com')
time.sleep(80) # TIME TO LOGIN IN
search = driver.find_element_by_class_name('_3Gy8WZD53wWAE41lr57by3 ')
print(driver.find_element_by_xpath(".//pre").text) # *LET'S CALL THIS 'S'*
And everything works, kinda. When I print: 's' it prints out the last message from that chat.
Note that whenever someone enters a message, it will be under the variable(class): '_3Gy8WZD53wWAE41lr57by3 '
My goal is to print out the first message from the that chat.
I had to edit it twice because of some mistakes that I had made
I would suggest 2 changes to your code which'll save you major frustration:
Avoid explicit sleep calls, instead, wait for presence of elements. This will allow your program to wait as little time as possible for the page you're trying to load.
Utilize css selectors instead of xpath --> you have much finer control over accessing elements, plus, your code becomes more robust and flexible.
In terms of execution, here's how that looks:
Wait up to 80 seconds for login:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
# Get the page, now the user will need to log in
driver.get('https://www.reddit.com')
# Wait until the page is loaded, up to 80 seconds
try:
element = WebDriverWait(driver, 80).until(
EC.presence_of_element_located((By. CSS_SELECTOR, "pre. _3Gy8WZD53wWAE41lr57by3"))
)
except TimeoutException:
print("You didn't log in, shutting down program")
driver.quit()
# continue as normal here
Utilize css selectors to find your messages:
# I personally like to always use the plural form of this function
# since, if it fails, it returns an empty list. The single form of
# this function results in an error if no results are found
# NOTE: utilize reddit's class for comments, you may need to change the css selector
all_messages = driver.find_elements_by_css_selector('pre._3Gy8WZD53wWAE41lr57by3')
# You can now access the first and last elements from this:
first_message = all_messages[0].text
last_message = all_messages[-1].text
# Alternatively, if you are concerned about memory usage from potentially large
# lists of messages, use css selector 'nth-of-type'
# NOTE: accessing first instance of list of the list exists allows None
# if no results are found
first_message = driver.find_elements_by_css_selector('pre._3Gy8WZD53wWAE41lr57by3:first-of-type')
first_message = first_message[0] if first_message else None
last_message = driver.find_elements_by_css_selector('pre._3Gy8WZD53wWAE41lr57by3:last-of-type')
last_message = last_message[0] if last_message else None
I hope this provides an immediate solution but also some fundamentals how to optimize your web scraping moving forward.
Object:
automate following process.
1. Open particular web page, fill the information in search box, submit.
2. from search results click on first result and download the PDF
Work done:
To reach to this object I have written a code as first step. The code works fine but opens up download pop up. Till the time I can't get rid of it, I can not automate the process further. Searched for very many solutions. But none has worked.
For instance, This solution is hard for me to understand and I think its more to do with Java then Python. I changed fire fox profile as suggested by many. This dose matches though not exactly same. I haven't tried as there is no much difference. Even this speaks about changing fire fox profile but that doesn't work.
My code is as below
import selenium.webdriver as webdriver
import selenium.webdriver.support.ui as ui
from time import sleep
import time
import wget
from wget import download
import os
#set firefox Profile
profile = webdriver.FirefoxProfile()
profile.set_preference('browser.download.folderList', 2)
profile.set_preference('browser.download.manager.showWhenStarting', False)
profile.set_preference("browser.download.manager.showAlertOnComplete", False)
profile.set_preference('browser.download.dir', os.getcwd())
profile.set_preference('browser.helperApps.neverAsk.saveToDisk', 'application/pdf')
#set variable driver to open firefox
driver = webdriver.Firefox(profile)
#set variable webpage to open the expected URL
webpage = r"https://documents.un.org/prod/ods.nsf/home.xsp" # edit me
#set variable to enter in search box
searchterm = "A/HRC/41/23" # edit me
#open the webpage with get command
driver.get(webpage)
#find the element "symbol", insert data and click submit.
symbolBox = driver.find_element_by_id("view:_id1:_id2:txtSymbol")
symbolBox.send_keys(searchterm)
submit = driver.find_element_by_id("view:_id1:_id2:btnRefine")
submit.click()
#list of search results open up and 1st occarance is clicked by coppying its id element
downloadPage = driver.find_element_by_id("view:_id1:_id2:cbMain:_id135:rptResults:0:linkURL")
downloadPage.click()
#change windiows. with sleep time
window_before = driver.window_handles[0]
window_after = driver.window_handles[1]
time.sleep(10)
driver.switch_to.window(window_after)
#the actual download of the pdf page
theDownload = driver.find_element_by_id("download")
theDownload.click()
Please guide.
The "Selections" popup is not a different window/tab, it's just an HTML popup. You can tell this because if you right click on the dialog, you will see the normal context menu. You just need to make your "Language" and "File type(s)" selections and click the "Download selected" button.
I've test class with Python and Selenium that has methods to fill forms. After filling one form, the program clicks on "next" to go to the next form. I've noticed that occasionally filling with chrome webdriver; one of the fields is left empty. The program will wait for the time-out of elements in other page before terminating, which can take a long time running the test-suite. So, I used sys.exit(1), when the url does not change to quit the test, so Bamboo server can interpret the exit status. However, with enough tests, sometimes I notice that the url of the UI changes but, the program does not detect it and quits at that point. I was looking for any recommendation to avoid sys.exit(1) and why is program quitting even the url changes in UI.
I needed a way to avoid long test time in failing case, so I used sys.exit(1).
I have tried to use the expected condition of the selenium library selenium.webdriver.support.
def goto_next_page(driver, _next_btn_id):
"""
This method handles going to next page, if the url changes after clicking next, continue otherwise exit with 1.
:param driver : web- driver instance
:param _next_btn_id: element id/ xpath of the next button
:return:
"""
this_url = driver.current_url
print("url before change {}".format(this_url))
# call Webdriver wait in the method.
wait_until_clickable(driver, 30, _next_btn_id)
# Click on the next button to go to the next form
click_button(driver, _next_btn_id)
# Changed url
changed_url = driver.current_url
print(this_url, changed_url)
# If the url does not change means the next page is not accessible, quit the script
if this_url == changed_url:
print("url did not change")
sys.exit(1)
Good Afternoon :) Having a problem with my Python3 Gtk3 application and Selenium WebDriver (ChromeDriver). Also, using Linux if it matters.
Basically, the user presses a button to start the Selenium webdriver automation and then as the automation process is going, it 'SHOULD' give feedback to the user in the GUI (See Content.content_liststore.append(list(item)) and LogBox.log_text_buffer).
However, it's not adding anything into the content_liststore until after fb_driver.close() is done. In the meantime, the Gtk window just "hangs".
Now, I've been looking into multithreading in hopes of the GUI being responsive to this feedback but I've also been reading that Selenium doesn't like multithreading (but I presume thats running multiple browsers/tabs (which this is not)).
So, my question is; Is multithreading the go-to fix for getting this to work?
# ELSE IF, FACEBOOK COOKIES DO NOT EXIST, PROCEED TO LOGIN PAGE
elif os.stat('facebook_cookies').st_size == 0:
while True:
try: # look for element, if not found, refresh the webpage
assert "Facebook" in fb_driver.title
login_elem = fb_driver.find_element_by_id("m_login_email")
login_elem.send_keys(facebook_username)
login_elem = fb_driver.find_element_by_id("m_login_password")
login_elem.send_keys(facebook_password)
login_elem.send_keys(Keys.RETURN)
except ElementNotVisibleException:
fb_driver.refresh()
StatusBar.status_bar.push(StatusBar.context_id, "m_login_password element not found, trying again...")
ProblemsLog.text_buffer.set_text("Facebook has hidden the password field, refreshing page...")
else:
query_elem = fb_driver.find_element_by_name("query")
query_elem.send_keys(target)
query_elem.send_keys(Keys.RETURN)
break
m_facebook_url_remove = "query="
m_facebook_url = fb_driver.current_url.split(m_facebook_url_remove, 1)[1] # Remove string before "query="
facebook_url = "https://www.facebook.com/search/top/?q=" + m_facebook_url # Merge left-over string with the desktop url
StatusBar.status_bar.push(StatusBar.context_id, "Facebook elements found")
fb_title = fb_driver.title
fb_contents = [(target_name.title(), "Facebook", facebook_url)]
for item in fb_contents:
Content.content_liststore.append(list(item))
#progress_bar.set_fraction(0.10)
LogBox.log_text_buffer.set_text("Facebook Search Complete")
with open('facebook_cookies', 'wb') as filehandler:
pickle.dump(fb_driver.get_cookies(), filehandler)
fb_driver.close()
I've considered it not working because of the 'while' loop, but another piece of code doesn't have a loop and does the exact same thing, it waits for Selenium to finish before adding content to the GUI.
Additionally, the user can select multiple websites to do this with, so the application can first go to Facebook (do it's business then close), go to LinkedIn (do it's business then close) and so fourth. And it still waits for all the Selenium code to finish before adding anything to the Gtk GUI.
I really hope that makes sense! Thank you :)
Your`s question is the answer you are lookig for. Take a read here https://wiki.gnome.org/Projects/PyGObject/Threading