Like instagram photo with selenium python - python-3.x

I'm trying to code my own bot python selenium bot for instagram to experiment a little. I succeeded to log in and to search an hashtag in the search bar which lead me to the following web page:
But I can not figure out how to like the photos in the feed, i tried to use a search with xpath and the following path:
"//[#id="reactroot"]/section/main/article/div1/div/div/div1/div1/a/div"
But it didn't work, does anybody have an idea?

First of all, in your case it is recommended to use the official Instagram Api for Python (documentation here on github).
It would make your bot much simpler, more readable and most of all lighter and faster. So this is my first advice.
If you really need using Selenium I also recommend downloading Selenium IDE add-on for Chrome here since it can save you much time, believe me. You can find a nice tutorial on Youtube.
Now let's talk about possible solutions and them implementations. After some research I found that the xpath of the heart icon below left the post behaves like that:
The xpath of the icon of the first post is:
xpath=//button/span
The xpath of the icon of the second post is:
xpath=//article[2]/div[2]/section/span/button/span
The xpath of the icon of the third post is:
xpath=//article[3]/div[2]/section/span/button/span
And so on. The first number near "article" corresponds to the number of the post.
So you can manage to get the number of the post you desire and then click it:
def get_heart_icon_xpath(post_num):
"""
Return heart icon xpath corresponding to n-post.
"""
if post_num == 1:
return 'xpath=//button/span'
else:
return f'xpath=//article[{post_num}]/div[2]/section/span/button/span'
try:
# Get xpath of heart icon of the 19th post.
my_xpath = get_heart_icon_xpath(19)
heart_icon = driver.find_element_by_xpath(my_xpath)
heart_icon.click()
print("Task executed successfully")
except Exception:
print("An error occurred")
Hope it helps. Let me know if find other issues.

i'm trying to do the same)
Here is a working method. Firstly, we find a class of posts(v1Nh3), then we catch link attribute(href).
posts = bot.find_elements_by_class_name('v1Nh3')
links = [elem.find_element_by_css_selector('a').get_attribute('href') for elem in posts]

I implemented a function that likes all the picture from an Instagram page. It could be used on the "Explore" page or simply on an user's profil page.
Here's how I did it.
First, to navigate to a profil page from Instagram's main page, I create a xPath for the "SearchBox" and a xPath to the element in the dropdown menu results corresponding to the index.
def search(self, keyword, index):
""" Method that searches for a username and navigates to nth profile in the results where n corresponds to the index"""
search_input = "//input[#placeholder=\"Search\"]"
navigate_to = "(//div[#class=\"fuqBx\"]//descendant::a)[" + str(index) + "]"
try:
self.driver.find_element_by_xpath(search_input).send_keys(keyword)
self.driver.find_element_by_xpath(navigate_to).click()
print("Successfully searched for: " + keyword)
except NoSuchElementException:
print("Search failed")
Then I open the first picture:
def open_first_picture(self):
""" Method that opens the first picture on an Instagram profile page """
try:
self.driver.find_element_by_xpath("(//div[#class=\"eLAPa\"]//parent::a)[1]").click()
except NoSuchElementException:
print("Profile has no picture")
Like each one of them:
def like_all_pictures(self):
""" Method that likes every picture on an Instagram page."""
# Open the first picture
self.open_first_picture()
# Create has_picture variable to keep track
has_picture = True
while has_picture:
self.like()
# Updating value of has_picture
has_picture = self.has_next_picture()
# Closing the picture pop up after having liked the last picture
try:
self.driver.find_element_by_xpath("//button[#class=\"ckWGn\"]").click()
print("Liked all pictures of " + self.driver.current_url)
except:
# If driver fails to find the close button, it will navigate back to the main page
print("Couldn't close the picture, navigating back to Instagram's main page.")
self.driver.get("https://www.instagram.com/")
def like(self):
"""Method that finds all the like buttons and clicks on each one of them, if they are not already clicked (liked)."""
unliked = self.driver.find_elements_by_xpath("//span[#class=\"glyphsSpriteHeart__outline__24__grey_9 u-__7\" and #aria-label=\"Like\"]//parent::button")
liked = self.driver.find_elements_by_xpath("//span[#class=\"glyphsSpriteHeart__filled__24__red_5 u-__7\" and #aria-label=\"Unlike\"]")
# If there are like buttons
if liked:
print("Picture has already been liked")
elif unliked:
try:
for button in unliked:
button.click()
except StaleElementReferenceException: # We face this stale element reference exception when the element we are interacting is destroyed and then recreated again. When this happens the reference of the element in the DOM becomes stale. Hence we are not able to get the reference to the element.
print("Failed to like picture: Element is no longer attached to the DOM")
This method checks if picture has a "Next" button to the next picture:
def has_next_picture(self):
""" Helper method that finds if pictures has button \"Next\" to navigate to the next picture. If it does, it will navigate to the next picture."""
next_button = "//a[text()=\"Next\"]"
try:
self.driver.find_element_by_xpath(next_button).click()
return True
except NoSuchElementException:
print("User has no more pictures")
return False
If you'd like to learn more feel free to have a look at my Github repo: https://github.com/mlej8/InstagramBot

Related

Selenium Web driver cannot find css selector even if its present (python)

I am trying to scrape data from seetickets.us. I am clicking on each org and then all events by that org. The scraper correctly scrape data from each event but the problem is that when I come back to all events page web driver cannot find the css selector.
Here is the site structure:
https://ibb.co/WBjMDJf
clicking on World Cafe Live get me here:
https://ibb.co/cLbMP19
clicking on any event will move me toward further info about event.
Now when the driver is coming back from extracting each event , It is not able to go into other event. I have also tried explicit wait anf time.sleep()
Here is my code:
#this is the func click on each event and extract data then come back to all event page
def get_all_events_in_each_event(self):
inner_events = self.get_all_inner_events()
print(len(inner_events))
for event in inner_events:
self.click_inner_event(event)
self.get_event_loc()
self.get_talent()
self.get_facebook()
self.get_date()
self.get_showtime_city()
self.get_ticket_doors()
self.back()
try:
WebDriverWait(self, 10).until(
EC.element_to_be_clickable((By.CLASS_NAME, "event-images-box")))
except Exception as e:
print("Wait Timed out")
print(e)
#this is the func to click on each event in all event pages
def click_inner_event(self , inner_event):
link = inner_event.find_element_by_css_selector('div[class="event-info"]')
link.click()
Here is HTML of all events page:
https://ibb.co/wcKWc68
Kindly help me with finding what's wrong here.
Thanks
As #Arundeep Chohan , correctly pointed that web driver loses reference when moving back and forth so I had to re grab all the elements.
Correct code is:
def get_all_events_in_each_event(self):
inner_events = self.get_all_inner_events()
for i in range(len(inner_events)):
self.click_inner_event(inner_events[i])
self.get_event_loc()
self.get_talent()
self.get_facebook()
self.get_date()
self.get_showtime_city()
self.get_ticket_doors()
self.back()
inner_events = self.get_all_inner_events() #regrabbing the elements
Thanks arundeep for the answer.

Python + Mechanize - Emulate Javascript button click using POST?

I'm trying to automate filling a car insurance quote form on a site:
(following the same format as the site URL lets call it: "https://secure.examplesite.com/css/car/step1#noBack")
I'm stuck on the rego section as once the rego has been added, a button needs to be clicked to perform the search and it seems this is heavy Javascript and I know mechanizer can't handle this. I'm not versed in JavaScript, but I can see that when clicking the button a POST request is made to this URL: ("https://secure.examplesite.com/css/car/step1/searchVehicleByRegNo") Please see image also.
How can I emulate this POST request in Mechanize to run the javascript? So I can see the response / interact with the response? Or is this not possible? Can I consider bs4/requests/robobrowser instead. I'm only ~4 months into learning! Thanks
# Mechanize test
import mechanize
br = mechanize.Browser()
br.set_handle_robots(False) # ignore robots
br.set_handle_refresh(False) # can sometimes hang without this
res = br.open("https://secure.examplesite.com/css/car/step1#noBack")
br.select_form(id = "quoteCollectForm")
br.set_all_readonly(False) # allow everything to be written to
controlDict = {}
# List all form controls
for control in br.form.controls:
controlDict[control.name] = control.value
print("type = %s, name = %s, value = %s" %(control.type, control.name, control.value))
# Enter Rego etc "example"
br.form["vehicle.searchRegNo"] = "example"
# Now for control name = vehicle.searchRegNo, value = example
# BUT Now how do I click the button?? Simulate POST? The post url is formatted like:
# https://secure.examplesite.com/css/car/step1/searchVehicleByRegNo
Javascript POST
Solved my own problem-
Steps:
open dev tools in browser
Go to network tab and clear
interact with form element (in my case car rego finder)
click on the event that occurs from interaction
copy the exact URL, Request header data, and payload
I used Postman to quickly test the request and responses were correct / the same as the Webform and found the relevant headers
in postman convert to python requests code
Now I can interact completely with the form

Facebook Python SDK - GET events

I am a newbie working with facebook python sdk. I found this file that enabled me to obtain event data from my facebook business page.
However, I am unable to fetch all of the fields I need. The missing field I am unable to obtain is the event videos. I am unable to retrieve the video source or video_url for the video that was posted with the event.
I have tried many different variations of fields =
to no avail. Thank You
import facebook
import requests
def some_action(post):
""" Here you might want to do something with each post. E.g. grab the
post's message (post['message']) or the post's picture (post['picture']).
In this implementation we just print the post's created time.
"""
print(post["id"])
# You'll need an access token here to do anything. You can get a temporary one
# here: https://developers.facebook.com/tools/explorer/
access_token = "My_Access_Token"
# Look at Bill Gates's profile for this example by using his Facebook id.
user = "397894520722082"
graph = facebook.GraphAPI(access_token)
profile = graph.get_object(user)
posts = graph.get_connections(profile['id'], 'posts')
print(graph.get_object(id=397894520722082,
fields='events{videos,id,category,name,description,cover,place,start_time,end_time,interested_count,attending_count,declined_count,event_times,comments}'))
# Wrap this block in a while loop so we can keep paginating requests until
# finished.
while True:
try:
# Perform some action on each post in the collection we receive from
# Facebook.
[some_action(post=post) for post in posts["data"]]
# Attempt to make a request to the next page of data, if it exists.
posts = requests.get(posts["paging"]["next"]).json()
except KeyError:
# When there are no more pages (['paging']['next']), break from the
# loop and end the script.
break

What is making the webbrowser close before it finishes?

I have the code bellow which I know has worked before but for some reason seems to be broken now. The code is mean't to open a search engine, search for a query and return a list of results by the href tag. The webbrowser will open and navigate to http://www.startpage.com success fully, it then puts the term I have entered at the bottom into the search box but then just closes the browser. No error, no links. Nothing.
import selenium.webdriver as webdriver
def get_results(search_term):
url = "https://www.startpage.com"
browser = webdriver.Firefox()
browser.get(url)
search_box = browser.find_element_by_id("query")
search_box.send_keys(search_term)
search_box.submit()
try:
links = browser.find_elements_by_xpath("//ol[#class='web_regular_results']//h3//a")
except:
links = browser.find_elements_by_xpath("//h3//a")
results = []
for link in links:
href = link.get_attribute("href")
print(href)
results.append(href)
browser.close()
return results
get_results("dog")
Does anyone know what is wrong with this? Basically it gets to search_box.submit() then skips everything until browser.close().
Unlike find_element_by_xpath (single WebElement) If find_elements_by_xpath won't find any results it won't throw an exception, it will return an empty list. links is empty so the for loop is never executed. You can change the try except to if condition, and check if it has values
links = browser.find_elements_by_xpath("//ol[#class='web_regular_results']//h3//a")
if not links:
links = browser.find_elements_by_xpath("//h3//a")
It is not recommended to use browser close function within the function that you are testing. Instead you can use after the get_results("dog") function and keep the test logic away.
get_results("dog")
browser.close()
By doing this way selenium will complete the execution of the function first and then close the browser window.
The problem with your solution is that the method is returning the result set after the browser is closing the window due to which you are facing logical problem with your script.

trouble getting the current url on selenium

I want to get the current url when I am running Selenium.
I looked at this stackoverflow page: How do I get current URL in Selenium Webdriver 2 Python?
and tried the things posted but it's not working. I am attaching my code below:
from selenium import webdriver
#launch firefox
driver = webdriver.Firefox()
url1='https://poshmark.com/search?'
# search in a window a window
driver.get(url1)
xpath='//input[#id="user-search-box"]'
searchBox=driver.find_element_by_xpath(xpath)
brand="freepeople"
style="top"
searchBox.send_keys(' '.join([brand,"sequin",style]))
from selenium.webdriver.common.keys import Keys
#EQUIValent of hitting enter key
searchBox.send_keys(Keys.ENTER)
print(driver.current_url)
my code prints https://poshmark.com/search? but it should print: https://poshmark.com/search?query=freepeople+sequin+top&type=listings&department=Women because that is what selenium goes to.
The issue is that there is no lag between your searchBox.send_keys(Keys.ENTER) and print(driver.current_url).
There should be some time lag, so that the statement can pick the url change. If your code fires before url has actually changed, it gives you old url only.
The workaround would be to add time.sleep(1) to wait for 1 second. A hard coded sleep is not a good option though. You should do one of the following
Keep polling url and wait for the change to happen or the url
Wait for a object that you know would appear when the new page comes
Instead of using Keys.Enter simulate the operation using a .click() on search button if it is available
Usually when you use click method in selenium it takes cared of the page changes, so you don't see such issues. Here you press a key using selenium, which doesn't do any kind of waiting for page load. That is why you see the issue in the first place
I had the same issue and I came up with solution that uses default explicit wait (see how explicit wait works in documentation).
Here is my solution
class UrlHasChanged:
def __init__(self, old_url):
self.old_url = old_url
def __call__(self, driver):
return driver.current_url != self.old_url:
#contextmanager
def url_change(driver):
current_url = driver.current_url
yield
WebDriverWait(driver, 10).until(UrlHasChanged(current_url))
Explanation:
At first, I created my own wait condition (see here) that takes old_url as a parameter (url from before action was made) and checks whether old url is the same like current_url after some action. It returns false when both urls are the same and true otherwise.
Then, I created context manager to wrap action that I wanted to make, and I saved url before action was made, and after that I used WebDriverWait with created before wait condition.
Thanks to that solution I can now reuse this function with any action that changes url to wait for the change like that:
with url_change(driver):
login_panel.login_user(normal_user['username'], new_password)
assert driver.current_url == dashboard.url
It is safe because WebDriverWait(driver, 10).until(UrlHasChanged(current_url)) waits until current url will change and after 10 seconds it will stop waiting by throwing an exception.
What do you think about this?
I fixed this problem by clicking on the button by using href. Then do driver.get(hreflink). Click() was not working for me!

Resources