I would like to download images with a certain tag from Instagram with their likes. With this post I hope to get some advice or tips on how to do this. I have no experience with web scraping related stuff or web API usages. One of my questions is: can you create a program like this in python code or can you only do this using a webpage?
So far I have understood the following. To get images with a certain tag you have to:
need a valid access_token to even gain access to images by tag, which can be done like this. However, when I sign in you need to give a website. Does this indicate that you can only use the API's on websites rather than a python program for instance?
you use a media Tag Endpoint to search for tags by name.
I have no idea what the latest step will return exactly, but I expect that it will give me a specific image id that contains the tag. Correct? Now I will also need to get the likes belonging to these images. Just like latest step from before:
you use a likes Tag Endpoint to get a list of users that liked the image of which of course you can get the length.
If I can accomplish all of these steps it seems like I can achieve my original goal. I googled if there was something out there already. The only thing I could find was InstaRaider, but this did not seem to fit my description because it web scraped only the images from a specific user and not by tag or its likes. Any suggestions or ideas would be very helpful, I have only programmed in python and Java before..
I can only tell you that for URL you can use the localhost as this:
http://127.0.0.1
OR
http://localhost
I have also tried to do exactly the same before, but I could not, so I used a website to search for tags and images:
http://iconosquare.com/search/[HASHTAG]
Related
So, I have a website where i need to insert a lot of products from our suppliers' website, and I would like to automate it.
I was reading this question, that would solve most of the "front-end" part, and on the backend than I only need to get the link of the page, and use a basic web crawler library to search on that page the given selector.
However I'm missing the part where I inject a possible JS library on the supplier website, in order to allow others to select the element with the mouse, and not to find it using something like the Chrome console.
I was thinking about fetching the source page on the server side, and that returning it, but all the resources like CSS, JS, images, will be blocked by CSRF protection, and so here's my question:
Are there ways to do this in any way?
Edit:
ok maybe it's not so clear what I want to achieve... here's how thing should go:
Someone decides to insert a new supplier, that he should set the CSS path for the product name, the CSS path for the product description etc etc (section where i appreciate suggestion), and then he only needs to insert the products links and the server will get the informations using the CSS path inserted before.
Can anybody give me sample code in python to find the folder Id (especially the last folder created) in google drive? Your help will be immensely appreciated.
Stackoverflow and the Drive API documentation have enough samples of python code for Google Drive API requests, you just need to define the basic steps and patch the corresponding code parts together
Any Google Drive API request need to be based on the Drive API Quickstart for Python which implements OAuth2 authorization flow and creation of an authenticated service.
Once you have this, you can list your files to retrieve their Ids.
In order to narrow down the results, you can define the search parameter q, e.g. specifying mimeType = 'application/vnd.google-apps.folder'.
Use the parameter orderBy to request that the most recent modified folders will be shown first.
Use the paramter pageSize to define how many results you want to obtain (if you want to obtain the newst folder Id only - 1 is a valid value).
Stackoverflow is not meant to help you write a code from scratch, I recommend you to search e.g. with the following specifications for similar questions and try to patch together your code yourself.
Then, if necessary, post a new question with your code explaining where you got stuck and asking for specific help.
Hint: Before implementing your request into Python, test it with the "Try this API" functionality of the Files:
list to make sure that you adapted the parameters correctly to your needs
I have been trying to figure out what this is used for.
What's the difference between using:
http://www.instagram.com/test
and
http://www.instagram.com/test/media
Why does the /media exist? What is its purpose?
The diference between http://www.instagram.com/test and http://www.instagram.com/test/media is the web response
http://www.instagram.com/test
This url give visual information, because you can see the pictures.
http://www.instagram.com/test/media
This url give you information to developer ( json data) like this.
This is information about your first image you can check.
I'm trying to build my app based on Windows Media Services REST API (i'm not using any SDK, just plain rest api requests).
My steps are more or less like:
Create Asset
Assign write Access Policy
Assign upload locator
Upload a file to URL specified by upload locator path
Assign download Access Policy
Assign download locator
All those steps seems to work great but - how can i actually get the video streaming URL? I can't see anything, which looks like such url (as far as i know, it should look similar to upload URL from upload locator). Should i "fold" it by myself using some segments from various parts of api ?
Based on this article, i should append the path parameter with name parameter and /manifest (so it should looke like that: <path_param>/<name_param>.ism/manifest) but it gives me ResourceNotFound error. Anyway, i've seen that other people (like SHIBSANKAR) have found some way to obtain all asset urls so i think there is some way to do it but they have not descrbed how they've done it.
After reading all the docs and talking with microsoft support, i have figured it out. All the URL parts are returned by download locator and the formula looks like that:
#{BaseUri}/video.mp4#{ContentAccessComponent}
I hope someone will find that usefull someday.
I am new to instagram and i am tasked to program an application to grab instagram photo uploads based on a certain hashtag. Meaning if the application is started and searching for the hashtag "#awesomeevent" any one that uploads a photo with that hashtags it will automatically be stored into our database.
The application should work something similar to http://statigr.am/tag/ but instead displaying the photos it should store the photos into the database.
What is the process of doing this. Any tutorials that has this from start to end. Even covering how to start creating a instagram app from scratch. any help would be greatly appreciated.
Thanks
Things we developers often overlook are the API Terms and Conditions. I've been there myself.
API TERMS OF USE
Before you start using the API, we have a few guidelines that we'd like to tell you about. Please make sure to read the full API Terms of Use
Terms of Use. Here's what you'll read about:
Instagram users own their images. It's your responsibility to make sure that you respect that right.
You cannot use the Instagram name in your application.
You cannot use the Instagram API to crawl or store users' images without their express consent.
You cannot replicate the core user experience of Instagram.com
Do not abuse the API. Too many requests too quickly will get your access turned off
However, a part in the terms also states that:
You shall not cache or store any Instagram user photos other than for reasonable periods in order to provide the service you are
providing to Instagram users.
Hope that's a start before you actually get coding and storing images.
API Terms of Use: http://instagram.com/about/legal/terms/api/
API: http://instagram.com/developer/
For starter, you should consult to instagram api.
As for the specific api you will need is:
/tags/tag-name/media/recent
For example, if you want to look for images from tag #awesomeevent, you will do an api query to:
https://api.instagram.com/v1/tags/awesomeevent/media/recent?access_token=ACCESS-TOKEN
I would have a look at the two libraries Instagram provides. The ruby library is at https://github.com/Instagram/instagram-ruby-gem and the python library is at https://github.com/Instagram/python-instagram
They both seem to have examples to get you started if you're programming with either libraries.
As far as the storing issue goes, could you instead store the URL address of the images instead of the actual images themselves? The API returns JSON information of which the URL of the images are returned.
Hope that helps.
You can use the below ruby script to retrieve the images and save them to a file. You can then either reference the file within the database or replace the last block with code for your particular database implementation. Without knowing your database type and schema, no one can tell you how to add something to it.
require "instagram"
require "restclient"
Instagram.configure do |config|
config.client_id = INSTAGRAM_CLIENT_ID
config.client_secret = INSTAGRAM_CLIENT_SECRET
end
instagram_client = Instagram.client(:access_token => INSTAGRAM_ACCESS_TOKEN)
tags = instagram_client.tag_search('cat')
urls = Array.new
for media_item in instagram_client.tag_recent_media(tags[0].name)
urls << media_item.images.standard_resolution.url
end
urls.each_with_index do |url, idx|
image = RestClient.get(url)
path = Dir.pwd + "/#{idx}.jpg"
File.open(path, 'w') {|f| f.write(image) }
end