I'm getting a click event from SendGrid for every link in an email sent to Gmail users, and it is definitely not me clicking the links, I even get the first click event to my SendGrid webhook before SendGrid sends the delivered event to me.
This adds needless hits to both the webhook endpoint and to the pages it hits, I can filter out the click events so I won't get false click counts by filtering for user agents with "Gmail-content-sampling" in it
as the user agent supplied is
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML,
like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246
Gmail-content-sampling
However it would still needlessly hit the site umpteen thousand times in a few minutes (I don't think that will agree with the standard click here to validate email address links) and I don't want to have to add a block user agent per for every request.
Although filtering would be a problem if other providers do the same thing or Gmail change the user agent.
I have tried adding rel="nofollow" to the links but it still does it.
Anyone experience this/know how to get gmail to f-off ?
Edit:
As at 10th August google have undone their stuff up.
Firstly, there are already questions to this topic, but none cover up my problem, entirely, because it's either not the data I need or it's not working properly.
There are services like InstaDP that are able to show you the HQ version of any profile picture from Instagram. Now, I wonder how this is possible?
I did some research and were able to find a higher quality URL when accessing https://www.instagram.com/instagramforbusiness/?__a=1 (see profile_pic_url_hd, answered here). However, InstaDP seems to have a backend that returns a different url that redirects to a way higher quality image: https://instadp-cors-222621.appspot.com/get-hd?id=1107766105 (see at hd_profile_pic_url_info, I extracted the ID for the URL from the result of the ?__a=1 link). I tested this with my personal profile and was able to get the image of myself in an outstanding quality of 1024x1024. However, the ?__a=1 link seem only to return a link for my profile picture in 320x320.
Since InstaDP seem to not be the only player who is able to fetch HQ profile pictures I went ahead and compared the backends of those players. It seems that each service seem to have a different URL to the HQ profile picture of the same Instagram account. So my conclusion is that the Instagram API is involved in all that.
So I created a client key at https://www.instagram.com/developer/. I was also able to get my auth token and determine my logged in csrftoken for the X-CSRFToken header. Now my question is how to continue?
I found a few answers to this topic stating I should request https://i.instagram.com/api/v1/users/1107766105/info/, but it always returns the login page as HTML.
I tried a REST client that uses my Chrome cookies and logged into Instagram before, I tried to set my HTTP headers to X-CSRFToken:<mycookietoken> and Content-Type:application/json. (If I don't set the CSRFToken it errors, so I need to add it, but if the header is set I get the HTML again, even when the CSRFToken is correct. I don't get an error when the CSRFToken is wrong.)
I also tried setting the Origin, Referer and Host to trick Instagram in believing the request came from its own window location, without luck. Setting the Host will even cause a 400 bad request. Even adding my access token in the URL had no effect (?access-token=########).
To sum my question up, how do those services obtain the profile pictures in a such a great quality of up to 1024x1024 from the cdninstagram servers?
Although I am late, but this might be of someone's help:
Step 1:
First thing you need to get HD Instagram profile picture is their profile ID. This can be found in the source code of user profile link. For example if you view source of the following link https://www.instagram.com/abdulhaq0/ and search for "logging_page_id" you will get "profilePage_1285389476". The numbers following the profilePage are the ID for this account.
Step 2:
Next you need put ID in the following URL https://i.instagram.com/api/v1/users/{ProfileID}/info/ and open it in browser. In our case link would be https://i.instagram.com/api/v1/users/1285389476/info/
Step 3:
Now on the link above search for "hd_profile_pic_url_info". There you can get the URL of HD Instagram profile picture.
Hope this helped.
Steps:
1. Get the instagram post link. Eg : https://www.instagram.com/p/Bo-Jru-g7Wa/
or if you don't have the link, the instagram api provides you with a permalink option in the result array which for the above link is Bo-Jru-g7Wa
Now just follow add media?size=l after the url ie.,
Result: High quality image url:
https://www.instagram.com/p/Bo-Jru-g7Wa/media?size=l
you can see it in action here: https://jsfiddle.net/nmj1z7wo/fiddle URL
This link can be considered as a shorthand code to instagram image URL's which are very much bigger
I believe those sites like https://instadp.site/ that show the hi-res of the user profile image do not use the official Instagram API.
In the past it was possible to hack Instagram's CDN URLs to change parameters and get the high resolution from them, but nowadays the URLs are signed and if you change any parameters the URL will fail.
So, the only solution they may be using is to emulate a client. There is a popular PHP client for this: https://github.com/mgp25/Instagram-API
The instagram API have updated. Now we will get only the 320x320 sized image from
https://www.instagram.com/{username}/?__a=1
Even if you get the user id from this endpoint the "user info" endpoint does not return the hd url in its response and should pass a header too now.
import requests
def get_user_by_user_id(user_id):
if user_id:
base_url = "https://i.instagram.com/api/v1/users/{}/info/"
headers = {
'user-agent':'Mozilla/5.0 (iPhone; CPU iPhone OS 12_3_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 Instagram 105.0.0.11.118 (iPhone11,8; iOS 12_3_1; en_US; en-US; scale=2.00; 828x1792; 165586599)'
}
try:
res = requests.get(base_url.format(user_id),headers=headers)
print(res.json())
except Exception as e:
print("getting user failed, due to '{}'".format(e.message))
get_user_by_user_id(userid)
You can check this code and try. Websites like instadp I guess they does not use instagram's official API
The easiest and most reliable way of getting HD (1080x1080) profile pic is Instaloader CLI.
instaloader USERNAME_OF_INTEREST --no-posts --login YOUR_USERNAME
I have a problem which was discussed in an other thread but while the author said he solved it is - for me at least - quit unclear how he did it.
Other thread: Yahoo finance historical stock price power query returns 301 response
I use Power Query an Add In for Excel which allows different query's, one is to grab website content, which I automated for yahoo data. Unfortunately it seems something changed with the yahoo site index and I am not able to use my query anymore.
If I try to recreate the query (build it from scratch again) I get the error "HTTP 301". Even with the "normal" query feature of excel the yahoo data is not available anymore.
Hopefully someone is able to help me.
Best wishes,
Andreas
You should set header "user-agent" to emulate browser
For examble Google Chrome
let
url = "https://finance.yahoo.com/quote/AAL/history?p=AAL",
#"user-agent"="Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.86 Safari/537.36",
web = Web.Contents(url, [Headers=[#"user-agent"=#"user-agent"]]),
html = Web.Page(web),
Data0 = html{0}[Data]
in
Data0
I am new for Chromecast Please help me out ,
I have to detect chromecast devices as same like google chromecast extension and send url for casting to selected device ...
I want to develop a html5 application where user can select chromcast devices and send url to it
Kinda new to J2ME. I was wondering how I could go about in getting a J2ME midlet to send the original user agent of the mobile device to a server instead of: "Profile/MIDP-1.0 Configuration/CLDC-1.0"?
The problem here is that I wont be able to kno the specific device the midlet runs on in order to manually set the useragent. This is because I'm developing a J2ME SDK that will be widely distributed for advertising purposes within other applications. It's like a plugin that fetches and display ads within an app.
For example, if the midlet runs on a sonyericsson, how can i get the midlet to send the original phone useragent:
"SonyEricssonW300i/R4EA Browser/NetFront/3.3 Profile/MIDP-2.0 Configuration/CLDC-1.1"
to my server without actually using
"setRequestProperty(" User-Agent", "SonyEricssonW300i/R4EA Browser/NetFront/3.3 Profile/MIDP-2.0 Configuration/CLDC-1.1")" ?
I want to detect the true device using WURFL on the server-side. If the midlet sends "Profile/MIDP-1.0 Configuration/CLDC-1.0" as the useragent, WURFL will detect the phone as "Generic J2ME midlet" instead of SonyEricsson W300.
I'd really appreciate it if someone assist.
The fool-proof way is to detect the user-agent when the web browser requests the jad, and set the value in the returned jad file.