(Duplicate of this question since I don't have enough rep to add a comment).
Essentially when using Search and using the page token to get more results, the results in the following pages tend to have results from the previous pages. The more pages you go through, the more and more repeating videos appear.
You can test this directly via the documentation which allows you to perform calls from there. Do a search query for anything, keep track of the video IDs in the results, wait a few seconds, and then do another query with the next page token, and repeat. It sometimes takes around 5 or so pages before a duplicate shows up. The same issue happens if you search for related videos instead of a query.
Is this intended behavior? I cannot seem to locate anything in the documentation mentioning this. I may be wrong, but I feel like this issue only started happening this month because I did not notice this behavior in an application I was working on around a month ago.
The Youtube API returns the response in a paginated manner. This means that if you use the search functionality, your search results will be available on different pages where each page has a different page token. The maxResults parameter determines the number of results on each page(default=50). To tackle this problem and return new/different responses with each call , pass the nextPageToken to your next API call.
For example, if your first API call looks like this :
GET https://youtube.googleapis.com/youtube/v3/search?part=snippet&maxResults=10&q=cricket&key=[YOUR_API_KEY]
Your API response would look like :
{
"kind": "youtube#searchListResponse",
"etag": "uN1c33JfiFaPBemlxN5kH8lSaHw",
"nextPageToken": "CAoQAA",
"regionCode": "IN",
"pageInfo": {
"totalResults": 1000000,
"resultsPerPage": 10
},
To get the next 10 results of those 1000000, add pagetoken = nextPageToken to your query ,something like this :
GET https://youtube.googleapis.com/youtube/v3/search?part=snippet&maxResults=10&pageToken=CAoQAA&q=cricket&key=[YOUR_API_KEY]
AND VOILA!
{
"kind": "youtube#searchListResponse",
"etag": "NeaA5DLyr3YIaKdX5ZxETA3GfhY",
"nextPageToken": "CBQQAA",
"prevPageToken": "CAoQAQ",
"regionCode": "IN",
"pageInfo": {
"totalResults": 1000000,
"resultsPerPage": 10
}
YOU GET THE NEXT 10 RESULTS!
THE FIRST PAGE WOULD NOT HAVE ANY PAGETOKEN, SO THE FIRST API CALL NEEDS TO BE MADE WITHOUT PAGETOKEN PARAMETER
Refer to the official documentation for more details:
https://developers.google.com/youtube/v3/docs/search/list
Related
I have developed an application that is powered by Google Places API. The problem is the places are loading when running locally but not after deploying it on google cloud. I am using a default keyword to fetch the desired results but surprisingly it is not working after its deployed. I tried changing the keyword but still, it returns 0 results. Please have a look at my code below
await axios
.get("https://maps.googleapis.com/maps/api/place/nearbysearch/json", {
params: {
key: process.env.GOOGLE_PLACE_API_KEY,
location: req.body.ll,
radius: 20000,
keyword: "popular destinations near me",
},
})
and the response I get
{ html_attributions: [], results: [], status: 'ZERO_RESULTS' }
Postman request of the same works without any issue
and the same request sent with a raw JSON data, I am getting an error
{
"key": "my key",
"location": "my location",
"radius": "20000",
"keyword": "popular destinations near me"
}
{
"error_message": "You must use an API key to authenticate each request to Google Maps Platform APIs. For additional information, please refer to http://g.co/dev/maps-no-account",
"html_attributions": [],
"results": [],
"status": "REQUEST_DENIED"
}
The same request sent using postman returns 20+ results. I have no clue what could possibly be wrong with the above request. Any help is appreciated, thanks.
The Place API Place Search Nearby Search requests required parameters which are key, location, and radius. There are optional parameters you can input too, they are
keyword(not keywords),
language,
minprice&maxprice,
name,
The name field is no longer restricted to place names. Values in this field are combined with values in the keyword field and passed as part of the same search string. We recommend using only the keyword parameter for all search terms.
opennow,rankby,type,pagetoken.
So you cannot assign more than one keyword to your request.
I'm using Azure Search for perform some customs search in a database.
I got this one field that have this kind of structure:
"STUFF": "05-05-16-00|"
but I'm having trouble by creating the filter, because its possible that I'll not have all the numbers that builds this structure. It all depends that what the final user will type. So I need a wildcard to fill the blanks with the missing numbers, like this
"05-05-??-??" -> the pipe is important, because this field can have more than 1 code inside.
Now I need to catch all the possible elements that STARTS WITH 05-05, like, for example: 05-05-11-01
I thought I suposed to use the search.ismatch() function, but it doesnt work.
here some code:
search.ismatch('05-05-??-??','STUFF');
And the results were:
"STUFF": "02-02-16-00|",
"STUFF": "02-02-14-00|",
this is driving me crazy, because I dont know why this results came back.
Maybe is important to know that Im performing a POST request to the Azure Search API with this code in 'filter'
Maybe i should to escape this especial characters like - and ? like this
search.ismatch('05\\-05\\-\\?\\?\\-\\?\\?','STUFF')
But the results were the same.
Can somebody please help me ?
EDIT 1
following this Article I change some things and make the following search:
search.ismatch('\"05-00*\"','STUFF','simple', 'all')
And I starting the get some results, but now this is my results:
"STUFF": "06-05-02-00|", //WRONG
"STUFF": "05-02-05-01|", //RIGHT
"STUFF": "05-02-02-07|", //RIGHT
For some reason, it's returing the right structure but not the in the front of the text.
EDIT 2
I made some changes and change all the "-" for the keyword "OU" and I'm trying to follow this question to make sore like a "contains", but i perfoming a POST request with the following parameters
{
"search": "*",
"filter": "search.ismatch('/.*08010000OU/.*','STUFF', 'full', 'all')",
"skip": "0",
"count": true
}
Im trying to use a wildcard in the begining of the query search because I still missing some information.
I believe you won't be able to solve this using the StandardAnalyzer. Try switching to WhitespaceAnalyzer for this particular field and it probably will work with "05-05*"
According to this document Total hits and Page Counts, if I want to implements paging, I should combine skip and top in my query.
The following is my POST query
{
"count": true ,
"search": "apple",
"searchFields": "content",
"select": "itemID, content",
"searchMode": "all",
"queryType":"full",
"skip": 100,
"top":100
}
It should return something like from 100 --> 200
{
"#odata.context": "https://example.search.windows.net/indexes('example-index-')/$metadata#docs(*)",
"#odata.count": 1611,
"#search.nextPageParameters": {
"count": true,
"search": "apple",
"searchFields": "content",
"select": "itemID, content",
"searchMode": "all",
"queryType": "full",
"skip": 200
},
But it just return top 100 , not paging offset to 100
"#odata.context": "https://example.search.windows.net/indexes('example-index')/$metadata#docs(*)",
"#odata.count": 1611,
Is there any thing I should setup?
The short answer of how to implement paging (from this article): Set top to your desired page size, then increment skip by that page size on every request. Here is an example of how that would look with the REST API using GET:
GET /indexes/onlineCatalog/docs?search=*$top=15&$skip=0
GET /indexes/onlineCatalog/docs?search=*$top=15&$skip=15
GET /indexes/onlineCatalog/docs?search=*$top=15&$skip=30
The REST API documentation also covers how to implement paging, as well as the true purpose of #odata.nextLink and #search.nextPageParameters (also covered in this related question).
Quoting from the API reference:
Continuation of Partial Search Responses
Sometimes Azure Search can't return all the requested results in a single Search response. This can happen for different reasons, such as when the query requests too many documents by not specifying $top or specifying a value for $top that is too large. In such cases, Azure Search will include the #odata.nextLink annotation in the response body, and also #search.nextPageParameters if it was a POST request. You can use the values of these annotations to formulate another Search request to get the next part of the search response. This is called a continuation of the original Search request, and the annotations are generally called continuation tokens. See the example in Response below for details on the syntax of these annotations and where they appear in the response body.
The reasons why Azure Search might return continuation tokens are implementation-specific and subject to change. Robust clients should always be ready to handle cases where fewer documents than expected are returned and a continuation token is included to continue retrieving documents. Also note that you must use the same HTTP method as the original request in order to continue. For example, if you sent a GET request, any continuation requests you send must also use GET (and likewise for POST).
Note
The purpose of #odata.nextLink and #search.nextPageParameters is to protect the service from queries that request too many results, not to provide a general mechanism for paging. If you want to page through results, use $top and $skip together. For example, if you want pages of size 10, your first request should have $top=10 and $skip=0, the second request should have $top=10 and $skip=10, the third request should have $top=10 and $skip=20, and so on.
The follwing example is Get method without top.
GET Method
https://example.search.windows.net/indexes('example-content-index-zh')/docs?api-version=2017-11-11&search=2018&$count=true&$skip=150&select=itemID
The Result:
{
"#odata.context": "https://example.search.windows.net/indexes('example-index-zh')/$metadata#docs(*)",
"#odata.count": 133063,
"value": [ //skip value ],
"#odata.nextLink": "https://example.search.windows.net/indexes('example-content-index-zh')/docs?api-version=2017-11-11&search=2018&$count=true&$skip=150"
}
No matter I use post or get, onces I add $top=# the #odata.nextLink will not show in result.
Finally, I figure out although #odata.nextLink or #search.nextPageParameters will not show. But the paging is working, I have to count myself.
I need to get number of Followers gained and lost or just the total followers during a selected time range.
For example:
if I send a request to: https://api.instagram.com/v1/users/3/
I will get this Json:
{
"data":
{
"username": "kevin",
"bio": "CEO & Co-founder of Instagram",
"website": "",
"profile_picture": "https://instagramimages-a.akamaihd.net/profiles/profile_3_75sq_1325536697.jpg",
"full_name": "Kevin Systrom",
"counts": {
"media": 1419,
"followed_by": 1138347,
"follows": 643
},
"id": "3"
}
But I can't get the "followed_by" for a select time range
I read the Instagram api documentation and I can't find anywhere a Endpoint to do a request that I want.
in MEDIA I can pass MIN_TIMESTAMP and MAX_TIMESTAMP as parameters, but I am not looking for medias, I am looking for the number of followers.
I know It's possible because there is a website https://minter.io that gets every information since the beginning of the account.
PS: I already have the Authentication with OAuth 2.0
There is no API to do this, I think you have to keep a track of your follower count and update it every day. Setup a cron job to do this via API.
I just tried minter.io, I dont know how they show followers from day 1, I can definitely tell u it is fake and not accurate, I had about 1400 followers at some point and I removed them few months ago, so its back to 200 something, and minter.io does not show this at all. I think they just show a fake linear graph for historical data and going forward they keep a track of followers every day.
It is possible to use the field 'followers_count' on the User node to get a starting point. This gives the total number of followers at the time of the request. From there you could calculate a running total by subtracting the daily follower count from the Insights api that you mention.
Get current follower count:
curl -i -X GET "https://graph.facebook.com/v8/<user id>?fields=followers_count&access_token=EAACwX..."
https://developers.facebook.com/docs/instagram-api/reference/user
Get new followers for single day:
curl -i -X GET "https://graph.facebook.com/v8/<user id>/insights?metric=follower_count&period=day&since=2020-10-5T07:00:01&until=2020-10-06T07:00:00
https://developers.facebook.com/docs/instagram-api/reference/user/insights
I'm trying to use Freebase to search for multiple items at a time (using one API call). For example, if I have two items:
Robert Downey, Jr.
The Avengers
I want to query Freebase once and get back results for both items. Basically all I need is the mid for the top 3 or 4 results for both items. I would like to rely on Freebase's search API to provide disambiguation for topics. For example, I'd like to be able to search for "Robert Downey, Jr." with the abbreviation: "RDJ".
This is easy to do when searching one item at a time:
https://www.googleapis.com/freebase/v1/search?query=rdj
Making two calls like this would give me exactly what I'm looking for, but I would like to stay away from making these calls individually.
Reconciliation
I did run across the json-rpc call for reconciliation, and I have tried the following:
Endpoint: https://www.googleapis.com/rpc
POST body:
[
{
"method": "freebase.reconcile",
"apiVersion": "v1",
"params": {
"name": ["RDJ"],
"key": "api_key",
"limit":10
}
},
{
"method": "freebase.reconcile",
"apiVersion": "v1",
"params": {
"name": ["the avengers"],
"key": "api_key",
"limit":10
}
}
]
This works fairly well for Robert Downey, Jr in that I get a result of type /film/actor as I did using the search api. However, for The Avengers, I get a set of results with type /book/book rather than the 2012 film. These results don't seem to be prioritized the same way as the search results.
I tried something similar using json-rpc for a Freebase search method:
{
"method": "freebase.search",
"apiVersion": "v1",
"params": {
"name": ["RDJ"],
"key": "api_key",
"limit":10
}
}
But the "freebase.search" method didn't seem to exist.
One thing to note is that I will not know the expected type of the items I am looking for before hand.
Long story short: I want the exact results the search API provides, but with multiple queries wrapped up into one call.
Am I missing something terribly simple like an OR operator for the search API?? I've been searching for days, but can't seem to find a good solution. I would appreciate any help at all!
Why not just make two calls asynchronously? That would give you the results you need with almost no penalty in latency.
A few relevant facts:
The Reconcile API is still experimental. It's intended for use in reconciling against a type at a minimum and usually scoring using additional property values.
The Search API isn't included in the RPC mechanism because its freeform output doesn't work with the assumptions of the RPC framework. Ditto for the Topic API, although that's not really relevant here.
The Search API has a fairly expressive S-expression language. You don't say if you want the queries scored independently or together, but if you want them ranked jointly, you can use a filter expression like [(any name:rdj name:"The Avengers")]
https://www.googleapis.com/freebase/v1/search?query=&limit=10&filter=%28any%20name:rdj%20name:%22the%20avengers%22%29