I have lots of sub-folders and here is my code for searching
"trashed = false
AND ( ".$folderIds." )
AND mimeType != 'application/vnd.google-apps.folder'"
where the $folderIds are the subfolders, ex:
0B_123dSaMpleFolderId1 in parents or 0B_123dSaMpleFolderId2 in parents and 300 more folders
I know that limits are already posted here: What is the limit on Google Drive API usage?
but I would like to know the limits for this.. of how long should the parameter be when sending it to google drive api
The Google Drive API v3 allowed me to search within 598 folders:
len(gdrive.search(
f"mimeType!='application/vnd.google-apps.folder' and
({build_folder_subquery(folders[0:598])})"
))
713
but not 599:
len(gdrive.search(
f"mimeType!='application/vnd.google-apps.folder' and
({build_folder_subquery(folders[0:599])})"
))
*** googleapiclient.errors.HttpError: <HttpError 400 when requesting https://www.googleapis.com/drive/v3/files returned "The query is too complex.">
I don't know whether the 'query is too complex' error is triggered by the number of operators in the query, by the number of characters in the query, or by some other measure. The number of characters in the queries above are 29949 and 29999.
Related
I am attempting to do a retrieve of all media items that a given Google Photos user has, irrespective of any album(s) that they are in. However when I attempt to use either the mediaItems.list or the mediaItems.search methods, the pageSize param I am including in either request is either being ignored or not fully fullfilled.
Details of mediaItems.list request
GET https://photoslibrary.googleapis.com/v1/mediaItems?pageSize=<###>
Details of mediaItems.search request
POST https://photoslibrary.googleapis.com/v1/mediaItems:search
BODY { 'pageSize': <###> }
I have made a simple implementation of these two requests here as an example for this question, it just requires a valid accessToken to use:
https://jsfiddle.net/zb2htog1/
Running this script with the following pageSize against a Google Photos account with 100s of photos and 10s of albums consistently returns the same unexpected amount of result for both methods:
Request pageSize
Returned media items count
1
1
25
9
50
17
100
34
I know that Google states the following for the pageSize parameter for both of these methods:
“Maximum number of media items to return in the response. Fewer media
items might be returned than the specified number. The default
pageSize is 25, the maximum is 100.”
I originally assumed that the reason fewer media items might be returned is because an account might have less media items in total than a requested pageSize, or that a request with a pageToken has reached the end of a set of paged results. However I am now wondering if this just means that results may vary in general?
Can anyone else confirm if they have the same experience when using these methods without an album ID for an account with a suitable amount of photos to test this? Or am I perhaps constructing my requests in an incorrect fashion?
I experience something similar. I get back half of what I expect.
If I don't set the pageSize, I get back just 13, If I set to 100, I get back 50.
I am calling this endpoint: /users/{email-address}/mailFolders/Inbox/messages?$select=isRead,hasAttachments,subject,from,bodyPreview,receivedDateTime&$top=25&search="{search-term}"&$count=true
the response only returns #odata.context, #odata.nextLink, and value.
When removing the $search parameter from the query, the response returns #odata.count as well.
I am using the #odata.count to calculate how many pages of results there are. I would prefer to not use the skipToken and only use $top and $skip to paginate the results.
Anyone else encounter this before/have a workaround?
Move comment to answer for others' reference.
Using $search and $count together won't return the `#odata.count currently.
As $search requests are limited to 250 results, you can only get up to 250 items in the response.
So in this case I assume that your data is less than 250. The workaround is, use $top=250&$search="{search-term}" in the request to get all the results, and then get the count of the results with the method provided by the package (other than Microsoft SDK) in your code.
Hi I am getting keyerror: 'groups' when trying to fetch nearby venues using Foursquare API. Following is my code:
LIMIT = 100 # limit of number of venues returned by Foursquare API
radius = 500 # define radius
venues_list = []
for lat, long, post, borough, neighborhood, hospital in zip(hospital_df['Latitude'], hospital_df['Longitude'], hospital_df['Pincode'], hospital_df['District'], hospital_df['Location'], hospital_df['Hospital_Name']):
print(neighborhood)
print(borough)
url = "https://api.foursquare.com/v2/venues/explore?client_id={}&client_secret={}&v=
{}&ll={},{}&radius={}&limit={}".format(
CLIENT_ID,
CLIENT_SECRET,
VERSION,
lat,
long,
radius,
LIMIT)
results = requests.get(url).json()["response"]['groups'][0]['items']
venues_list.append([(
post,
borough,
neighborhood,
hospital,
lat,
lng,
v['venue']['name'],
v['venue']['location']['lat'],
v['venue']['location']['lng'],
v['venue']['categories'][0]['name']) for v in results])
nearby_venues = pd.DataFrame([item for venue_list in venues_list for item in venue_list])
nearby_venues.columns = ['PostalCode', 'Borough', 'Neighborhood', 'Hospital', 'Neighborhood_Latitude', 'Neighborhood_Longitude', 'VenueName', 'VenueLatitude', 'VenueLongitude', 'VenueCategory']
I keep getting the following error:
KeyError: 'groups'
I had the same issue "KeyError: 'groups'" with very similar code. What I found was that while my URL variable that I built was incorrectly formed (I added a filter for categoryId, but I passed an incorrectly formatted variable for this value).
Once I corrected this formula, the "results = requests.get(url).json()["response"]['groups'][0['items']" was able to process without errors.
My guess is that when I submitted the incorrectly formed URL, the JSON return was an error message without the correct formatting that included 'groups'.
This was due to exceeding the number of free calls to the Foursquare API in my case. I have very similar code, prewritten as part of an online course. I was running it fine for many days with no issues. Then, suddenly I got the 'groups' key error a few times. Then, I stopped working, and the next morning the code ran fine. Then, after a few calls I got the error again. So I checked the .json file, and it didn't contain the key 'groups' because it was basically a .json file telling me the quota was exceeded.
Try resetting your CLIENT SECRET from Foursquare account. It worked for me.
i want to search-tweet related 'data' and count more than 100
this is python grammer
from twython import Twython
twitter= Twython(app_key=APP_KEY,app_secret=APP_SECRET)
for status in twitter.search(q='"data"',count =10000)["statuses"]:
user =status["user"]["screen_name"].encode('utf-8')
text =status["text"]
data = "{0} {1} {2}".format(user ,text,'\n\n')
print(data)
f.writelines(data)
So what you're trying to do uses the Twitter API. Specifically the GET search/tweets endpoint.
In the docs for this endpoint:
https://dev.twitter.com/rest/reference/get/search/tweets
We can see that count has a maximum value of 100:
So even though you specify 10000, it only returns 100 because that's the max.
I've not tried either, but you can likely use the until or max_id parameters also mentioned in the docs to get more results/the next 100 results.
Keep in mind: "that the search index has a 7-day limit. In other words, no tweets will be found for a date older than one week" - the docs
Hope this helps!
You can use the field next_token of the response to get more tweets.
Refer to these articles:
https://lixinjack.com/how-to-collect-more-than-100-tweets-when-using-twitter-api-v2/
https://developer.twitter.com/en/docs/twitter-api/tweets/search/integrate/paginate
The max_id parameter is the key and it is further explained here:
To use max_id correctly, an application’s first request to a timeline
endpoint should only specify a count. When processing this and
subsequent responses, keep track of the lowest ID received. This ID
should be passed as the value of the max_id parameter for the next
request, which will only return Tweets with IDs lower than or equal to
the value of the max_id parameter.
https://developer.twitter.com/en/docs/tweets/timelines/guides/working-with-timelines
In other words, using the lowest id retrieved from a search, you can access the older tweets. As mentioned by Tyler, the non-commercial version is limited to 7-day, but the commercial version can search up to 30 days.
I've been trying to use the twitter API 1.1 Linq to Twitter (for c#) to search for tweets at #something.
This is what I came up with:
var query = (from tweet in twitterContext.Search
where tweet.Type == SearchType.Search &&
tweet.Query == "#something"
select tweet).SingleOrDefault();
The problem is that this query only returns 6 statuses and I want to get at least 100.
I've tried adding:
tweet.Count == 100
and
tweet.Statuses.Count == 100
with no luck, does anybody know what I am doing wrong?
tweet.Count is correct. Twitter is only returning 6 statuses. Twitter search isn't a normal search engine. Their search algorithm only goes back a certain number of days (undefined), doesn't necessarily return all matches, and won't return results for certain types of queries. You can test by performing the same search at https://twitter.com/search-home.