I'm searching for events at a particular place and also successfully retrieving all the events. But suppose I'm getting about 1500 events as "total_items" and the first page of response shows up the first 10 events.Then how could I reach to the remaining set of events, there isn't any "next page link".
And the most important question that are there any restrictions or upgradation plan for Eventful. Just because I was not able to find the required answers on their web portal. I'm posting it over here.
You are getting 10 events as default page size is 10 .If you want to change page size ,you can change it in your query.
http://api.eventful.com/rest/events/search?l=GREEN+BAY+WEST&page_size=100
In the above query it will return 100 events,you can get 100 events per page maximum.
If you want to go to next page,you can change page_number in your query.
Related
I wanted to get the first 2 items from the AdAnalyticsV2 endpoint. As explained here you should be able to do this for any service by adding a count parameter to the query, but adding e.g. count=2 to the query doesn't seem to do anything. I still get the same amount items back as usual whenever I add it. Also, changing the start, so it just gets the next 10 items, results in the following error: StatusCodeError: 400 - {"serviceErrorCode":2,"message":"Setting a paging start value is not permitted","status":400}
Is pagination not available for the AdAnalyticsV2 endpoint?
There is a weird bug with pagination where if there are exactly 10 results and you didn't specify a count it was giving me that message. I set the count to 5000 and have had no problems.
This seems like a pretty simple thing but I can't find any discussions that really explain how to do it.
I'm building a scraper with MongoDB and Node.js. It runs once daily and scrapes several hundred urls and records to the database. Example:
Scraper goes to this google image search page for "stack overflow"
Scraper gets the top 100 links from this page
A record of the link's url, img src, page title and domain name are saved to MongoDB.
Here's what I'm trying to achieve:
If the image is no longer in the 100 scraped links, I want to delete it from the databqse
If the image is still in the 100 scraped links, but details have changed (e.g. new page title) I want to find the mongodb record and update it.
If the image doesn't exist already, I want to create a new record
The bit I'm having trouble with is deleting entries that haven't been scraped. What's the best way to achieve this?
So far my code successfully checks whether entries exist, updates them. It's deleting records that are no longer relevant that I'm having trouble with. Pastebin link is here:
http://pastebin.com/35cXcXzk
You either need to timestamp items (and update them on every scrape) and periodically delete items which haven't been updated in a while, or you need to associate items with a particular query. In the latter case, you would gather all of the items previously associated with the query, and mark them off as the new results come in. Any items not marked off the list at the end, need to be deleted.
another possibility is to use the new TTL index option in mongodb 2.4 allowing you to set time to live on documents
http://docs.mongodb.org/manual/tutorial/expire-data/
This will let the server expire them over time instead of having to perform big expensive remove executions.
Another optimization is to use the power of 2 option for collections to avoid the high fragmentation of memory that write, remove cycles create
http://docs.mongodb.org/manual/reference/command/collMod/
I have added a search functionality to my ExpressionEngine site, using the standard Search Module tags. However, I occasionally run into this error message:
You are only allowed to search every 15 seconds.
This only seems to happen when I'm logged out.
I want to decrease the time required between searches. How can I adjust this?
It's a per-group setting (hence why it's only happening when you're logged out). The default for logged out members (the Guests group) is 15 seconds.
You can adjust this value for each group under Members > Member Groups > Edit Group in the control panel.
I'm working on an instagram scraper for something and I'm trying to figure out if it's possible to get all photos for a tag that have an id or timestamp later than the last one I have.
The instagram API docs are useless in that they don't have any real info on pagination (which I presume I'll have to abuse).
Does anyone have any ideas?
I've been slogging through the Instagram API for the last couple of days so here's my 2 cents worth:
As far as I can see it if you call the api with /tags/tag-name/media/recent it only return a list if items. If the amount exceeds about 25 you have to make another request with the pagination value returned in the previous request.
In order to gain some control I am initially iterating through all images and storing the results (just the URL not the actual image) to a database. Now I can manipulate however I want. When I feel like updating (I'm doing it manually now but could be a cron job or use the real-time api) I re-read all the images, compare to what I have in my DB and add possible new images. My app then reads out the url and info from my DB (which btw is a heck of a lot faster than going through the instagram api, which will only return about 25 images per request - regardless of any 'count' parameter value you put in the request url) and displays it.
I am developing this for a client who is afraid of people posting nsfw or whatever pics using their dedicated hashtag (for a contest) - with the above set up I can offer them an interface where they can check and mark images that are then displayed in the app.
One thing to watch out for is when a user deletes his picture; you will have to find a way to check for this. Currently (since I'm lazy) I load all images and use jquery to check for an error loading the image. If there is one I delete the image from the DB (via ajax).
I'm not sure the pagination is going to help you: as far as I can see the pagination response has no relation to the id's of the actual image objects on each page - so theoretically a pagination id that jumps to a certain page (i.e. date) might not work tomorrow if enough images have been deleted in the mean time.
to get all images instead of latest 20, just append &count=-1 to your api call - it's that simple.
In either case, there is a timestamp on each json object - or if you prefer, you can use max_tag_id
check out my post here: there any way to show more than 20 photos of the instagram API?
* Update April 2014: count=-1 is no longer available.
I have developed a Web Interface for a db. The db and Web Interface are for my own use in my hobby running on my private intranet. Currently the db has 1800+ records which is going to increase with usage. Ver 1 of the Web Interface listed all records (~2.5KB) on a single page requiring a ton of scrolling. Vers 2 of the Web interface introduced pagination where records are grouped into a non-fixed size of roughly 100 records. On page load all 1800+ records are still transferred to the client but only the first page is "visible", the other 17 are hidden. I use a series of "non-submit" buttons with JS on-click functionality to hid the current page and make the selected page visible. Better in that scrolling is limited to ~100 records. Vers 3 only transfers the first page and the paging buttons on page load. Now, the on-click function using fetch() API sends a GET Request with parameters to fetch the desired page then swaps it into the DOM. The parameters specify a starting and ending points for the page. These values come from the paging buttons supplied by the server on page load. Works well with significantly reduced data transfer size. In Vers 4 I am generalize the fetch() API GET Request parameters to send user specified parameters to allow the user to choose any page starting and ending point. (Note: The user can not specify a page size directly.) So if the user selects a start and end point that no records fall into my plan is to use HTTP Response Code 204 "No Content" to tell the JS code that there are no matching records and nothing to swap. Is this the appropriate Response Code? Should I be including any other Header information in the Response with the 204 code?
Take a look at what the RFC says about 204:
https://www.rfc-editor.org/rfc/rfc7231#section-6.3.5
It's really intended for PUT requests. I think for what you are doing, it's fine to return 200 and no body, with a Content-Length header of 0.