I want to scrape the Scratchpad from Khan academy but I don't know how to use the Scratchpad API.
I tried using articles and videos but it doesn't work.
Any ideas would be appreciated.
You can access the Khan Academy scratchpad API by using JSON. For example, to get the amount of votes in a program, you can use JSON like this:
$.getJSON("https://www.khanacademy.org/api/labs/scratchpads/" + programID + "&callback=?", function(data) {
console.log(data.sumVotesIncremented);
});
You can see what data you can collect here:
https://www.khanacademy.org/api/labs/scratchpads/6693037677641728 (or another id)
Related
I want to fetch the Hollywood movies data. I have referred many APIs like IMDB, omdb, etc but all these APIs are taking a query string as a required parameter.
But i want some dummy data of movies without passing query string as a parameter with all details of movies like title, poster_image etc.
So please tell me something so that I can move further on this.
This is not a direct answer, however, have you tried googling "imdb dataset"
Here is what i have found, you need to register on their site and download CSV. Which you can later import to your DB.
IMDB Dataset - Kaggle
Google deprecated their Freebase Search API, and is transferring things over to Wikidata, however there appears to be no replacement for their Freebase Search API (https://developers.google.com/freebase/v1/search-overview) that:
Autosuggesting entities (e.g. Freebase Suggest Widget)
Getting a ranked list of the most notable entities with a given name.
Finding entities using Search Metaschema.
Moreover, it would also take in malformed strings and correct them, and return nice detailed relevancy rankings, along with the associated freebase topic id. I can't find anything in their Custom Search API that returns any information relevant to their, or any other knowledge graph.
Ideally would like something that I can query similar to this and returns a result like they used to:
For example, a query of "Nirvana" in the Freebase Search API would return:
{
"status":"200 OK",
"result":[
{
"mid":"/m/0b1zz",
"name":"Nirvana",
"notable":{"name":"Record Producer","id":"/music/producer"},
"score":55.227268
},{
"mid":"/m/05b3c",
"name":"Nirvana",
"notable":{"name":"Belief","id":"/religion/belief"},
"score":44.248726
},{
"mid":"/m/01h89tx",
"name":"Nirvana",
"notable":{"name":"Musical Album","id":"/music/album"},
"score":30.371510
},{
"mid":"/m/01rn9fm",
"name":"Nirvana",
"notable":{"name":"Musical Group","id":"/music/musical_group"},
"score":30.092449
},{
"mid":"/m/02_6qh",
"name":"Nirvana",
"notable":{"name":"Film","id":"/film/film"},
"score":29.003593
},{
"mid":"/m/01rkx5",
"name":"Nirvana Sutra",
"score":21.344824
}
],
"cost":10,
"hits":0
}
Note the relevance, and Freebase mid.
Essentially are there any alternatives out there, either open source, or commercial that replaces this much needed functionality?
I've used the Prismatic Interest graph API for somewhat similar functionality. My use-case was a bit different (tagging documents with topics) but looking at their API endpoints you might be able to duplicate the functionality you described above with a query to topic/search (search for topics that match a search string) and a query to topic/topic to search for similar topics (sorted by score).
Edit
As David notes in the comments below, the Prismatic Interest Graph API has been discontinued.
Also, the Google Knowledge Graph Search API now seems to be the intended replacement for the Freebase Search API.
How about the Google Knowledge Graph Search API? There is also a web application exposing the API.
The :BaseKB project offers Freebase data (plus some other data) as RDF. :BaseKB's data can be downloaded for free or easily run on an AWS instance for live queries. The AWS machine image contains a Virtuoso database so you can query it with the SPARQL query language.
I am trying to find a list of relevant types to a certain string from Freebase, lets say for example i enter Jordan, then i will have a list with types country, person, athlete .. etc.
I have found several ways for the query, for example:
First Query
trying to get the JSON fails, using:
$.getJSON('http://api.freebase.com/api/service/search?query=jordan',function (data) {
console.log(data);
});
There is another query that gives me better result, as i only get the types here but i also cannot get the JSON file from it.
Will appreciate any help.
Your problem has probably less to do with freebase and more to do the fact that you can't do cross domain http requests. You are requesting data from api.freebase.com but you are probably hosting this page in another domain.
You can use the JSONP mechanism to circumvent that restriction, here is some documentation:
http://api.jquery.com/jQuery.getJSON/
Read the section JSONP.
Another couple of points:
Are you trying to search for all entities that somehow match the word "jordan" or are you looking for exactly all the entities that are named "jordan" ? Your best bet is to use the /search API instead of /mqlread which is for structured database queries.
You are also using the legacy API that is deprecated. Here is some docs on the new API:
http://wiki.freebase.com/wiki/API
Here's how your request will look (note that you 'll need an API key for production):
https://www.googleapis.com/freebase/v1/search?query=jordan&mql_output=[{%22name%22%20:%20null,%22type%22:[]}]
I have some couchDB database.
and i want, as in mongodb, find one item.
something like db.find({user : "John"})
That the easiest way to do it?
If you have your queries predefined, you can use views to query your database.
There is also the ability to use temporary views for ad-hoc searches, but they are never recommended for production use because the index is not saved.
If you need something more along the lines of full-text search, check out couchdb-lucene.
What is your programming language of choice?
CouchDB's API is HTTP based.
Basically you could setup a view which uses the username as key and query that via HTTP request or with the help of a "driver" for your specific language.
Views are defined as map/reduce functions. An easy introduction can be found at the official wiki for example.
Also taking a look at the CouchDB guide is a good place to start with.
I prefer using elasticsearch.
It has a couchdb _river for integration. It will listen to _changes of couchdb, then fetch and index documents.
That way you get awesome power of elasticsearch (powered by lucene), with it's RESTful interfaces and clustering ability.
You get good separation of "searching" vs. your core documents.
Which means you can index and search across different document stores.
Admittedly you don't get a nice small all in one package, but for flexibility for my use cases it wins hands down.
I have a new project to do this: http://github.com/iriscouch/query_couchdb
(Hopefully I can add an intro and documentation today.)
The idea is to copy the Google App Engine Python API.
new Query("User")
.filter("name =", "John")
.order('-age')
.get(function(er, view) {
if(er)
throw(er);
console.log("Got " + view.rows.length + " rows!");
for(var a = 0; a < view.rows.length; a++) {
var row = view.rows[a];
console.log("Row " + a + " = " + JSON.stringify(row));
}
});
Unfortunately it is missing unit tests and examples, but I am already using this in production.
There is an initiative to implement a mongo-like find with the query syntax offered by mongo db. Cloudant announced the initiative and they started contributing through mango, a MongoDB inspired query language interface for Apache CouchDB.
The Cloudant project should allow this type of queries find({user : "John"}), find({user:{$in : ["Doe", "Smith"]}}) or find({"age": {"$gt": 21}}) for age > 21
A similar alternative pouchdb-find is also being developed for pouch db.
the problem is they limit the result they return to 1,500.
but I only want to know how many results there are, not to get all of them
There is no way to know this without using the Streaming API to track the keywords you want an maintaining your own count. The Search API only contains data for about the last 7-14 days depending on tweet volume so even if you did paginate through all available tweets you would only have a recent count.
You can use the Twitter Count API:
http://urls.api.twitter.com/1/urls/count.json?url=www.apple.com&callback=twttr.receiveCount
Response:
twttr.receiveCount({"count":2596822,"url":"http://www.apple.com/"})
Note: this is not officially support by Twitter, so use at your own risk.
There's a pretty easy tutorial on grabbing Tweet counts why you can do using JSON and the TweetMeme API.
$.getJSON('http://api.tweetmeme.com/url_info.jsonc?url='+url+'&callback=?',
function(data) {
$('#twitter').append(beforecounter + data.story.url_count + aftercounter);
});
where the url variable corresponds to the url in the query you are matching.