The GitHub API documentation says that the url
https://api.github.com/users
will give all users in the order they signed up, but I only seem to get the first 135.
Any ideas how to get the real full list?
Please use since parameter in your GET request.
https://api.github.com/users?since=XXX
Probably it's done this way to limit the resources needed to handle such request. Without such limit it's just asking for DoS attack.
If you check the response headers for that request Github provides pagination links under the header Links
Link: <https://api.github.com/users?since=135>; rel="next", <https://api.github.com/users{?since}>; rel="first"
I believe since their api v3 Github has been moving towards a hypermedia api.
Github Hypermedia API
EDIT
This is beyond the scope of this question but its related. To learn more about hypermedia API and REST. Take a look at these slides by Steve Klabnik
http://steveklabnik.github.com/hypermedia-presentation/#1
Both of the existing answers are 100% correct, but I would advise you to use a wrapper for whatever language you happen to be doing this in. There are plenty of them and there is an official one for ruby (Octokit). Here is a list of all of them.
You can filter on type:user like this:
https://api.github.com/search/users?q=type:user
See Also: GitHub API get total number of users/organizations
Related
I have followed the flow described here https://developer.xero.com/documentation/oauth2/auth-flow and can then get a tokenset which works with api requests.
However, looking at https://github.com/XeroAPI/xero-node-oauth2-app/blob/master/src/app.ts I don't see how/where the authorisation code provided to the callback is used to obtain the tokenset. (compare with Steps 2 & 3 of the auth-flow.)
Looking at https://github.com/XeroAPI/xero-node/blob/master/src/XeroClient.ts I think that apiCallback() looks like it should be the place - but nowhere any mention of the authorisation code.
The example provided (and the Xero client) relies on express being the handler framework. In any other scenario there is a lot of legwork required to imitate that.
Then finally I also discovered that the Xero client insists on using openid scope otherwise the client simply doesn't work. Nothing in the docs to either indicate this, or explain why this restriction is built-in.
The xero-node package uses the openid-client package, which retrieves the code and uses it to get the token set here: https://github.com/panva/node-openid-client/blob/master/lib/client.js#L461-L481
can you elaborate on what you mean by authorization code? You are correct that the apiCalback fn returns the tokenSet which you should be saving in your database associate with each user.
const tokenSet: TokenSet = await xero.apiCallback(req.url);
One of the benefits of using the SDKs is that you don’t have to do that code exchange step. The openid client handles that exchange for you. If you are to roll your own with solution you will have to follow all the steps as described in the first documentation link ^^
—
code a temporary code that may only be exchanged once and expires 5 minutes after issuance.
So lets look at https://developer.xero.com/documentation/oauth2/auth-flow again.
Step 1 looks simple enough, but but eventually I opted to go with https://developer.xero.com/documentation/oauth2/sign-in
That reference also shows how Step 2 should be performed, something the first link omitted.
However, if you then switch back to the 1st link you'll find the next Steps easier to follow.
I want to save the data column from this url to an array in python. I tried it with, for instance, pandas.save_table:
import pandas as pd
pd.read_table('https://adventofcode.com/2019/day/1/input', sep='')
but I get HTTPError: HTTP Error 400: Bad Request and I think this is not the right way to do that.
Can someone help me with that?
If you try to open the link in your question (in a browser using incognito mode or something similar i.e. delete your cookies) you'll see that you need login into the website to access the page. This is why the you're getting a 400 Bad Request error as a response from the server.
From the FAQ section of the website that you're trying to access:
How does authentication work? Advent of Code uses OAuth to confirm
your identity through other services. When you log in, you only ever
give your credentials to that service - never to Advent of Code. Then,
the service you use tells the Advent of Code servers that you're
really you. In general, this reveals no information about you beyond
what is already public; here are examples from Reddit and GitHub.
Advent of Code will remember your unique ID, names, URL, and image
from the service you use to authenticate.
The website uses OAuth to handle logins to the url that you create will need these access tokens. You can use a library like python-oauth2 to help you with this (there are others so you can read around and decide which you'd like to use). Creating and understanding how to make http requests is beyond the scope of this answer. I'd suggest you have a look around on the internet for some explanations and try again, if you have get stuck please ask another question. Otherwise it'll probably be easier to save the file from your browser...But I'll leave this answer here for the next person who runs into the same problem.
Our tool is submitting blog entries to the idation blog for a configured community by using the Connections API.
Therefore, I use the following workflow, given only a community ID:
1) query /blogs/api/blogs?commUuid=<ID_HERE>&blogType=ideationblog
2) retrieve the link to the communities ideation blog from the xml result of aboves query. the xPath for this is "/app:service/app:workspace/app:collection[a:category[#term='entries']][1]/#href"
3) post the created blog entry payload to this url.
This all worked fine in our environment. However, when I deployed this at a customer, it did not work anymore. The url from the first step returns an empty xml document, and the following steps thus cannot be executed. I tried to query different urls on the customers server like /blogs/{homepageHandle}/api/blogs?commUuid=&blogType=ideationblog which work fine, however the query to the api service document above is the only one which contains the collection element with the link I need.
Is there any other API call I can do, to get this url? Do you know of any reason, why the call is working just fine in our environment, but fails at the customer? Might this be an access rights problem?
I am aware, that I could probably just create a url like "blogs//api/entries" and post to it, however I would prefer the above way, since I only have the communityUuid configured, and also because it is exactly the way that the API Documentation describes:
http://www-10.lotus.com/ldd/appdevwiki.nsf/xpDocViewer.xsp?lookupName=IBM+Connections+4.5+API+Documentation#action=openDocument&res_title=Creating_blog_posts_ic45&content=pdcontent
ServiceDoc -> Collection -> href
UPDATE:
This might be a problem with the SBT really. My assumption, that an empty xml document was returned was wrong, it is rather that calls via the SBT Endpoint classes are returning null.
Endpoint endpoint = EndpointFactory.getEndpoint("connections");
Object result = endpoint.xhrGet("/blogs/api"); // also tried for /blogs/<homepage>/api
When I again tried those URLs in the Browser, I got the complete results. Problem with all this is, that I can neither reproduce this in our own environment nor am I able to debug this at the customer. I tried to catch possible exceptions from this, but none are thrown. It's just that the result is null.
To clarify: The same requests work perfectly fine in our own (Connections 4.0) environment, and also from the browser at the customer. I am of course using the same user to authenticate as well in the browser as in the API calls.
endpoint.isAuthenticationValid();
also returns true, so seemingly no problem there...
I have long ago given up trying to follow the IBM documented REST API instructions (not least of all because it always ends in a myriard of REST requests just to get to the URL I need to send my request to).
I tried both your URLs (/blogs/api/blogs?commUuid=... and /blogs/<homepage>/api/blogs...) against all our Connections 4.5 systems, but although I do get an xml document back it doesn't contain a reference to the ideationblog anywhere (and yes, I made sure to quest against a Community that does contain an ideation blog).
This is a dirty workaround, which you mentioned you did not want to do, but which I do use because the documented way doesn't work:
To post blog entries, you need to POST against
/blogs/<bloghandle>/api/entries
To find out the handle (<snx:handle>) of the ideation blog in your community, you can do the following:
1.) Get the widgets-feed for the community: /communities/service/atom/community/widgets?communityUuid=...
2.) Navigate to the entry of the Ideation Blog widget: <snx:widgetDefId>IdeationBlog</snx:widgetDefId>.
Unless someone in your customer system has messed with the widgets-config.xml, the widgetDefId will be IdeationBlog.
3.) Take the <snx:widgetInstanceId> text of the Ideation Blog entry.
That is the handle of your ideation blog. (Yes, community ideation blogs are created with the widgetInstanceId of the Ideation Blog widget as handle. Normal blogs are created with some mashup of their title as handle). You can now construct the URL to post the entries to.
I have been using Foursquare Explore API endpoint since I have started developing an application. But recently it seems to be broken. It keeps on giving me following response
warning: {
text: "There aren't a lot of results near you. Try something more general, reset your filters, or expand the search area."
}
I am trying to get the response from the following HTTP URL -
https://api.foursquare.com/v2/venues/explore?ll=40.7,-74&v=20121124
(adding my credentials to the above URL)
the issue has been noticed couple days ago on this page
Foursquare venues/explore returning empty data set
I'd really like foursquare to keep us informed, because we can't rely on their API if no information about what's going on is provided :(
This bug has been fixed, all queries should be working now. Thanks for bringing this issue to our attention.
I'm trying to get NTLM Authentication working w/ Node.js. I've been reading this ( http://davenport.sourceforge.net/ntlm.html#theNtlmMessageHeaderLayout ). I send the header and get a Base64 authentication header.
I tried converting it from Base64 to UTF8 by making a new Buffer with base64 encoding and then calling toString('utf8') which returns a string something like
NTLMSSP\u0000\u0001\u0000\u0000\u0000\u0007�\b�\u0000
This is where I need help. I understand the NTLMSSP\u0000 is the null terminated signature, but and what the rest is supposed to indicate, but to me it's just garbage. It's unicode characters, but how am I supposed to get actual data out of that? I may be converting it incorrectly, which may be adding to my troubles, but I'm hoping someone can help.
Have a look at http://www.innovation.ch/personal/ronald/ntlm.html
What you receive is a Type-2 Message. The pages explains it in a very practical way. You have to extract the server challenge (nonce) and the server flags.
I just implemented a module for node.js to do just that: https://github.com/SamDecrock/node-http-ntlm
Have you looked at NTLMAPS?
You may be able to solve your problem by using it as a proxy server, but if you really want to implement NTLM auth in Javascript, then NTLMAPS provides lots of working code to study.
Sam posted the best resource I've seen for understanding what's going on.
jclulow on GitHub seems to have implemented it in a Samba library he built.
Take a look here:
https://github.com/jclulow/node-smbhash
under lib\ntlm.js you can see how he's handled the responses.
I've built client a couple of months ago using javascript, ntlm.js. Maybe that can help you get along. It was based on the documentation # innovation.ch and Microsofts own official documentation (see the references on the github page).