I am trying to web scrape a site using a node.js server. I am using axios to make my http request. When I make the request I get a 403 error from the server.
Using Postman I can successfully make the request and return the HTML file. Why is Postman able get a 200 code and my request fails? What are some things I can try to successfully make the request?
//basic axios request Im using
axios.get(url)
.then(response => {
console.log(response.data);
})
.catch(error => {
console.log(error);
})
Got exactly same situation: authorized only POST method working on postman, not working with axios.
Fixed it instead of calling axios.post(), calling:
axios({
method: 'POST',
url: 'http://localhost:1337/post',
headers: {
Authorization: `Bearer ${localStorage.getItem('token')}`
},
data: payload
})
For hint : Refer this github issue
Related
I tried to call external API in my API server with axios and it worked. this is the code
const result = await axios.get(
`http://external-api-url-here.com/example-endpoint/xyz`,
{
headers: {
"Content-Type": "application/json",
"App-key": 'ExAmPl3_KeY-H3r3',
"key-platform": 'admin-desktop',
},
}
);
but the problem is when I try the code in my web client application, when it runs in the browser and tries to make a request with the code it doesn't work, getting a 307 response.
Then I added withCredentials: true in the axios header. getting a 401 response,
then I copied cURL from the browser and imported it into postman and it turned out that what made the API response fail was Cookies and Referer. When I remove those 2 key headers from postman, I get response 200.
what is the solution for this?
I want to that code get response 200 at the browser
I have below API for fetching all jobs on a project in gitlab:
fetch("https://git.nmlv.nml.com/api/v4/projects/project_id_here/jobs", {
method: 'GET',
headers: {
'Authorization': 'Basic my-token-here'
},
'Accept': 'application/json',
'mode': "no-cors"
})
.then(response => {
console.log("hello")
return response.text();
})
.then(
text => {
var result= JSON.parse(text);
console.log(text)
}).catch(err=>{
console.log(err);
})
The above request works fine in Postman with the same token but here it is saying that the request is unauthorized. The problem is on the Gitlab API docs, they haven't specified how the request in javascript should look like. for your reference, here is the API that I want to call. I know something is incorrect in the way I have framed the API's header. Can anyone help me to find how to frame the request correctly.
Thanks
EDIT
The problem now is that when I run same request on browser inside an html page, response is coming fine. But it is not working inside a node script. The control is not going to then or catch block.
I'm trying to integrate our website with Converge API with Hosted Payments Page. Here is the link to their documentation https://developer.elavon.com/#/api/eb6e9106-0172-4305-bc5a-b3ebe832f823.rcosoomi/versions/5180a9f2-741b-439c-bced-5c84a822f39b.rcosoomi/documents?converge-integration-guide/book/integration_methods/../../book/integration_methods/hosted_payments.html
I'm having troubles getting past the first step which is requesting a transaction token from their API endpoint. I'm sending a POST request from my server using axios with the correct parameters and URL, but when I try and POST i get 400 Bad Request. When I make the same request in POSTMAN I get a 200 response with the transaction token. I talked to their developers and they said that everything I was doing was correct and that nothing seemed odd within my code, so even they were stumped as to why I couldn't make a POST request to their endpoint. Obviously there is something within my code that their API is not liking, or else I wouldn't be here trying to find answers for this.
Here is how I'm making the POST request:
app.get('/converge_token_req', (request, response) => {
let params = {
ssl_merchant_id: '*****',
ssl_user_id: '*****',
ssl_pin: '*****',
ssl_transaction_type: 'ccsale',
ssl_amount: '1.00'
}
axios.post('https://api.demo.convergepay.com/hosted-payments/transaction_token', params, {
headers: { 'Content_Type' : 'application/x-www-form-urlencoded' }
}).then((res) => {
response.send(res.data)
}).catch((error) => {
console.log('there was an error getting transaction token')
response.send(error.message)
})
})
Here are the Request Headers:
I'm honestly out of ideas to try. The developers say that everything looks just fine yet I'm unable to make a successful request to their API. If anyone has any thoughts on this that would be great. Thanks!
This code below worked for me:
app.get('/converge_token_req', (request, response) => {
let params = {
ssl_merchant_id: '*****',
ssl_user_id: '*****',
ssl_pin: '*****',
ssl_transaction_type: 'ccsale',
ssl_amount: '1.00'
}
axios({
method: 'post',
url: 'https://api.demo.convergepay.com/hosted-payments/transaction_token',
params: params
}).then((res) => { response.send(res.data)
}).catch((error) => {
console.log('there was an error getting transaction token: ',
error)
})
})
I've since found out the solution to my problem. The issue here is that converge expects a x-www-form-urlencoded string that needs to be Stringified before submitting the request. I found a library that works well for this called qs and I used it like so:
let params = qs.stringify({ // need this if content_type is application/x-www-form-urlencoded
ssl_merchant_id: env.CONVERGE.MERCHANT_ID,
ssl_user_id: env.CONVERGE.USER_ID,
ssl_pin: env.CONVERGE.PIN,
ssl_transaction_type: request.query.type,
ssl_amount: request.query.amount,
ssl_email: request.query.email,
ssl_company: request.query.company,
ssl_avs_address: request.query.address,
ssl_avs_zip: request.query.zip,
ssl_description: request.query.desc,
})
axios.post('https://api.convergepay.com/hosted-payments/transaction_token', params, {
headers: {
'Content_Type' : 'application/x-www-form-urlencoded; charset=UTF-8'
}
}).then((res) => {
response.send(res.data)
}).catch((error) => {
console.log('there was an error getting transaction token')
response.send(error.message)
})
I think you could also get away with just using JSON.stringify() but this way worked for me.
Using the sample Python code provided by the Bureau of Labor Statistics I was able to successfully access their API and print the JSON containing the data series in the console. Python code below:
#BLS query.
headers = {'Content-type': 'application/json'}
data = json.dumps({"seriesid": ['CES0000000001'],"startyear":"2010", "endyear":"2019"})
result = requests.post('https://api.bls.gov/publicAPI/v2/timeseries/data/', data=data, headers=headers)
print(result.text)
While the Python code works just fine, I would prefer to use JS, but have been unsuccessful in doing so. I have tried using fetch, but I am not making any progress. See JS code below:
fetch('https://api.bls.gov/publicAPI/v2/timeseries/data/', {
method: 'POST',
headers: {
'Content-type': 'application/json'
},
body: JSON.stringify({seriesid: ['CES0000000001'], startyear:"2010", endyear:"2019"})
})
.then(function(response) {response.json()})
.then(function(json) {console.log(json)});
I am sure I am messing up something simple here, but I'm at a loss. Any help would be greatly appreciated. For reference, additional info from the BLS on their API can be found at this link:
https://www.bls.gov/developers/api_signature_v2.htm
Try using this,
const data = { username: 'example' };
fetch('https://api.bls.gov/publicAPI/v2/timeseries/data', {
method: 'POST', // or 'PUT'
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify(data),
})
.then((response) => response.json())
.then((data) => {
console.log('Success:', data);
})
.catch((error) => {
console.error('Error:', error);
});
But what I think the real problem might be that you are using an API which has authentication used, so make sure that you are using the api key along with the post req itself.
if you can have the api documentation, please refer it and see how to make a authenticated request to the server.
If you want to use regular JavaScript's fetch in Node.js, it won’t work, One reason for that is because, NodeJs doesn't make requests via the browser, but Fetch API was made to make requests via the browser
You’d have to use a package called node-fetch, it's just like the regular fetch, but for NodeJs.
You can get it here -> https://www.npmjs.com/package/node-fetch
or you can also use the standard NodeJS HTTP package.
or packages like axios or request to make HTTP requests in NodeJS
Authorization for Client Credentials Flow
Hi I have read other Questions or not working properly the current response is 400 (Bad Request) what I had
The following is my code (Authorization has converted Base64)
const testAuth = () => {
return () => {
Axios({
url: 'https://accounts.spotify.com/api/token',
method: 'post',
params: {
grant_type: 'client_credentials'
},
headers: {
'Authorization': 'Basic MWM3NGFkOGQyNDgzNDI0Y2E4NGVmYWRlNzI1MzI5YzE6MDBmMGFmNDE1ZTZhNDgxOThiOWRlYzFmNmE2NTk5NDQ=',
'Content-Type': 'application/x-www-form-urlencoded'
},
}).then((respond) => {
console.log(respond);
}).catch((error) => {
console.log(error);
});
};
But things that are working with returned tokens as well as using spotify wep api normally by using Postman to send out the same content . Is my code uncorrent or is there any problem? (Authorization in Postman is the same as above)
Thank everyone, I just realized is an error of cors. However, there are many ways for trying still can't solve the 400.
For eaxmple: Chrome extension access-control-expose-headers
Preflighted Requests image
400 Respond image