I'm failing at trying to scrape a collection of polygon NFTs from Opensea. Could someone please provide an example that prints the html returned in console? I tried the code below:
const https = require("https");
https.get("https://opensea.io/collection/orathai", response => {
console.log(response)
});
const axios = require('axios').default;
axios.get('https://api.opensea.io/api/v1/collection/orathai/?format=json')
.then(resp => {
console.log(resp.data);
})
.catch(err => {
// Handle Error Here
console.error(err);
});
don't forget : npm i axios
Refrence :
opensea api docs
Axios
Related
enter image description here
I need to get data from this api endpoint, and send it to my client (React). Everything works fine between the frontend and backend, but I cant seem to figure out how to get the data within /dailyscores endpoint and send it using axios. Any help on why res.send is not a function inside .then, and a way to get it work?
The way you are using res as argument name for both express and axios callback is the issue here.
app.get('...', (req, res) => {
axios.get('...').then((res) => {
res.send(res.data); // here the res of axios.then is used
})
});
Instead use different names
app.get('...', (req, res) => {
axios.get('...').then((response) => {
res.send(response.data);
})
});
checkout variable scopes for more info
To get data from API please try as below:
const [apiResp, setApiResp] = useState([]);
useEffect(() => {
axios.get(`<api>`)
.then(res => {
const response = res.data;
setApiResp([response]);
})
});
I have a react project with NODE API backend. I am facing issues with the very basic fetch GET request. When passing parameters through link, it cannot be accessed at the server side.
My react function:
const loadOptions = async (cnt) => {
const response = await fetch(`${baseurl}/pdt/prev=${prev}&cnt=${cnt}`);
const resJSON = await response.json();
};
NodeJS express router code:
router.get("/pdt/:prev/:cnt", async (req, res) => {
try {
console.log(req.params.cnt);
console.log(req.params.prev);
} catch (err) {
res.json(err.message);
}
});
The result is :
Edited React code based on the answers I got above. Thank you #Phil
const response = await fetch(`${baseurl}/pdt/${prev}/${cnt}`);
It's working now.
another solution from backend
router.get("/pdt", async (req, res) => {
try {
console.log(req.query.prev);
console.log(req.query.cnt);
} catch (err) {
res.json(err.message);
}
});
and modify request
fetch(`${baseurl}/pdt?prev=${prev}&cnt=${cnt}`)
My end goal is that I want to be able to create a test that satisfies the following statement:
verify that requests to valid URLs return a 200 HTTP status code
A valid URL for example would be /about-page or /jobs, basically any directory that I add in my content folder that contains a file with the extension /index.md.
This is my code so far:
app.js
const readFilePromise = util.promisify(fs.readFile)
app.get('/*', (req, res) => {
readFilePromise(path.join(__dirname, 'content', req.url) + '/index.md', 'utf8')
.then(data => {
convertData(data, res)
})
.catch(err => {
res.status(404).send('Page doesn\'t exist!')
})
})
const convertData = (data, res) => {
const convertedData = md.render(data)
readFilePromise(path.join(__dirname, '/template.html'), 'utf8')
.then(data => {
data = data.replace(/\{\{content\}\}/, convertedData)
res.send(data)
})
.catch(err => {
console.log(err)
})
}
app.listen(3000)
module.exports = app
After reading this article, it mentions that
Requests are asynchronous, which means you must be able to conduct asynchronous tests.
So I wrote the following test:
app.test.js
const app = require('./app.js')
const request = supertest(app)
const supertest = require('supertest')
it('Gets the test endpoint and returns a 200 status', async done => {
const res = await request.get('/*')
expect(res.status).toBe(200)
done()
})
When I run the test, it fails with a 404 status, rather than returning a 200 status. I thought this might be due to my app.js not being in the async/await style, so I changed app.js to:
const readFilePromise = util.promisify(fs.readFile)
app.get('/*', async (req, res) => {
try {
await readFilePromise(path.join(__dirname, 'content', req.url) + '/index.md', 'utf8')
} catch (err) {
res.status(404).send('Page doesn\'t exist!')
}
try {
const convertedData = md.render(data)
await readFilePromise(path.join(__dirname, '/template.html'), 'utf8')
data = data.replace(/\{\{content\}\}/, convertedData)
res.send(data)
} catch (err) {
console.log(err)
}
})
app.listen(3000)
module.exports = app
I tried running the test again, but it still fails with a 404. I think my set up within app.test.js is wrong, but I'm not sure exactly what, as I've tried using the various set ups as the article. How would I fix this?
Separately, when I try going to a URL using the async/await style in app.js, I get a ReferenceError: data is not defined error, but I'm not sure how to define data in the async/await format.
I explained here how to set up app for the test environment: supertest not found error testing express endpoint
You did not mention how you set the database environment, make sure your database is not empty. Then make your get request. but just checking status for get request is not enough because if your db is empty you will still get 200.
const response = await request(app).get("/route").send().expect(200);
expect(response.body.length).toBeGreaterThan(0)
Better approach would be connect to a different database, post your data first and then check the response
const response = await request(app).get("/api/tickets").send().expect(200);
expect(response.body.length).toEqual(2); // if you post two items
Also before you every test make sure you start with empty database inside beforeEach()
I was learning to build a weather app using Node (Express) + React. I successfully fetched weather data from open weather API.
However I was directly using the open weather API key in my React app like this const weatherURL = 'http://api.openweathermap.org/data/2.5/weather?q=london,uk&APPID=1234567qwerty';. Obviously this is not safe as it exposed the API key to the client. I thought about storing the API key in .env file, but according to [this answer][1], I should never store API key in .env file or .gitignore. The right way is to make a request to backend API and make an API call to backend and send the data back. I could not find out how to do it. Can anyone help?
Following is my node js code:
const express = require('express');
const cors = require('cors');
const app = express();
const SELECT_ALL_QUERY = 'SELECT * FROM `mySchema`.`myTable`;';
app.use(cors());
app.get('/', (req, res) => {
res.send('go to /myTable to see content')
});
const pool = require('./awsPool');
pool.getConnection((err, connection) => {
if (err) {
return console.log('ERROR! ', err);
}
if(!connection) {
return console.log('No connection was found');
}
app.get('/myTable', (req, res) => {
console.log(connection);
connection.query(SELECT_ALL_QUERY, (err, results) => {
if (err) {
return res.send(err)
}
else {
return res.json({
data: results
})
};
});
});
});
let port=process.env.PORT||4000;
app.listen(port, () => {
console.log(`App running on port ${port} `);
});```
[1]: https://stackoverflow.com/a/57103663/8720421
What the linked answer was suggesting is to create a route in your Node/Express backend API that will make the call to the weather API for you, instead of the front end. This way the request and your API key are not public-facing whenever your front end makes a call.
The method for doing this would essentially be the same as what you have done in React, making an HTTP request using a built-in or 3rd party library. This resource I just found has some information on how to do both.
The simplest pure http-request in node looks like this:
const http = require('http')
const url = 'http://api.openweathermap.org/data/'
http.request(url, callback).end()
function callback (weatherResponse) {
let jsonString = ''
weatherResponse.on('data', chunk => {
jsonString += chunk
})
weatherResponse.on('end', () => {
// Now you have the complete response and can do whatever you want with it
// like return it to your user `res.send(jsonString)`
console.log(jsonString)
})
}
Many people find it bulky to having to handle chunks and the whole asynchronous thing, so there are many popular npm modules, like: https://www.npmjs.com/package/axios. (And here's a list of other contenders https://github.com/request/request/issues/3143).
Also, it is normal to store API-keys in environment variables on the backend. It makes things easy if you ever try to dockerize your app, or just scale up to using two backend servers instead of one.
I found a solution based on #ippi answer, add the following part to the original code:
const request = require('request');
const url = 'http://api.openweathermap.org/data/2.5/weather?q=london,uk&APPID=1234567';
app.get('/weather', (req, res) => {
request(url, (error, response, body) => {
if (!error && response.statusCode == 200) {
var info = JSON.parse(body)
res.send(info);
}
})
})
The url can be stored in .env file and passed into the above code. The returned weather data can be viewed in JSON format at http://localhost:4000/weather. In React the weather data can be fetched via this localhost url.
EDIT: request is deprecated, so here is a solution using axios
app.get('/weather', (req, res) => {
axios.get(url)
.then(response => {res.json(response.data)})
.catch(error => {
console.log(error);
});
})
User Passport middleware for nodeJs/Express. They provide passport-headerapikey strategy using which you can create and authorize apiKeys. http://www.passportjs.org/packages/passport-headerapikey/
I am trying to post data from my nodejs app to a an endpoint.
I tested sending some data to that endpoint using Postman and all works fine, I got the posted data and all got printed to the console.
but I am always getting empty post data at my endpoint when sending from my nodejs using axios.
here is my code:
const FormData = require('form-data');
const axios = require('axios');
function send_to_test() {
const endpoint = 'http://localhost:5000/test';
const form = new FormData();
form.append('string_var', 'some string');
form.append('integer_var', 100);
axios.post(endpoint, form, { headers: form.getHeaders() }).then((res) => {
console.log(res.data);
});
};
and here is my endpoint (I am using express js server):
app.post('/test', function(req, res) {
console.log(req.body);
res.json({
status: 'success'
});
});
Unable to get this working, please advise and thanks in advance.
You need to setup a form parser at the your endpoint. You can make use of formidable. code for reference:
const form = new formidable.IncomingForm();
form.parse(req, function(err, fields, files) {
if (err) {
return res.status(400).json({ error: err.message });
}
res.json({
status: 'success'
});
}
I hope it gives you better picture!