I am storing a single array of objects in a json file. I have a worker calling an api at an interval to update the json file. The json file is being served to my site from an expressjs server. When the api is called however it serves the first version of the json file, not with the most recent data.
I saw some mention that the browser may be caching the file, however when I log the length of the array before it is served it still has the original length.
I also tried requiring the file inside the get function, thinking that might be when the file is read, however there was no change.
Here is some code:
const express = require('express')
const app = express()
const port = 3001
let jobData = require('/exampleFile')
app.get('/jobs', async (req, res) => {
let data = await jobData;
console.log(data.length)
await res.header('Access-Control-Allow-Origin', 'http://localhost:3000')
res.set('Cache-Control', 'no-store, no-cache, must-revalidate, private')
return await res.send(data)}
)
app.listen(port, () => console.log(`Example app listening on port ${port}!`))
I just found a solution, while writing the question. I changed the way the file is read to use the fs module instead of using require:
const jobData = await JSON.parse(fs.readFileSync('/exampleFile', 'utf8'));
Related
I use a REST client to test my app (Insomnia and postman). My app literally does nothing:
const express = require('express');
const app = express();
const fileUpload = require('express-fileupload');
app.use(express.json())
app.use(fileUpload());
app.post('/', fileUpload(), function(req, res) {
console.log(req.files)
res.send('a')
});
const PORT = 9999;
app.listen(PORT, () => {
console.log(`Server listening on port ${PORT}...`);
});
Whenever I try to upload a file to this service req.files is undefined. I took this code from express docs page and from a few stackoverflow questions and it just doesn't work for me. Content-length is good (40k seems right)
Content-type is auto set by my REST client. Do I have to change this?
I tried simply printing whole request object and body is empty and files is not even present there
So for anyone wondering. fileupload requires form-data Content-type. Moreover, it also requires a file to have a "key" within this form-data. It's a shame documentation fails to mention this.
Just do one thing: remove file fileUpload() from post endpoint and check, find below code for your reference.
app.post('/', function(req, res) {
console.log(req.files)
res.send('a')
});
I've got google extension, react frontend app and express server.
I use mediaRecorder to record my screen and insert it into frontend page.There is no problem, video works just fine in frontend
const blob = new Blob(chunks, { type: "video/mp4;" });
const savedVideo = document.getElementById("savedVideo");
chunks = [];
const videoURL = window.URL.createObjectURL(blob);
savedVideo.src = videoURL;
var tracks = stream.getTracks();
tracks[0].stop();
let response = await fetch('http://localhost:3001/upload', {
method: 'POST',
headers: {
'Content-Type': 'application/octet-stream',
},
body: blob
});
The problem starts when i send blob to server.I want to save video(Only on server side)I suppose problem is in handling blob on the server side, maybe I doing smth wrong, here my server code:
const express = require("express");
const cors = require('cors');
const fs = require('fs');
const app = express();
const port = 3001;
app.use(cors({
origin: 'http://localhost:3000'
}));
app.post("/upload", (req, res) => {
console.log('req.body', req.body)
req.on('readable', function(){
const data = req.read();
if(data) {
fs.createWriteStream('videeoo.mp4').write(data);
// also i didnt sure about this method to write file
}
console.log('data', data);
});
});
app.listen(port, () => {
console.log(`Server started at http://localhost:${port}`);
});
express logs
I'am waiting your best practices)Grasias!
In order to handle blobs in nodejs app.post() you should introduce express.raw() into it. Then you can create a blob from the buffer:
app.post('/raw/:cmd', express.raw({type: "*/*"}), async (req, res) => {
const buffer = req.body
const blob = new Blob([buffer], {type: "application/octet-stream"})
})
well... this is problematic... the req.read() doesn't normally process binary data. There's also a conceptual issue here: a video can potentially be huge, but in your application you're waiting for the whole file to be uploaded before you start writing it. So if you have 10 users, each uploading 10GB files, this is a problem. So you really want to store the file as it arrives, so that you only keep a few bytes in your memory at a time... but then what if you want to limit the size of the file? probably 10GB files is not something you want to deal with?
So... there are really a lot of corner cases and things to consider. In general, you don't want to handle these things manually. Luckily there are libraries like multer that can handle all these issues for you: https://expressjs.com/en/resources/middleware/multer.html you just define the destination directory, the max file size, etc and the library takes care of everything for you
I'm trying to weather Api app using node, express and Axios in backend part without using any framework like Angular or react.
I have 3 main file for my codes.
index.html
customer.js (for front end part)
server.js (for backend part)
My backend part like below;
const express = require('express');
const app = express();
const axios = require('axios').default;
API_KEY = "***";
const PORT =3000;
// app.use("/static", express.static(__dirname + '/customer'));
app.get('/', (req, res) =>{
axios
.get(`http://api.openweathermap.org/data/2.5/forecast?q=amsterdam&appid=${API_KEY}`)
.then(resp => {
let weatherDetail = resp.data;
console.log('a single country details: ', weatherDetail);
res.send(weatherDetail);
})
.catch(err => console.log(err));
});
app.listen(PORT, () => console.log(`My app listening on port ${PORT}! `));
When I write localhost:3000 on browser, I can see the weather api's data. However I want to see html file with functions in customer.js and api's data. Therefore I tried to write res.sendFile((__dirname + '/index.html')); inside app.get('/', (req, res)) function. However, in this situation I can see only html page without getting data from backend.
How can I call data getting from backend part in frontend part inside customer.js file?
My codes in customer.js like below (but I'm not sure if I use axios agan inside this file)
const apiCall = cityName => {
let apiKey = "***";
let apiUrl = `https://api.openweathermap.org/data/2.5/weather?q=${cityName}&appid=${apiKey}&units=metric`
axios
.get(apiUrl)
.then(getWeather)
.catch(err => {
console.log(err);
err.response.status === 404 ? alert(`The country ${cityName} doesn't exist.`) : alert('Server error! Sorry.');
});
};
apiCall(amsterdam)
function getWeather (response) {
let city = document.querySelector("#city");
city.innerHTML = response.data.name;
.
.
.
.
}
I would recommend to use a templating engine like handlebars or ejs.There are tons of examples for it, and sending data from backend to frontend becomes a piece of cake when using any templating engine. my personal favourite is handlebars because of its simple syntax.
It is advisable not to use document.querySelector if you're using Angular or React. React/Angular will have the browser repaint the DOM by making updates in the "root" div element of the index.html file whenever there is new data available to update.
Also, why do you want to send a HTML file? You could have a route in Node like below
route.get('/weather', (req, res) => {
// do your api call with axios to get weather data
res.json(weatherData);
});
from your front-end you could make an API call to '/weather' route and consume the JSON data
axios.get('baseUrl/weather').then(res=>{
console.log("weather data", res);
}).catch(...);
You could also fetch weather data directly from front-end like above.
So I have a node js code that updates and modifies a file content but I would like the data being inserted to come from a JavaScript code. How do I connect the two? Basically how do I have a function in node js that can be called from JavaScript?
Considering there's not much information to go off in the question, I am going to make a the assumption that you're trying to pass information from JS in a web browser to a node application.
The easiest and best documented way to do this would be to set up a simple web server using a package like expressJS and send data as a POST request using the fetch command in the browser.
Install express on the node application using the getting started guide
Write a http path where you can process the data
Start the node app
Make a call to the path we just created
Example backend code:
const express = require('express')
var bodyParser = require('body-parser')
const app = express()
const port = 3000
app.use(bodyParser);
app.post('/mypath', (req, res) => {
const myInputData = req.body.data;
//Do whatever you want with the data
})
app.listen(port, () => {
console.log(`Example app listening at http://localhost:${port}`)
})
Example front-end code:
var data = new FormData();
data.append('data', YOUR_DATA_VAR_HERE)
var options = {
method: 'POST',
body: data
}
fetch('http://localhost:3000/mypath',options)
.then(function(response){ console.log("Data was sent successfully") })
.catch(function(error) { console.log("There was an error sending data") })
Im working with a React App where I present a list top Podcasts. I'm using iTunes Search API to dynamically present data to the user. For now, I working with a Node Express server to setup my custom endpoints. The problem is that the API has a request limit, so I tought that I could save what I get from the response to Firebase and present the data from firebase instead.
To my question;
Can in some way save the response I get from iTunes Search API to Firebase?
For now my code for fetching data from my API Endpoints looks like this in my Node+Express server:
const express = require('express');
const unirest = require('unirest');
const app = express();
const port = process.env.PORT || 5000;
// Get all Episodes from a specific podcast
app.get('/api/podcast/episodes', (req, res) => {
const feedurl = req.query.feedurl
unirest.get(feedurl)
.end((response) => {
res.status(200).send(response.body)
});
});
// Get Podcast by ID
app.get('/api/podcast/:id', (req, res) => {
const podID = req.params.id;
unirest.get(`https://itunes.apple.com/lookup?id=${podID}&country=se`)
.end((response) => {
res.status(200).send(response.body)
});
});
// Get Podcast Categorys
app.get('/api/podcast/:category/:amount', (req, res) => {
const categoryID = req.params.category;
const amount = req.params.amount;
unirest.get(`https://itunes.apple.com/se/rss/toppodcasts/limit=${amount}/genre=${categoryID}/explicit=true/json`)
.end((response) => {
res.status(200).send(response.body)
});
});
// Get Podcast Categorys
app.get('/api/categorys', (req, res) => {
unirest.get('https://itunes.apple.com/WebObjects/MZStoreServices.woa/ws/genres?id=26&cc=se')
.end((response) => {
res.status(200).send(response.body)
});
});
app.listen(port, () => console.log(`Listening on port ${port}`));
Im just looking for someone who could point me in the right direction how to proceed. Cause for now I'm stuck, big time.
Depending on how long you want to cache the response, you can use a whole different things - a physical database like MySql, Sqlite, MongoDB etc to locally persist data.
If you only want to keep the cached result for a short period of time, you can use in-memory cache or just any other tool that offers you same functionality. Redis is also a good contender as a temporary store, especially when you expect to scale to more than one node instance for your application.
Below, I have modified a part of your code to cache result for 10mins, using memory-cache npm module
const express = require('express');
const unirest = require('unirest');
const cache = require('memory-cache');
const CACHE_DURATION = 10 * 60 * 1000; //10mins
const app = express();
const port = process.env.PORT || 5000;
// Get all Episodes from a specific podcast
app.get('/api/podcast/episodes', (req, res) => {
const cacheKey = req.query.feedurl; //Or anything unique to this route
const cachedData = cache.get(cacheKey);
if(cachedData) {
return res.json(cachedData);
}
const feedurl = req.query.feedurl
unirest.get(feedurl)
.end((response) => {
res.status(200).send(response.body);
cache.put(cacheKey, response.body, CACHE_DURATION);
});
});
---- the rest of your code ----
You can hit the route as many times as you want and be guaranteed that data will be fetched from iTunes only once in 10mins.
The second and subsequent requests will be served a lot faster from cache.
Let me know if this is what you are looking for.