I've been developing for a year and change and this maybe a novice question but I've tried EVERYTHING (~150 hours worth of tries, YIKES) I will post my React Frontend and my Nodejs backend to hopefully get some clarity.
Key notes:
-I am using Auth0 authentication to build an api with a nodeJs server
-Auth0 says to use an https:// call which my localhost:3000 is not. However, everything about AUTH0 works except the API call invoked when a user logs in to redirect them and display their information on their profile. I have only found one solution to this which is a reverse proxy https:// server to make calls (I can stop here if this is the issue lol unless another easier method is out there). Also why would AUTH0 require production https servers to test???
-I have the correct CORS enabled on AUTH0's site and 99% sure NodeJs (I can get a console.log response from my API) and have tried many ways on the front end and backend
to solve.
Help would greatly, greatly, be appreciated.
Code:
function URLChecker() {
// setTimeout(function(){
// console.log("Executed immediately");
if (location.pathname.indexOf('/profile/') === 0) {
//setToken(true);
return true;
}
}
function tokenChanger() {
setToken(true);
console.log("Your token is presented as...", token)
}
useEffect(()=> {
//console.log("url checker is:" + URLChecker());
if(URLChecker() == true){
tokenChanger();
console.log(location)
if (token) {
console.log("token exists");
axios.defaults.headers.get['Access-Control-Allow-Origin'] = 'http://localhost:3000';
axios.get('http://localhost:8080/profile')
.then(res => {
//console.log(res);
console.log(res.data);
}).catch(error => {
console.log(error);
console.log("API for user FAILED")
})
}
app.get('/profile', requiresAuth(), (req, res, next ) => {
console.log(req.oidc.user);
res.header("Access-Control-Allow-Origin", "*");
res.redirect(http://localhost:3000/profile/${((req.oidc.user.nickname))})
});
(res(req.oidc.user) returns a localhost:8080/profile page that is blank with the JSON of the user's information displayed. My next step is to obviously make my frontend call a different API instead of /profile to hit an authentication required api that will return user data, however no matter what I've tried I always get stuck with the same error message. I am so close and don't know whether to stick with AUTH0 to solve this error or going with Google authentication which I hear is nice.
Thank you,
imgur link to error message on my frontend
app.get('/dbpage', function(req, res){
Chapters.find({}, function(err, allChapters){
if(err){
console.log(err);
}else{
res.render('dbpage', {chapters: allChapters});
}
});
});
The code above is a small portion of my project, and it's working fine. Chapters is the collection's name which is part of my books database. But I have another collection (from the same database) that I'm trying to pass through the same route /dbpage. How would add this other collection to the code above? So I can access the information stored in this other collection in the /dbpage route.
I've seen some other answers that are more or less suited for what I'm trying to do, but they all seen overkill for such a simple thing. Any suggestions? Thanks!!
You can't invoke the same route name. If you use duplicate route definitions, the one "listed first" will take precedence. The only time the same route definition is allowed is if you utilize a different HTTP verb... such as POST, PUT, PATCH, etc.
Thus, if you truly want to use the same route, you need to pass query params and then push conditional logic in the route, such as:
app.get('/dbpage', function(req, res){
let {someQueryParam} = req.query;
if(someQueryParam === 'someSpecialValue'){
//... do something different...
} else {
Chapters.find({}, function(err, allChapters){
if(err){
console.log(err);
}else{
res.render('dbpage', {chapters: allChapters});
}
}
});
});
And, you'd invoke it with some endpoint such as:
yourDomain.com:3000/dbPage?someQueryParam="someSpecialValue"
Honestly though I'd advise against introducing conditional logic when at all possible. If possible, just set up another endpoint.
I'm studying MEAN stack these day, So I make some sample apps following guidance. I made up "Bookshelf" application just few hours ago, this is provided by google cloud service, so I should delve into sample code to understand how it works.
Whole source code : https://github.com/GoogleCloudPlatform/nodejs-getting-started/tree/master/2-structured-data
Sample application : http://mymongo-1165.appspot.com/books
books/api.js
router.get('/', function list(req, res) {
model.list(10, req.query.pageToken,
function(err, entities, cursor) {
if (err) { return handleRpcError(err, res); }
res.json({
items: entities,
nextPageToken: cursor
});
});
});
books/curd.js
router.get('/', function list(req, res) {
model.list(10, req.query.pageToken,
function(err, entities, cursor) {
if (err) { return handleRpcError(err, res); }
res.render('books/list.jade', {
books: entities,
nextPageToken: cursor
});
}
);
});
these 2 codes are similar, but I don't know why these similar codes comes up. I think crud.js enough, but why api.js comes up. Could you explain how these 2 codes work?
In this sample application, there are two interface:
graphic user interface (GUI) - curd.js handles generating HTML that is rendered later in the browser (in our case jade tempting language is involved)
application programming interface (API) - api.js provides the way to interact with application programmatically, without browser (ex: create new record in database, or query some data by making specific call to particular route)
For deeper understanding I would suggest learning more about express.js, that will give better idea what those outputs are.
P.S. Welcome to MEAN world :)
From web application design and development point of view, how do Express and Hapi compare to each other? For basic examples they seem similar, however I'm interested to learn more about key differences in overall application structure.
For example, as far as I have learned, Hapi uses a different routing mechanism which does not take registration order into account, can do faster lookups, but is limited comparing to Express. Are there other important differences?
There is also an article about choosing Hapi (over Express) for developing the new npmjs.com website, this article states that "Hapi’s plugin system means that we can isolate different facets and services of the application in ways that would allow for microservices in the future. Express, on the other hand, requires a bit more configuration to get the same functionality", what does it exactly mean?
This is a big question and requires a long answer to be complete, so I'll just address a subset of the most important differences. Apologies that it's still a lengthy answer.
How are they similar?
You're absolutely right when you say:
For basic examples they seem similar
Both frameworks are solving the same basic problem: Providing a convenient API for building HTTP servers in node. That is to say, more convenient than using the lower-level native http module alone. The http module can do everything we want but it's tedious to write applications with.
To achieve this, they both use concepts that have been around in high level web frameworks for a long time: routing, handlers, plugins, authentication modules. They might not have always had the same names but they're roughly equivalent.
Most of the basic examples look something like this:
Create a route
Run a function when the route is requested, preparing the response
Respond to the request
Express:
app.get('/', function (req, res) {
getSomeValue(function (obj) {
res.json({an: 'object'});
});
});
hapi:
server.route({
method: 'GET',
path: '/',
handler: function (request, reply) {
getSomeValue(function (obj) {
reply(obj);
});
}
});
The difference is not exactly groundbreaking here right? So why choose one over the other?
How are they different?
The simple answer is hapi is a lot more and it does a lot more out-of-the-box. That might not be clear when you just look at the simple example from above. In fact, this is intentional. The simple cases are kept simple. So let's examine some of the big differences:
Philosophy
Express is intended to be very minimal. By giving you a small API with just a thin dusting on top of http, you're still very much on your own in terms of adding additional functionality. If you want to read the body of an incoming request (quite a common task), you need to install a separate module. If you're expecting various content-types to be sent to that route, you also need to check the Content-type header to check which it is and parse it accordingly (form-data vs JSON vs multi-part for example), often using separate modules.
hapi has a rich feature set, often exposed through configuration options, rather than requiring code to be written. For instance, if we want to make sure a request body (payload) is fully read into memory and appropriately parsed (automatically based on content-type) before the handler is ran, it's just a simple option:
server.route({
config: {
payload: {
output: 'data',
parse: true
}
},
method: 'GET',
path: '/',
handler: function (request, reply) {
reply(request.payload);
}
});
Features
You only need to compare the API documentation on both projects to see that hapi offers a bigger feature set.
hapi includes some of the following features built-in that Express does not (as far as I know):
Input and response validation (through Joi)
Server-side caching with multiple storage options (mongo, S3, redis, riak), that can be enabled with a few lines of configuration
Cookie-parsing
Sessions
File uploading/multipart parsing
CORS support
Logging
Extensibility & modularity
hapi and Express go about extensibility in quite a different way. With Express, you have middleware functions. Middleware functions are kind of like filters that you stack up and all requests run through them before hitting your handler.
hapi has the request lifecycle and offers extension points, which are comparable to middleware functions but exist a several defined points in the request lifecycle.
One of the reasons that Walmart built hapi and stopped using Express was a frustration with how difficult it was to split a Express app into separate parts, and have different team members work safely on their chunk. For this reason they created the plugin system in hapi.
A plugin is like a sub-application, you can do everything you can in a hapi app, add routes, extensions points etc. In a plugin you can be sure that you're not breaking another part of the application, because the order of registrations for routes doesn't matter and you can't create conflicting routes. You can then combine this plugins into a server and deploy it.
Ecosystem
Because Express gives you so little out of the box, you need to look outside when you need to add anything to your project. A lot of the times when working with hapi, the feature that you need is either built-in or there's a module created by the core team.
Minimal sounds great. But if you're building a serious production app, the chances are you're going to need all of this stuff eventually.
Security
hapi was designed by the team at Walmart to run Black Friday traffic so security and stability have always been a top concern. For this reason the framework does a lot of things extra such as limiting incoming payload size to prevent exhausting your process memory. It also has options for things like max event loop delay, max RSS memory used and max size of the v8 heap, beyond which your server will respond with a 503 timeout rather than just crashing.
Summary
Evaluate them both yourself. Think about your needs and which of the two addresses your biggest concerns. Have a dip in the two communities (IRC, Gitter, Github), see which you prefer. Don't just take my word. And happy hacking!
DISCLAIMER: I am biased as the author of a book on hapi and the above is largely my personal opinion.
My organization is going with Hapi. This is why we like it.
Hapi is:
Backed by major corps. This means the community support will be strong, and there for you throughout future releases. It's easy to find passionate Hapi people, and there are good tutorials out there (though not as numerous and sprawling as ExpressJs tutorials). As of this post date npm and Walmart use Hapi.
It can facilitate the work of distributed teams working on various parts of the backend services without having to have comprehensive knowledge of the rest of the API surface (Hapi's plugins architecture is the epitome of this quality).
Let the framework do what a framework is supposed to: configure things. After that the framework should be invisible and allow devs to focus their real creative energy on building out business logic. After using Hapi for a year, I definitely feel Hapi accomplishes this. I... feel happy!
If you want to hear directly from Eran Hammer (Hapi's lead)
Over the past four years hapi grew to be the framework of choice for many projects, big or small. What makes hapi unique is its ability to scale to large deployments and large teams. As a project grows, so does its complexity – engineering complexity and process complexity. hapi’s architecture and philosophy handles the increased complexity without the need to constantly refactor the code [read more]
Getting started with Hapi won't be as easy as ExpressJs because Hapi doesn't have the same "star power"... but once you feel comfortable you'll get A LOT of mileage. Took me about ~2 months as a new hacker who irresponsibly used ExpressJs for a few years. If you're a seasoned backend developer you'll know how to read the docs, and you probably won't even notice this.
Areas the Hapi documentation can improve on:
how to authenticate users and create sessions
handling Cross-Origin-Requests (CORS)
uploading files (multipart, chunked)
I think authentication would be the most challenging part of it because you have to decide on what kinda auth strategy to use (Basic Authentication, Cookies, JWT Tokens, OAuth). Though it's technically not Hapi's problem that the sessions/authentication landscape is so fragmented... but I do wish that they provided some hand-holding for this. It would greatly increase developer happiness.
The remaining two aren't actually that difficult, the docs could just be written slightly better.
Quick Facts about Hapi Or Why Hapi JS ?
Hapi is configuration-centric
It has authentication and authorization built into the framework
It was released in a battle-tested atmosphere and has really proven its worth
All the modules have 100% test coverage
It registers the highest level of abstraction away from core HTTP
Easily compassable via the plugin architecture
Hapi is a better choice performance wise
Hapi uses a different routing mechanism, which can do faster lookups, and take registration order into account.
Nevertheless, it is quite limited when compared to Express. And thanks to the Hapi plugin system, it is possible
to isolate the different facets and services that would help the application in many ways in the future.
Usage
Hapi is the most preferred framework when compared to Express. Hapi is used mainly for large-scale enterprise applications.
A couple of reasons why developers do not choose Express when creating enterprise applications are:
Routes are harder to compose in Express
Middleware gets in the way most of the time; each time you are defining the routes, you have to write as many numbers of codes.
Hapi would be the best choice for a developer looking to build RESTful API. Hapi has micro-service architecture and it is also possible to transfer the control from one handler to another based on certain parameters. With the Hapi plugin, you can enjoy a
greater level of abstraction around HTTP because you can divvy up the business logic into pieces easily manageable.
Another huge advantage with Hapi is that it provides detailed error messages when you misconfigure. Hapi also lets you configure your file upload size by default. If the maximum upload size is limited, you can send an error message to the user conveying that the file size is too large. This would protect your server from crashing because the file uploads will no longer try to buffer a whole file.
Whatever you can achieve using express can also be easily achieved using hapi.js.
Hapi.js is very stylish and organizes the code very well. If you see how it does routing and puts the core logic in controllers
you will defiantly be going to love it.
Hapi.js officially provide several plugins exclusively for hapi.js ranges from token based auth to session management and many more,
which is an ad on. It doesn't mean the traditional npm can't be used, all of them are supported by hapi.js
If you code in hapi.js, a code would be very maintainable.
I've started using Hapi recently and I'm quite happy with it. My reasons are
Easier to test. For example:
server.inject allows you to run the app and get a response without it running and listening.
server.info gives the current uri, port etc.
server.settings accesses the configuration e.g. server.settings.cache gets current cache provider
when in doubt look at the /test folders for any part of the app or supported plugins to see suggestions on how to mock/test/stub etc.
my sense is that the architectural model of hapi allows you to trust but verify e.g. Are my plugins registered? How can I declare a module dependency?
It works out of the box e.g. file uploads, return streams from endpoints etc.
Essential plugins are maintained along with the core library. e.g template parsing, caching etc. The added benefit is the same coding standards are applied across the essential things.
Sane errors and error handling. Hapi validates config options and keeps an internal route table to prevent duplicate routes. This is quite useful while learning because errors are thrown early instead of unexpected behaviours which require debugging.
Just another point to add, Hapi has started supporting 'http2' calls from version 16 onwards (if I am not wrong ). However, express is yet to support 'http2' module directly till express 4. Although they have released the feature in the alpha version of express 5.
'use strict';
const Hapi = require('hapi');
const Basic = require('hapi-auth-basic');
const server = new Hapi.Server();
server.connection({
port: 2090,
host: 'localhost'
});
var vorpal = require('vorpal')();
const chalk = vorpal.chalk;
var fs = require("fs");
var utenti = [{
name: 'a',
pass: 'b'
},
{
name: 'c',
pass: 'd'
}
];
const users = {
john: {
username: 'john',
password: 'secret',
name: 'John Doe',
id: '2133d32a'
},
paul: {
username: 'paul',
password: 'password',
name: 'Paul Newman',
id: '2133d32b'
}
};
var messaggi = [{
destinazione: 'a',
sorgente: 'c',
messsaggio: 'ciao'
},
{
destinazione: 'a',
sorgente: 'c',
messsaggio: 'addio'
},
{
destinazione: 'c',
sorgente: 'a',
messsaggio: 'arrivederci'
}
];
var login = '';
var loggato = false;
vorpal
.command('login <name> <pass>')
.description('Effettua il login al sistema')
.action(function (args, callback) {
loggato = false;
utenti.forEach(element => {
if ((element.name == args.name) && (element.pass == args.pass)) {
loggato = true;
login = args.name;
console.log("Accesso effettuato");
}
});
if (!loggato)
console.log("Login e Password errati");
callback();
});
vorpal
.command('leggi')
.description('Leggi i messaggi ricevuti')
.action(function (args, callback) {
if (loggato) {
var estratti = messaggi.filter(function (element) {
return element.destinazione == login;
});
estratti.forEach(element => {
console.log("mittente : " + element.sorgente);
console.log(chalk.red(element.messsaggio));
});
} else {
console.log("Devi prima loggarti");
}
callback();
});
vorpal
.command('invia <dest> "<messaggio>"')
.description('Invia un messaggio ad un altro utente')
.action(function (args, callback) {
if (loggato) {
var trovato = utenti.find(function (element) {
return element.name == args.dest;
});
if (trovato != undefined) {
messaggi.push({
destinazione: args.dest,
sorgente: login,
messsaggio: args.messaggio
});
console.log(messaggi);
}
} else {
console.log("Devi prima loggarti");
}
callback();
});
vorpal
.command('crea <login> <pass>')
.description('Crea un nuovo utente')
.action(function (args, callback) {
var trovato = utenti.find(function (element) {
return element.name == args.login;
});
if (trovato == undefined) {
utenti.push({
name: args.login,
pass: args.pass
});
console.log(utenti);
}
callback();
});
vorpal
.command('file leggi utenti')
.description('Legge il file utenti')
.action(function (args, callback) {
var contents = fs.readFileSync("utenti.json");
utenti = JSON.parse(contents);
callback();
});
vorpal
.command('file scrivi utenti')
.description('Scrive il file utenti')
.action(function (args, callback) {
var jsontostring = JSON.stringify(utenti);
fs.writeFile('utenti.json', jsontostring, function (err) {
if (err) {
return console.error(err);
}
});
callback();
});
vorpal
.command('file leggi messaggi')
.description('Legge il file messaggi')
.action(function (args, callback) {
var contents = fs.readFileSync("messaggi.json");
messaggi = JSON.parse(contents);
callback();
});
vorpal
.command('file scrivi messaggi')
.description('Scrive il file messaggi')
.action(function (args, callback) {
var jsontostring = JSON.stringify(messaggi);
fs.writeFile('messaggi.json', jsontostring, function (err) {
if (err) {
return console.error(err);
}
});
callback();
});
// leggi file , scrivi file
vorpal
.delimiter(chalk.yellow('messaggi$'))
.show();
const validate = function (request, username, password, callback) {
loggato = false;
utenti.forEach(element => {
if ((element.name == username) && (element.pass == password)) {
loggato = true;
console.log("Accesso effettuato");
return callback(null, true, {
name: username
})
}
});
if (!loggato)
return callback(null, false);
};
server.register(Basic, function (err) {
if (err) {
throw err;
}
});
server.auth.strategy('simple', 'basic', {
validateFunc: validate
});
server.route({
method: 'GET',
path: '/',
config: {
auth: 'simple',
handler: function (request, reply) {
reply('hello, ' + request.auth.credentials.name);
}
}
});
//route scrivere
server.route({
method: 'POST',
path: '/invia',
config: {
auth: 'simple',
handler: function (request, reply) {
//console.log("Received POST from " + request.payload.name + "; id=" + (request.payload.id || 'anon'));
var payload = encodeURIComponent(request.payload)
console.log(request.payload);
console.log(request.payload.dest);
console.log(request.payload.messaggio);
messaggi.push({
destinazione: request.payload.dest,
sorgente: request.auth.credentials.name,
messsaggio: request.payload.messaggio
});
var jsontostring = JSON.stringify(messaggi);
fs.writeFile('messaggi.json', jsontostring, function (err) {
if (err) {
return console.error(err);
}
});
console.log(messaggi);
reply(messaggi[messaggi.length - 1]);
}
}
});
//route leggere (json)
server.route({
method: 'GET',
path: '/messaggi',
config: {
auth: 'simple',
handler: function (request, reply) {
messaggi = fs.readFileSync("messaggi.json");
var estratti = messaggi.filter(function (element) {
return element.destinazione == request.auth.credentials.name;
});
var s = [];
console.log(request.auth.credentials.name);
console.log(estratti.length);
estratti.forEach(element => {
s.push(element);
//fare l'array con stringify
//s+="mittente : "+element.sorgente+": "+element.messsaggio+"\n";
});
var a = JSON.stringify(s);
console.log(a);
console.log(s);
reply(a);
}
}
});
server.start(function () {
console.log('Hapi is listening to ' + server.info.uri);
});
function EseguiSql(connection, sql, reply) {
var rows = [];
request = new Request(sql, function (err, rowCount) {
if (err) {
console.log(err);
} else {
console.log(rowCount + ' rows');
console.log("Invio Reply")
reply(rows);
}
});
request.on('row', function (columns) {
var row = {};
columns.forEach(function (column) {
row[column.metadata.colName] = column.value;
});
rows.push(row);
});
connection.execSql(request);
}
server.route({
method: 'POST',
path: '/query',
handler: function (request, reply) {
// Qui dovrebbe cercare i dati nel body e rispondere con la query eseguita
var connection = new Connection(config);
// Attempt to connect and execute queries if connection goes through
connection.on('connect', function (err) {
if (err) {
console.log(err);
} else {
console.log('Connected');
console.log(request.payload.sql);
EseguiSql(connection, request.payload.sql, reply);
}
});
}
});
server.connection({
host: process.env.HOST || 'localhost',
port: process.env.PORT || 8080
});
var config = {
userName: process.env.DB_USER,
password: process.env.DB_PASSWORD,
server: process.env.DB_SERVER,
options: {
database: process.env.DB_NAME,
encrypt: true
}
}
I'm a relatively new web programmer and I'm currently working on my first major project. I'm using angular, express (on top of node), and the graph database neo4j. Right now I'm trying to determine the best (in terms of security and speed optimization) way to set up how the web app interacts with the database.
Right now I feel like I'm going into this somewhat blindly- What I'm looking for is a guide of best practices, security issues to take into account, and any other relevant advice or pitfalls to be aware of in setting up a web app backend.
To put this into a bit more concrete terms I'll give you an idea of how I'm setting up routes right now. The following are the routes setup in the app.js file.
//match database query functions
function dataQuery(req, res) {
var func = database[req.param('query')];
func(req, res);
}
//match database create functions
function dataCreate(req, res) {
var func = database[req.param('create')];
func(req, res);
}
//handle data queries
app.get('/query/:query', dataQuery);
//handle adding new content
app.post('/create/:create', dataCreate)
Essentially I have it set up so that I POST or GET to a url that just goes and executes a function. I'm essentially naming the function I want to run in the url: /query/theNameOfTheFunction. These functions then go and either build a cypher query (neo4j's query language) utilizing information in the request to interact with the database or handles things like adding user uploaded images.
Example: Creating Content (URL: /query/createContent)
exports.createContent = function (req, res) {
var content = JSON.parse(req.query.content);
var query = ("CREATE (n:Content {Title: {title}, URL: {url}, Description: {description}, Source: {source}, Links: {links}, Value: {valueStatement} })");
query = query.replace("{title}", "\"" + content.title + "\"");
query = query.replace("{url}", "\"" + content.url + "\"");
query = query.replace("{description}", "\"" + content.description + "\"");
query = query.replace("{source}", "\"" + content.source + "\"");
query = query.replace("{links}", "\"" + content.links + "\"");
query = query.replace("{valueStatement}", "\"" + content.valueStatement + "\"");
db.query(query, function (err, results) {
if (err) {res.send()};
res.send();
});
}
Here I've got a template for the query and just drop in user generated information using replace.
Example: Adding images to server (URL: /create/addImage)
exports.addImage = function (req,res) {
var url = req.query.url;
var fileName = req.query.fileName;
console.log(req.query);
request(url).pipe(fs.createWriteStream("./img/submittedContent/" + fileName));
res.send();
}
It seems that this approach is probably not very scalable but I'm not sure how to best organize the code on the server side.
One other specific example I would like to mention is the following case. The query itself is complicated and I've pushed creating it to the client side for now (the query looks for content related the terms that the user has selected and varies in length accordingly). The client sends the query that is created it is passed into the neo4j api. Obviously there are concerns here- if the user is able to define the query they could perform any action on the database (deleting everything or whatever). I'm not clear on how someone could go about doing this exactly, but it certainly seems feasible.
exports.getContent = function (req, res) {
var query = req.query.query;
//would checking for black/white list key terms be enough security? (remove, create, set, etc)
db.query(query, function (err, results) {
if (err) {throw err};
res.send(results);
});
}
Am I going about this stuff completely wrong headed? I've never gotten a formal introduction to server side scripting and am only going off of things I've read. I would like to do it the 'right way' but I need to know what that way is first...
Just some random pointers:
I would suggest setting up a RESTful web API to handle the communication between Angular and your database; it takes the hassle out of having to invent all the routes yourself and it also means you can use great libraries like Restangular (for the client) and Restify (for the server) to handle the communications;
not sure which Neo4j driver you're using, but I'm pretty sure they all support parameterized queries, meaning that you don't need to do all those query.replace() calls (see);
depending on the number of images that might get uploaded, storing them in the filesystem might be okay, although you should never trust the passed filename; if you want a bit more scalability, you could consider using MongoDB's GridFS;
never trust queries being passed from the client to be performed on the server; if you can build the query on the client side, you can also build it on the server side with information passed from the client to the server (again, use parameterized queries);