I'm new to Node.js and Loopback. I have been using Deployd so far and I'm trying to migrate to Loopback. The S3 bucket module on Deployd was working great.
So...:
I'm on this website https://github.com/strongloop/loopback-component-storage
I run, in my project folder,
npm install loopback-component-storage
I then need to create a datasource.
To setup the new datasource, I tried
slc loopback:datasource
It doesn't provide me with the option to create a source that is a storage. So I rule that option out I guess
I see there is this piece of code on the github (link above):
var ds = loopback.createDataSource({
connector: require('loopback-component-storage'),
provider: 'filesystem',
root: path.join(__dirname, 'storage')
});
var container = ds.createModel('container');
app.model(container);
I guess this is the right way to create a datasource, but where do I place this code and how do I execute it?
How do I adapt this code to work with Amazon?
{ provider: 'amazon', key: '...', keyId: '...' }
I suppose key is my secret key and keyId my access key id, but can you confirm?
I'm just having trouble getting started... thanks for your help in advance
Where to put the code: https://github.com/strongloop/loopback-component-storage/blob/master/example/app.js
tl;dr, just put it in app.js (1.x structure) or server/server.js (2.x structure)
This example I linked to is using the old LoopBack 1.x structure. I will be updating that example in the coming weeks to use the new LoopBack 2.x structure.
Amazon provider example: http://docs.strongloop.com/display/LB/Storage+service
You can add a datasource manually in server/datasources.json too. This way, you should be able to create a container model using the storage data source.
To do this by code as you illustrated, you can either modify server/server.js or drop a JS file into server/boot with an exported function as:
module.exports = function(app) {
// your code
};
Thanks #Raymond, I took the second option.
I created file server/boot/xyz.js and put this in there:
module.exports = function(server) {
var path = require('path');
var ds = server.loopback.createDataSource({
connector: require('loopback-component-storage'),
provider: 'filesystem',
root: path.join(__dirname, '../../storage')
});
var container = ds.createModel('container');
server.model(container);
};
I cannot see the model in the explorer but I can call the service with:
http://localhost:3000/api/containers
Related
I recently discovered DynamoDB Local and started building it into my project for local development. I decided to go the docker image route (as opposed to the downloadable .jar file.
That being said I've gotten image up and running and have created a table and can successfully interact with the docker container via the aws cli. aws dynamodb list-tables --endpoint-url http://localhost:8042 successfully returns the table I created previously.
However, when I run my lambda function and set my aws config like so.
const axios = require('axios')
const cheerio = require('cheerio')
const randstring = require('randomstring')
const aws = require('aws-sdk')
const dynamodb = new aws.DynamoDB.DocumentClient()
exports.lambdaHandler = async (event, context) => {
let isLocal = process.env.AWS_SAM_LOCAL
if (isLocal) {
aws.config.update({
endpoint: new aws.Endpoint("http://localhost:8042")
})
}
(which I have confirmed is getting set) it actually writes to the table (with the same name of the local dynamodb instance) in the live AWS Webservice as opposed to the local container and table.
It's also worth mentioning I'm unable to connect to the local instance of DynamoDB with the AWS NoSQL Workbench tool even though it's configured to point to http://localhost:8042 as well...
Am I missing something? Any help would be greatly appreciated. I can provide any more information if I haven't already done so as well :D
Thanks.
SDK configuration changes, such as region or endpoint, do not retroactively apply to existing clients (regular DynamoDB client or a document client).
So, change the configuration first and then create your client object. Or simply pass the configuration options into the client constructor.
Setting up a new project, I wanted to have separate databases for test, dev and prod:
d:/mongodb/project/test
d:/mongodb/project/dev
d:/mongodb/project/prod
I got these up with mongod --dbpath d:/monodb/project/<env>
When I try to connect I get Error: More than 1 database name in URL
const { MongoClient } = require('mongodb')
MongoClient.connect('mongodb://localhost:27017/project/dev')
The example given in the api docs doesn't help much
var MongoClient = require('mongodb').MongoClient,
test = require('assert');
// Connection url
var url = 'mongodb://localhost:27017/test';
// Connect using MongoClient
MongoClient.connect(url, function(err, db) {
What is the correct specification for the url connection? (Or, if I am going about this the wrong way entirely, what is the best way to separate databases for testing?)
You can connect to mongodb using this driver as instructed in their documentation:
http://mongodb.github.io/node-mongodb-native/2.2/quick-start/quick-start/
So the URL you have there is correct.
If you want to have separate databases (which could also be on different hosts with different credentials) then I suggest you use a config package:
https://www.npmjs.com/package/config
This allows you to define a configuration for each environment where default will be a catch all if environment variable cannot be matched to a json file. In other words, NODE_ENV=prod would map to prod.json, NODE_ENV=test would map to test.json and NODE_ENV=[empty] would map to default.json. This is one possible setup.
You definitely don't want to create multiple connections for each environment. This is not necessary.
I'm currently testing Loopback and using the Arc interface.
I can connect to my oracle database and create models from the tables all served up in explorer. (all good).
but now I seem to be struggling when wanting to write some custom code and call an oracle procedure. I am unable to connect due to the oracle module not being found. (Arc works and I have all the correct packages installed)
I'm trying to connect to oracle for custom route but keep getting 'Cannot find module 'oracle''.
Code Example to try and connect is as follows;
module.exports = function(app) {
app.get('/oracletest', function(req, res) {
var settings = {};
var oracle = require("oracle")(settings);
var connectData = { "user": "userhere", "password": "passwordhere", "tns": "tnshere" };
oracle.connect(connectData, function(err, connection) {
console.log('must have connected woooohoooo');
connection.close(); // call this when you are done with the connection
});
}
This is following the guide from strong-oracle which is a dependancy installed with the loopback-oracle-connector.
Cheers
Thank you for catching the typo. The module name should be strong-oracle. Please use:
var oracle = require("strong-oracle")(settings);
I just fixed the README at https://github.com/strongloop/strong-oracle.
I'm a bit new to developing in nodejs, so this is probably a simple problem. I'm building a typical webapp based on express + sequelize. I'm using sqlite in-memory since I'm just prototyping at the moment. I understand if I were to use a persistent sqlite file, this may not be a problem, but that's not my goal at the moment. Consider the following:
var User = sequelize.define("User", {
"username": DataTypes.STRING,
// etc, etc, etc
});
sequelize.sync();
User.build({
"username": "mykospark"
});
At first, I got an error on User.build() about the Users table not existing yet. I realized that sequelize.sync() was being called async, and the insert was happening before the table was created. I then re-arranged my code so that the User.build() call was inside of sequelize.sync().complete() which fixes the problem, but I'm not sure how to apply this to the rest of my project.
My project uses models in a bunch of different places. It is my understanding that I just want to call sequelize.sync() once after my models are defined, then they can be used freely. I could probably find some way to block the entire nodejs app until sequelize.sync() finishes, but that doesn't seem like good form. I suppose I could wrap every single model operation into a sequelize.sync().complete() call, but that doesn't seem right either.
So how do people usually deal with this?
Your .sync() call should be called once within your app.js file. However, you might have additional calls if you manage multiple databases in one server. Typically your .sync() call will be in your server file and the var User = sequelize.define("ModelName"... will be in your models/modelName.js file. Sequelize suggests this type of guidance to "create a maintainable application where the database logic is collected in the models folder". This will help you as your development grows. Later in the answer, I'll provide an easy step to follow for initializing the file structure.
So for your case, you would have app.js, models/index.js and models/users.js. Where app.js would be your server running the .sync() method. In the models folder you will have the required index.js folder where you configure a connection to the database and collect all the model definitions. Finally you have your user.js files where you add your model with class and instance methods. Below is an example of the models/user.js file you might find helpful.
user.js
module.exports = function(sequelize, DataTypes) {
return sequelize.define('User', {
username: DataTypes.STRING,
},{
classMethods: {
doSomething: function(successcb, errcb, request) {}
},
instanceMethods: {
someThingElse: function(successcb, errcb, request) {}
}
});
};
models/index.js --> See here
EDIT 03/14/17
Now the best option to setup your node app with sequelize is to use sequelize-cli. This is sequelize migrations and has very useful functionality in development and production environments. For the scope of this question and revision to the answer, the best approach is the following:
npm install sequelize-cli
Use npm install sequelize-cli -g if you want it installed globally.
Then initialize sequelize migrations:
sequelize init
It should install the following folders and files structure in the folder you initiated the command:
config:
-config.json
models:
-index.js
seeders:
migrations:
If you want to create a model you can run the following command and it will auto generate the file structure for you. Here is an example
sequelize model:create --name User --attributes "user:string email:string"
Next you should be able to see the new model page in models/page.js.
config:
-config.json
models:
-index.js
-user.js
-page.js
seeders:
migrations:
You'll need to then go into you models/index.js and define your new model for your database to access the correct path for that model. Here is an example:
models/index.js
var sq = new Sequelize(dbname, user, password, config);
db = {
Sequelize: Sequelize,
sequelize: sq,
page: sq.import(__dirname + '/page.js'),
user: sq.import(__dirname + '/user.js')
}
module.exports = db;
If you need to make changes to the model you can go into the migrations folder and add methods. Follow the sequelize migration docs here. Now, about the app.js server. Before you run your server you need to initialize your databases. I use the following script to initialize the database before running the server to setup a postgres db:
postgresInit.sh
[...]
`sudo -u postgres createdb -U postgres -O $PG_USER $PG_DB. `
If you prefer a javascript solution, there is an SO solution here
app.js
[...]
console.log('this will sync your table to your database')
console.log('and the console should read out Executing (default): CREATE TABLE IF NOT EXISTS "TABLE NAME"....')
db.sequelize.sync(function(err){});
Note: Please read the edited portion of this post before answering, it might save you time and answers one of my questions.
The problem I'm having is pretty simple but I'm pretty new to this overall and I'm having issues figuring out how to implement a mongodb database connection properly in a node/express app.
I'm using express 3.x and am basing my app layout around this project supplied by the author of Express:
https://github.com/expressjs/express/tree/d8caf209e38a214cb90b11ed59fd15b717b3f9bc/examples/blog (now removed from repo)
I have no interest in making a blog however the way the app is structured appears to be quite nice. The routes are separated and everything is organized nicely.
My problem is I might have 5-6 different route js files and each route js file might have anywhere between 1 and 15 routes; of those routes 1 or 15 might want to access the db.
So my problem is it seems like a really terrible idea to do a db.open(...) every single time I want to query the db. I should mention at this point I'm using the native mongo-db driver (npm install mongodb).
I would also need to include a file like this:
http://pastebin.com/VzFsPyax
...in all of those route files and all of my model files. Then I'm also dealing with dozens upon dozens of open connections.
Is there a way I can structure my app in such a way where I only make 1 connection and it stays open for the duration of the session (having a new one made every request would be bad too)?
If so, how can I do this? If you know the answer please post a code sample using tj's blog app (the one linked earlier in this post) structure as a base guide. Basically have a way where the routes and models can use the db freely while being in separate files than the db open code.
Thanks.
EDIT
I made some progress on solving one of my issues. If you look at tj's blog example he initializes his routes in the app.js like so:
require('./routes/site')(app);
require('./routes/post')(app);
And in the routes js file it starts like this:
module.exports = function(app){
I stumbled on a project earlier today where I saw someone pass 2 variables in the modules.exports call -> function(app, db). Then figured wow could it be that easy, do I need to just adjust my routes to be (app, db) too? Yeah, it seems so.
So now part 1 of the problem is solved. I don't have to require a mongo.js file with the connection boilerplate in every route file. At the same time it's flexible enough where I can decide to pick and choose which route files pass a db reference. This is standard and has no downside right?
Part 2 of the problem (the important one unfortunately) still exists though.
How can I bypass having to do a db.open(...) around every query I make and ideally only make a connection once per session?
Other solution is to pass database to the router via request, like this:
app.js
var db = openDatabase();
var app = express();
app.all('*', function(request, response, next)
{
request.database = db;
next();
});
app.get('/api/user/:id', Users.getByID);
users.js
var Users =
{
getByID: function(request, response)
{
request.database.collection('users').findOne(...)
response.send(user);
}
};
module.exports = Users;
I made a very simple module hub for this case that replaces the use of a global space.
In app.js you can create db connection once:
var hub = require('hub');
hub.db = new Db('foobar', new Server('10.0.2.15', 27017, {}), {native_parser: false});
And use it from any other files:
var hub = require('hub');
// hub.db - here link to db connection
This method uses a feature of 'require'. Module is only loaded for the first time and all the other calls gets a reference to an already loaded instance.
UPDATE
That's what I mean:
In main file like app.js we create Db connection, open it and store into hub:
app.js:
var hub = require('hub');
hub.mongodb = require('mongodb');
hub.mongodbClient = new hub.mongodb.Db('foobar', new hub.mongodb.Server('10.0.2.15', 27017, {}), {native_parser: false});
hub.mongodbClient.open(function(error) {
console.log('opened');
});
Now in any other file (message for example) we have access to opened connection and can simple use it:
message.js:
var hub = require('hub');
var collection = new hub.mongodb.Collection(hub.mongodbClient, 'message');
module.exports.count = function(cb) {
collection.count({}, function(err, count) {
cb(err, count);
});
};
Really silly. In the documentation it seems like db.open requires to be wrapped around whatever is using it, but in reality you can use it without a callback.
So the answer is to just do a db.open() in your database connection module, app.js file or where ever you decide to setup your db server/connection.
As long as you pass a reference to the db in the files using it, you'll have access to an "opened" db connection ready to be queried.