res.render doesn't render correctly after 1023 characters - node.js

I have a parent Express app, and a Ghost app as a child app, using Ghost as an npm module here.
I routed Ghost to be rendered at http://localhost:9000/blog. All the configuration works fine (Ghost will throw an error if the basic configuration isn't being provided correctly).
Here is my Ghost startup code
ghost({
config: path.join(__dirname, '/config/ghost.config.js')
}).then(function (ghostServer) {
app.use(ghostServer.config.paths.subdir, ghostServer.rootApp);
ghostServer.start(app);
});
here is my Ghost config
// # Ghost Configuration
// Setup your Ghost install for various [environments](http://support.ghost.org/config/#about-environments).
// Ghost runs in `development` mode by default. Full documentation can be found at http://support.ghost.org/config/
var path = require('path'),
config;
config = {
// ### Production
// When running Ghost in the wild, use the production environment.
// Configure your URL and mail settings here
production: {
url: 'http://my-ghost-blog.com',
mail: {},
database: {
client: 'sqlite3',
connection: {
filename: path.join(__dirname, '/content/data/ghost.db')
},
debug: false
},
server: {
host: '127.0.0.1',
port: '2368'
}
},
// ### Development **(default)**
development: {
// The url to use when providing links to the site, E.g. in RSS and email.
// Change this to your Ghost blog's published URL.
url: 'http://localhost:9000/blog/',
// Example mail config
// Visit http://support.ghost.org/mail for instructions
// ```
// mail: {
// transport: 'SMTP',
// options: {
// service: 'Mailgun',
// auth: {
// user: '', // mailgun username
// pass: '' // mailgun password
// }
// }
// },
// ```
// #### Database
// Ghost supports sqlite3 (default), MySQL & PostgreSQL
database: {
client: 'sqlite3',
connection: {
filename: path.join(__dirname, '../blog/data/ghost-dev.db')
},
debug: false
},
// #### Server
// Can be host & port (default), or socket
server: {
// Host to be passed to node's `net.Server#listen()`
host: '127.0.0.1',
// Port to be passed to node's `net.Server#listen()`, for iisnode set this to `process.env.PORT`
port: '9000'
},
// #### Paths
// Specify where your content directory lives
paths: {
contentPath: path.join(__dirname, '../blog/')
}
},
// **Developers only need to edit below here**
// ### Testing
// Used when developing Ghost to run tests and check the health of Ghost
// Uses a different port number
testing: {
url: 'http://127.0.0.1:2369',
database: {
client: 'sqlite3',
connection: {
filename: path.join(__dirname, '/content/data/ghost-test.db')
}
},
server: {
host: '127.0.0.1',
port: '2369'
},
logging: false
},
// ### Testing MySQL
// Used by Travis - Automated testing run through GitHub
'testing-mysql': {
url: 'http://127.0.0.1:2369',
database: {
client: 'mysql',
connection: {
host : '127.0.0.1',
user : 'root',
password : '',
database : 'ghost_testing',
charset : 'utf8'
}
},
server: {
host: '127.0.0.1',
port: '2369'
},
logging: false
},
// ### Testing pg
// Used by Travis - Automated testing run through GitHub
'testing-pg': {
url: 'http://127.0.0.1:2369',
database: {
client: 'pg',
connection: {
host : '127.0.0.1',
user : 'postgres',
password : '',
database : 'ghost_testing',
charset : 'utf8'
}
},
server: {
host: '127.0.0.1',
port: '2369'
},
logging: false
}
};
module.exports = config;
So basically, when I go to http://localhost:9000/blog, it isn't being rendered at all. Nothing. I was using Chrome and also testing it using Safari. Also tested those two without JavaScript turned on.
And then I try to do curl http://localhost:9000/blog, and try using a requester app (like Postman) and they returned the correct html string. I also tried to do a curl using the user agent as Chrome and as Safari, it also returns the correct html.
I traced down to ghost node_modules, and the renderer is in ghost > core > server > controllers > frontend > index.js in this line res.render(view, result)
I changed the res.render to be like this
res.render(view, result, function(err, string) {
console.log("ERR", err);
console.log("String", string);
res.send(string);
})
and there is no error, it logs the current string, but it doesn't render anything on the browser.
I tried curl, postman, works, but browser doesn't work.
then I tried to send a hello world string, it works, the browser rendered it.
Then I add the string length one by one, and it turns out, any str.length < 1023 will be able to be rendered by the browser, but once it get past that, it doesn't.
And I tried in my parent Express app, it is able to send string which length is more than 1023, and if I use the ghost module as a standalone, it also able to send string more than 1023.
So something must have happened between those two, but I don't know how to debug this.
Please help

Related

Strapi CMS, Heroku error: no pg_hba.conf entry for host

Three months ago, I created an Strapi App that has deployed on Heroku, and everything works fine. I used macOS 10.13.6 and node 14.15.4 for the local environment
The configuration of database was created inside a file named database.js which located in rootApp/config/env/production/database.js
The following are everything config inside these file (database.js):
const parse = require('pg-connection-string').parse;
const config = parse(process.env.HEROKU_POSTGRESQL_MAROON_URL);
module.exports = ({ env }) => ({
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: 'postgres',
host: "ec2-35-169-184-61.compute-1.amazonaws.com",
port: 5432,
database: "d3d9tcukxxx",
username: "mwtwuvkwxxxx",
password: "42f0337xxxxx",
},
options: {
ssl:true,
},
},
},
});
But after 3 months (right now), I checked from heroku logs --tail then these app getting an error and the message was:
error error: no pg_hba.conf entry for host "3.86.36.125", user "mwtwuvkwtrqpir", database "d3d9tcukrk5fgh", SSL off
I used Strapi 3.2.5 , and I was deployed on Heroku Postgres with Plan free (Hobby).
I hope everyone helping me for this questions, and hope helping others for same case.
Thank you
We had the same issue on our Heroku instances and just recently found a fix.
Adding rejectUnauthorized to the database config appears to work.
config/database.js
module.exports = ({ env }) => ({
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: 'postgres',
host: env('DATABASE_HOST', 'localhost'),
port: env.int('DATABASE_PORT', 5432),
database: env('DATABASE_NAME', 'strapi'),
username: env('DATABASE_USERNAME', 'strapi'),
password: env('DATABASE_PASSWORD', 'strapi'),
schema: env('DATABASE_SCHEMA', 'public'), // Not Required
ssl: {
rejectUnauthorized: env.bool('DATABASE_SSL_SELF', false), // For self-signed certificates
},
},
options: {
ssl: env.bool('DATABASE_SSL', false),
},
},
},
});
I cannot take full credit however, it was this post on the Strapi forum that led me to the answer:
https://forum.strapi.io/t/error-no-pg-hba-conf-entry-for-host-ssl-off/3409
subsequently this link:
https://strapi.io/documentation/developer-docs/latest/setup-deployment-guides/configurations.html#database

How to add an SSL certificate (ca-cert) to node.js environment variables in order to connect to Digital Ocean Postgres Managed Database?

I am currently using node-postgres to create my pool. This is my current code:
const { Pool } = require('pg')
const pgPool = new Pool({
user: process.env.PGUSER,
password: process.env.PGPASSWORD,
host: process.env.PGHOST,
database: process.env.PGDATABASE,
port: process.env.PGPORT,
ssl: {
rejectUnauthorized: true,
// Would like to add line below
// ca: process.env.CACERT,
},
})
I found another post where they read in the cert using 'fs' which can be seen below.
const config = {
database: 'database-name',
host: 'host-or-ip',
user: 'username',
password: 'password',
port: 1234,
// this object will be passed to the TLSSocket constructor
ssl: {
ca: fs.readFileSync('/path/to/digitalOcean/certificate.crt').toString()
}
}
I am unable to do that as I am using git to deploy my application. Specifically Digital Oceans new App Platform. I have attempted reaching out to them with no success. I would prefer not to commit my certificate in my source control. I see a lot of posts of people suggesting to set
ssl : { rejectUnauthorized: false}
That is not the approach I want to take. My code does work with that but I want it to be secure.
Any help is appreciated thanks.
Alright I finally was able to figure it out. I think the issue was multiline and just unfamiliarity with dotenv for my local developing environment.
I was able to get it all working with my code like this. It also worked with the fs.readFileSync() but I didn't want to commit that to my source control.
const { Pool } = require('pg')
const fs = require('fs')
const pgPool = new Pool({
user: process.env.PGUSER,
password: process.env.PGPASSWORD,
host: process.env.PGHOST,
database: process.env.PGDATABASE,
port: process.env.PGPORT,
ssl: {
rejectUnauthorized: true,
// ca: fs.readFileSync(
// `${process.cwd()}/cert/ca-certificate.crt`.toString()
// ),
ca: process.env.CA_CERT,
},
})
.on('connect', () => {
console.log('connected to the database!')
})
.on('error', (err) => {
console.log('error connecting to database ', err)
})
Now in my config.env I had to make it look like this:
CA_CERT="-----BEGIN CERTIFICATE-----\nVALUES HERE WITH NO SPACES AND A \n
AFTER EACH LINE\n-----END CERTIFICATE-----"
I had to keep it as a single line string to have it work. But I was finally to connect with
{rejectUnauthorized:true}
For the digital ocean app platform environment variable, I copied everything including the double quotes and pasted it in there. Seems to work great. I do not think you will be able to have this setting set to true with their $7 development database though. I had to upgrade to the managed one in order to find any CA cert to download.

Upload file on the FTP server via node js

I tried simple file upload on the FTP path via node js. For that I have installed ftp-client module.
index.js
const ftpClient = require("ftp-client");
// FTP Configuration
const config = {
host: '',
port: 22,
user: '',
password: ''
};
const options = {
logging: 'basic'
};
client = new ftpClient(config, options);
client.connect(function (result) {
client.upload(['./upload/test.txt'], '/opt/test_api', {
baseDir: 'ogrds',
overwrite: 'older'
}, function (result) {
console.log(result);
})
}
);
When tried to execute by node index.js command, this is neither giving any results nor moved the file to the FTP server.

NODEJS Marklogic - Write document list cannot process response with 404 status when using documents.write()

I'm new to nodejs and marklogic, and I'm following a tutorial for a simple app, I have setup and configured my marklogin login credentials,
when I run this sample code by running node sample.js
the output is write document list cannot process response with 404 status
I wonder why I'm encountering this error,
here is the code from the tutorial,
my-connection.js
module.exports = {
connInfo: {
host: '127.0.0.1',
port: 8001,
user: 'user',
password: 'password'
}
};
sample.js
const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);
const documents = [
{ uri: '/gs/aardvark.json',
content: {
name: 'aardvark',
kind: 'mammal',
desc: 'The aardvark is a medium-sized burrowing, nocturnal mammal.'
}
},
{ uri: '/gs/bluebird.json',
content: {
name: 'bluebird',
kind: 'bird',
desc: 'The bluebird is a medium-sized, mostly insectivorous bird.'
}
},
{ uri: '/gs/cobra.json',
content: {
name: 'cobra',
kind: 'mammal',
desc: 'The cobra is a venomous, hooded snake of the family Elapidae.'
}
},
];
db.documents.write(documents).result(
function(response) {
console.log('Loaded the following documents:');
response.documents.forEach( function(document) {
console.log(' ' + document.uri);
});
},
function(error) {
console.log('error here');
console.log(JSON.stringify(error, null, 2));
}
);
I hope someone can tell me what is wrong with the code,
Thank You!
The MarkLogic NodeJS Client library is meant to run against a so-called MarkLogic REST-api instance. There is typically one running at port 8000, but you can also deploy other ones at different ports by issuing a POST call to :8002/v1/rest-apis, as described here:
http://docs.marklogic.com/REST/POST/v1/rest-apis
Port 8001 however is reserved for the MarkLogic Admin UI, which doesn't understand the REST calls that the NodeJS Client library is trying to invoke, hence the 404 (not found)..
HTH!

Syntax for defining 'pool' in KNEX/BookshelfJS

Firstly, I'm new to everything here... and new to StackOverflow, so apologies in advance for being a newbie and I'm ready for my thrashing... LOL.
We use a Heroku.addon for Postgres and utilize/reference environment variables globally to access the right database.
We have a config.js file in our application's root directory like:
db: process.env.DB_URL || {
client: 'pg',
connection: {
database: 'db_name',
user: 'user_name'
}
},
Would someone here be able to guide me on how to integrate code that initializes 'custom pool' information into this setup like the example found on
[http://knexjs.org/#Installation-pooling][http://knexjs.org/#Installation-pooling]
var knex = require('knex')({
client: 'mysql',
connection: {
host : '127.0.0.1',
user : 'your_database_user',
password : 'your_database_password',
database : 'myapp_test'
},
pool: { min: 0, max: 7 }
});
On Heroku, process.env.DB_URL is a complex URL that is similar to:
postgres://(redacted)#ec5-87-1-47-54.compute-1.amazonaws.com:5432/d8n2e9ebd0q9it
So, I am hoping there is a clean way to also pass 'custom pool' information as well either here or in another file/location.
The database is referenced throughout the backend of our application via Bookshelf/Knex. The reference to bookshelf looks similar to:
var knex = require('knex')(config.db);
var bookshelf = require('bookshelf')(knex);
You might be able to do it by reading just connection details from env variable, like this:
db: {
client: 'pg',
connection: process.env.DB_URL || {
database: 'db_name',
user: 'user_name'
},
pool: { min:5, max:20 }
},

Resources