Mongodb - replica set - max connections - node.js

I have a replicaset of 3 mongo node, 1 primary, 1 secondary and 1 arbiter.
Connected on this replicaset, i have 20 node process, on 20 different serveur using their own connections to the replicaset. All those process use mongoose.
My primary replicaset show the following :
rsProd:PRIMARY> db.serverStatus().connections
{ "current" : 284, "available" : 50916, "totalCreated" : NumberLong(42655) }
From time to time, when i restart some nodejs node i have the following errors :
mongodb no valid seed servers in list
My connection string to the replicaset is the following :
"mongodb://mongo2aws.abcdef:27017/dbname,mongo1.abcdef:27017/dbname"
And my db options are the following :
config.db_options = {
user: "MYUSER",
pass: "MYPASSWORD",
replset: {
rs_name: "RSNAME",
ssl: true,
sslValidate:false,
sslCA: ca,
ca: ca,
sslKey: key,
sslCert: key
},
socketOptions : {
keepAlive : 1,
connectTimeoutMS : 1000
},
server: {
ssl: true,
sslValidate:false,
sslCA: ca,
ca: ca,
sslKey: key,
sslCert: key
},
auth: {
authdb: 'MYAUTHDB'
}
};
I haven't this error when i was running only 16 node process.
According to this i suppose that i have reach a limit of max concurrent connections or something like this.
But, if i restart again crashing node, it finally seems to work.
Why mongo / mongoose raise this error ?
What can i do to prevent this / increase limit ?
Thanks in advance
Best regards.

Solved by increasing ulimit open files
Default ulimit for open files in AWS EC2 ubuntu server is set to 1000 by default.
In addition, adding reconnect options prevent this problem :
config.db_options.reconnectTries=10;
config.db_options.reconnectInterval=500;
config.db_options.poolSize=20;
config.db_options.connectTimeoutMS=5000;

Related

RethinkDB pub/sub ReqlPermissionError

I'm using Publish-Subscribe with RethinkDB.
For exchange I use one db named 'RPI_messages' with three tables:
Connector_messages
MAC_messages
Orders
I want to introduce some basic authentication on producer and consumer sides like this:
r.db('rethinkdb').table('users').insert({id: 'lis', password: 'somepassword'})
r.db('rethinkdb').table('users').insert({id: 'rpi', password: 'someotherpassword'})
r.db('RPi_messages').grant('lis', {read: false, write: true, config: true}) //producer
r.db('RPi_messages').grant('rpi', {read: true, write: false, config: true}) //consumers
r.db('rethinkdb').table('permissions')
shows this:
{
"database": "RPi_messages" ,
"id": [
"lis" ,
"007928e5-c654-4311-b3aa-a834c62dcf88"
] ,
"permissions": {
"config": true ,
"read": false ,
"write": true
} ,
"user": "lis"
}
Problem:
When I try to publish or to subscribe to exchange it throws an exception:
ReqlPermissionError: User `lis` does not have the required `config` permission in:
r.db_create('RPi_messages')
rethinkdb.errors.ReqlPermissionError: User `rpi` does not have the required `config` permission in:
r.db_create('RPi_messages')
Does this mean that my users need to have global permissions?
Any help greatly appreciated.
So, with fairly bit of try and error I made it work.
For anyone else: You need config permission on global scope for every user in your pub-sub system like this:
r.grant('lis', {read: false, write: false, config: true});
r.grant('rpi', {read: false, write: false, config: true});
...and following permissions on table(s) or, in my case, on database scope:
r.db('RPi_messages').grant('lis', {read: true, write: true, config: true}); //publisher
r.db('RPi_messages').grant('rpi', {read: true, write: false, config: true}); //sunscriber
Correct me if I'm wrong but this doesn't look very secure to me. Those permissions are needed because Exchanger class in rethink's pub-sub system looks if exchange table exists and creates on if it doesn't. Which means that anyone who gets one of your client subscriber devices can create as many tables in your db as they want.

I am unable to connect Mongodb atlas Cluster from node js getting following unable to connect DB error

{ error: 1, message: 'Command failed: mongodump -h cluster0.yckk6.mongodb.net --port=27017 -d databaseName -p -u --gzip --archive=/tmp/file_name_2022-09-19T09-42-05.gz\n' + '2022-09-19T14:42:08.931+0000\tFailed: error connecting to db server: no reachable servers\n' }
Can anyone help me to solve this problem and following is my backup code
function databaseBackup() {
let backupConfig = {
mongodb: "mongodb+srv://<username>:<password>#cluster0.yckk6.mongodb.net:27017/databaseName?
retryWrites=true&w=majority&authMechanism=SCRAM-SHA-1", // MongoDB Connection URI
s3: {
accessKey: "SDETGGAKIA2GL", //AccessKey
secretKey: "Asad23rdfdg2teE8lOS3JWgdfgfdgfg", //SecretKey
region: "ap-south-1", //S3 Bucket Region
accessPerm: "private", //S3 Bucket Privacy, Since, You'll be storing Database, Private is HIGHLY Recommended
bucketName: "backupDatabase" //Bucket Name
},
keepLocalBackups: false, //If true, It'll create a folder in project root with database's name and store backups in it and if it's false, It'll use temporary directory of OS
noOfLocalBackups: 5, //This will only keep the most recent 5 backups and delete all older backups from local backup directory
timezoneOffset: 300 //Timezone, It is assumed to be in hours if less than 16 and in minutes otherwise
}
MBackup(backupConfig).then(onResolve => {
// When everything was successful
console.log(onResolve);
}).catch(onReject => {
// When Anything goes wrong!
console.log(onReject);
});
}

Login problems connecting with SQL Server in nodejs

I'm working in osx with SQL Server using a docker image to be able to use it, running:
docker run -d --name sqlserver -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=myStrongPass' -e 'MSSQL_PID=Developer' -p 1433:1433 microsoft/mssql-server-linux:2017-latest
I can connect successfully in Azure Data Studio GUI with the following configuration
But the connection does not works in my nodejs code using mssql module.
const poolConnection = new sql.ConnectionPool({
database: 'myDbTest',
server: 'localhost',
port: 1433,
password: '*******',
user: 'sa',
connectionTimeout: 5000,
options: {
encrypt: false,
},
});
const [error, connection] = await to(poolConnection.connect());
The error always is the same:
ConnectionError: Login failed for user 'sa'
Is my first time working with SQL Server and is confusing for me the fact that I can connect correctly in the Azure Studio GUI but I can't do it in code.
I'm trying create new login users with CREATE LOGIN and give them privileges based on other post here in stackoverflow but nothing seems to work.
UPDATE:
I realize that i can connect correctly if i put master in database key.
Example:
const poolConnection = new sql.ConnectionPool({
database: 'master', <- Update here
server: 'localhost',
port: 1433,
password: '*******',
user: 'sa',
connectionTimeout: 5000,
options: {
encrypt: false,
},
});
1) Db that i can connect
2) Db that i want to connect but i can't.
Container error
2020-03-18 03:59:14.11 Logon Login failed for user 'sa'. Reason: Failed to open the explicitly specified database 'DoctorHoyCRM'. [CLIENT: 172.17.0.1]
I suspect a lot of people miss the sa password complexity requirement:
The password should follow the SQL Server default password policy, otherwise the container can not setup SQL server and will stop working. By default, the password must be at least 8 characters long and contain characters from three of the following four sets: Uppercase letters, Lowercase letters, Base 10 digits, and Symbols. You can examine the error log by executing the docker logs command.
An example based on: Quickstart: Run SQL Server container images with Docker
docker pull mcr.microsoft.com/mssql/server:2017-latest
docker run -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=myStr0ngP4ssw0rd" -e "MSSQL_PID=Developer" -p 1433:1433 --name sqlserver -d mcr.microsoft.com/mssql/server:2017-latest
docker start sqlserver
Checking that the docker image is running (it should not say "Exited" under STATUS)...
docker ps -a
# CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
# af9f01eacab2 mcr.microsoft.com/mssql/server:2017-latest "/opt/mssql/bin/nonr…" 45 seconds ago Up 34 seconds 0.0.0.0:1433->1433/tcp sqlserver
Testing from within the docker container that SQL Server is installed and running...
docker exec -it sqlserver /opt/mssql-tools/bin/sqlcmd \
-S localhost -U "sa" -P "myStr0ngP4ssw0rd" \
-Q "select ##VERSION"
# --------------------------------------------------------------------
# Microsoft SQL Server 2017 (RTM-CU19) (KB4535007) - 14.0.3281.6 (X64)
# Jan 23 2020 21:00:04
# Copyright (C) 2017 Microsoft Corporation
# Developer Edition (64-bit) on Linux (Ubuntu 16.04.6 LTS)
Finally, testing from NodeJS...
const sql = require('mssql');
const config = {
user: 'sa',
password: 'myStr0ngP4ssw0rd',
server: 'localhost',
database: 'msdb',
};
sql.on('error', err => {
console.error('err: ', err);
});
sql.connect(config).then(pool => {
return pool.request()
.query('select ##VERSION')
}).then(result => {
console.dir(result)
}).catch(err => {
console.error('err: ', err);
});
$ node test.js
tedious deprecated The default value for `config.options.enableArithAbort` will change from `false` to `true` in the next major version of `tedious`. Set the value to `true` or `false` explicitly to silence this message. node_modules/mssql/lib/tedious/connection-pool.js:61:23
{
recordsets: [ [ [Object] ] ],
recordset: [
{
'': 'Microsoft SQL Server 2017 (RTM-CU19) (KB4535007) - 14.0.3281.6 (X64) \n' +
'\tJan 23 2020 21:00:04 \n' +
'\tCopyright (C) 2017 Microsoft Corporation\n' +
'\tDeveloper Edition (64-bit) on Linux (Ubuntu 16.04.6 LTS)'
}
],
output: {},
rowsAffected: [ 1 ]
}
Hope this helps.

Sequelize "Connection closed" on Azure SQL

I have a Node v10.17.0 app on Ubuntu 16.04 using Sequelize 4.41.1 calling Azure SQL.
On every start up, the node app runs sync on each database model for any migrations. Sequelize sends queries to the database IF OBJECT_ID('[Shop]', 'U') IS NULL CREATE TABLE [Client] ... etc to create the tables if they don't exist. Then EXEC sys.sp_helpindex #objname = N''[Shop]. The first couple of tables sync successfully (there aren't any changes).
Then we get to the query that returns the error SequelizeDatabaseError message":"Connection closed before request completed." "code":"ECLOSE"
IF OBJECT_ID('[Client]', 'U') IS NULL CREATE TABLE [Client] ([id] NVARCHAR(255) , [name] NVARCHAR(255) NOT NULL, [timezone] VARCHAR(255) CHECK ([timezone] IN(N'Africa/Abidjan', N'Africa/Accra', N'Africa/Addis_Ababa', N'Africa/Algiers', N'Africa/Asmara', N'Africa/Asmera', N'Africa/Bamako', N'Africa/Bangui', N'Africa/Banjul', N'Africa/Bissau', N'Africa/Blantyre', N'Africa/Brazzaville', N'Africa/Bujumbura', N'Africa/Cairo', N'Africa/Casablanca', N'Africa/Ceuta', N'Africa/Conakry', N'Africa/Dakar', N'Africa/Dar_es_Salaam', N'Africa/Djibouti', N'Africa/Douala', N'Africa/El_Aaiun', N'Africa/Freetown', N'Africa/Gaborone', N'Africa/Harare', N'Africa/Johannesburg', N'Africa/Juba', N'Africa/Kampala', N'Africa/Khartoum', N'Africa/Kigali', N'Africa/Kinshasa', N'Africa/Lagos', N'Africa/Libreville', N'Africa/Lome', N'Africa/Luanda', N'Africa/Lubumbashi', N'Africa/Lusaka', N'Africa/Malabo', N'Africa/Maputo', N'Africa/Maseru', N'Africa/Mbabane', N'Africa/Mogadishu', N'Africa/Monrovia', N'Africa/Nairobi', N'Africa/Ndjamena', N'Africa/Niamey', N'Africa/Nouakchott', N'Africa/Ouagadougou', N'Africa/Porto-Novo', N'Africa/Sao_Tome', N'Africa/Timbuktu', N'Africa/Tripoli', N'Africa/Tunis', N'Africa/Windhoek', N'America/Adak', N'America/Anchorage', N'America/Anguilla', N'America/Antigua', N'America/Araguaina', N'America/Argentina/Buenos_Aires', N'America/Argentina/Catamarca', N'America/Argentina/ComodRivadavia', N'America/Argentina/Cordoba', N'America/Argentina/Jujuy', N'America/Argentina/La_Rioja', N'America/Argentina/Mendoza', N'America/Argentina/Rio_Gallegos', N'America/Argentina/Salta', N'America/Argentina/San_Juan', N'America/Argentina/San_Luis', N'America/Argentina/Tucuman', N'America/Argentina/Ushuaia', N'America/Aruba', N'America/Asuncion', N'America/Atikokan', N'America/Atka', N'America/Bahia', N'America/Bahia_Banderas', N'America/Barbados', N'America/Belem', N'America/Belize', N'America/Blanc-Sablon', N'America/Boa_Vista', N'America/Bogota', N'America/Boise', N'America/Buenos_Aires', N'America/Cambridge_Bay', N'America/Campo_Grande', N'America/Cancun', N'America/Caracas', N'America/Catamarca', N'America/Cayenne', N'America/Cayman', N'America/Chicago', N'America/Chihuahua', N'America/Coral_Harbour', N'America/Cordoba', N'America/Costa_Rica', N'America/Creston', N'America/Cuiaba', N'America/Curacao', N'America/Danmarkshavn', N'America/Dawson', N'America/Dawson_Creek', N'America/Denver', N'America/Detroit', N'America/Dominica', N'America/Edmonton', N'America/Eirunepe', N'America/El_Salvador', N'America/Ensenada', N'America/Fort_Nelson', N'America/Fort_Wayne', N'America/Fortaleza', N'America/Glace_Bay', N'America/Godthab', N'America/Goose_Bay', N'America/Grand_Turk', N'America/Grenada', N'America/Guadeloupe', N'America/Guatemala', N'America/Guayaquil', N'America/Guyana', N'America/Halifax', N'America/Havana', N'America/Hermosillo', N'America/Indiana/Indianapolis', N'America/Indiana/Knox', N'America/Indiana/Marengo', N'America/Indiana/Petersburg', N'America/Indiana/Tell_City', N'America/Indiana/Vevay', N'America/Indiana/Vincennes', N'America/Indiana/Winamac', N'America/Indianapolis', N'America/Inuvik', N'America/Iqaluit', N'America/Jamaica', N'America/Jujuy', N'America/Juneau', N'America/Kentucky/Louisville', N'America/Kentucky/Monticello', N'America/Knox_IN', N'America/Kralendijk', N'America/La_Paz', N'America/Lima', N'America/Los_Angeles', N'America/Louisville', N'America/Lower_Princes', N'America/Maceio', N'America/Managua', N'America/Manaus', N'America/Marigot', N'America/Martinique', N'America/Matamoros', N'America/Mazatlan', N'America/Mendoza', N'America/Menominee', N'America/Merida', N'America/Metlakatla', N'America/Mexico_City', N'America/Miquelon', N'America/Moncton', N'America/Monterrey', N'America/Montevideo', N'America/Montreal', N'America/Montserrat', N'America/Nassau', N'America/New_York', N'America/Nipigon', N'America/Nome', N'America/Noronha', N'America/North_Dakota/Beulah', N'America/North_Dakota/Center', N'America/North_Dakota/New_Salem', N'America/Ojinaga', N'America/Panama', N'America/Pangnirtung', N'America/Paramaribo', N'America/Phoenix', N'America/Port-au-Prince', N'America/Port_of_Spain', N'America/Porto_Acre', N'America/Porto_Velho', N'America/Puerto_Rico', N'America/Punta_Arenas', N'America/Rainy_River', N'America/Rankin_Inlet', N'America/Recife', N'America/Regina', N'America/Resolute', N'America/Rio_Branco', N'America/Rosario', N'America/Santa_Isabel', N'America/Santarem', N'America/Santiago', N'America/Santo_Domingo', N'America/Sao_Paulo', N'America/Scoresbysund', N'America/Shiprock', N'America/Sitka', N'America/St_Barthelemy', N'America/St_Johns', N'America/St_Kitts', N'America/St_Lucia', N'America/St_Thomas', N'America/St_Vincent', N'America/Swift_Current', N'America/Tegucigalpa', N'America/Thule', N'America/Thunder_Bay', N'America/Tijuana', N'America/Toronto', N'America/Tortola', N'America/Vancouver', N'America/Virgin', N'America/Whitehorse', N'America/Winnipeg', N'America/Yakutat', N'America/Yellowknife', N'Antarctica/Casey', N'Antarctica/Davis', N'Antarctica/DumontDUrville', N'Antarctica/Macquarie', N'Antarctica/Mawson', N'Antarctica/McMurdo', N'Antarctica/Palmer', N'Antarctica/Rothera', N'Antarctica/South_Pole', N'Antarctica/Syowa', N'Antarctica/Troll', N'Antarctica/Vostok', N'Arctic/Longyearbyen', N'Asia/Aden', N'Asia/Almaty', N'Asia/Amman', N'Asia/Anadyr', N'Asia/Aqtau', N'Asia/Aqtobe', N'Asia/Ashgabat', N'Asia/Ashkhabad', N'Asia/Atyrau', N'Asia/Baghdad', N'Asia/Bahrain', N'Asia/Baku', N'Asia/Bangkok', N'Asia/Barnaul', N'Asia/Beirut', N'Asia/Bishkek', N'Asia/Brunei', N'Asia/Calcutta', N'Asia/Chita', N'Asia/Choibalsan', N'Asia/Chongqing', N'Asia/Chungking', N'Asia/Colombo', N'Asia/Dacca', N'Asia/Damascus', N'Asia/Dhaka', N'Asia/Dili', N'Asia/Dubai', N'Asia/Dushanbe', N'Asia/Famagusta', N'Asia/Gaza', N'Asia/Harbin', N'Asia/Hebron', N'Asia/Ho_Chi_Minh', N'Asia/Hong_Kong', N'Asia/Hovd', N'Asia/Irkutsk', N'Asia/Istanbul', N'Asia/Jakarta', N'Asia/Jayapura', N'Asia/Jerusalem', N'Asia/Kabul', N'Asia/Kamchatka', N'Asia/Karachi', N'Asia/Kashgar', N'Asia/Kathmandu', N'Asia/Katmandu', N'Asia/Khandyga', N'Asia/Kolkata', N'Asia/Krasnoyarsk', N'Asia/Kuala_Lumpur', N'Asia/Kuching', N'Asia/Kuwait', N'Asia/Macao', N'Asia/Macau', N'Asia/Magadan', N'Asia/Makassar', N'Asia/Manila', N'Asia/Muscat', N'Asia/Nicosia', N'Asia/Novokuznetsk', N'Asia/Novosibirsk', N'Asia/Omsk', N'Asia/Oral', N'Asia/Phnom_Penh', N'Asia/Pontianak', N'Asia/Pyongyang', N'Asia/Qatar', N'Asia/Qyzylorda', N'Asia/Rangoon', N'Asia/Riyadh', N'Asia/Saigon', N'Asia/Sakhalin', N'Asia/Samarkand', N'Asia/Seoul', N'Asia/Shanghai', N'Asia/Singapore', N'Asia/Srednekolymsk', N'Asia/Taipei', N'Asia/Tashkent', N'Asia/Tbilisi', N'Asia/Tehran', N'Asia/Tel_Aviv', N'Asia/Thimbu', N'Asia/Thimphu', N'Asia/Tokyo', N'Asia/Tomsk', N'Asia/Ujung_Pandang', N'Asia/Ulaanbaatar', N'Asia/Ulan_Bator', N'Asia/Urumqi', N'Asia/Ust-Nera', N'Asia/Vientiane', N'Asia/Vladivostok', N'Asia/Yakutsk', N'Asia/Yangon', N'Asia/Yekaterinburg', N'Asia/Yerevan', N'Atlantic/Azores', N'Atlantic/Bermuda', N'Atlantic/Canary', N'Atlantic/Cape_Verde', N'Atlantic/Faeroe', N'Atlantic/Faroe', N'Atlantic/Jan_Mayen', N'Atlantic/Madeira', N'Atlantic/Reykjavik', N'Atlantic/South_Georgia', N'Atlantic/St_Helena', N'Atlantic/Stanley', N'Australia/ACT', N'Australia/Adelaide', N'Australia/Brisbane', N'Australia/Broken_Hill', N'Australia/Canberra', N'Australia/Currie', N'Australia/Darwin', N'Australia/Eucla', N'Australia/Hobart', N'Australia/LHI', N'Australia/Lindeman', N'Australia/Lord_Howe', N'Australia/Melbourne', N'Australia/NSW', N'Australia/North', N'Australia/Perth', N'Australia/Queensland', N'Australia/South', N'Australia/Sydney', N'Australia/Tasmania', N'Australia/Victoria', N'Australia/West', N'Australia/Yancowinna', N'Brazil/Acre', N'Brazil/DeNoronha', N'Brazil/East', N'Brazil/West', N'CET', N'CST6CDT', N'Canada/Atlantic', N'Canada/Central', N'Canada/Eastern', N'Canada/Mountain', N'Canada/Newfoundland', N'Canada/Pacific', N'Canada/Saskatchewan', N'Canada/Yukon', N'Chile/Continental', N'Chile/EasterIsland', N'Cuba', N'EET', N'EST', N'EST5EDT', N'Egypt', N'Eire', N'Etc/GMT', N'Etc/GMT+0', N'Etc/GMT+1', N'Etc/GMT+10', N'Etc/GMT+11', N'Etc/GMT+12', N'Etc/GMT+2', N'Etc/GMT+3', N'Etc/GMT+4', N'Etc/GMT+5', N'Etc/GMT+6', N'Etc/GMT+7', N'Etc/GMT+8', N'Etc/GMT+9', N'Etc/GMT-0', N'Etc/GMT-1', N'Etc/GMT-10', N'Etc/GMT-11', N'Etc/GMT-12', N'Etc/GMT-13', N'Etc/GMT-14', N'Etc/GMT-2', N'Etc/GMT-3', N'Etc/GMT-4', N'Etc/GMT-5', N'Etc/GMT-6', N'Etc/GMT-7', N'Etc/GMT-8', N'Etc/GMT-9', N'Etc/GMT0', N'Etc/Greenwich', N'Etc/UCT', N'Etc/UTC', N'Etc/Universal', N'Etc/Zulu', N'Europe/Amsterdam', N'Europe/Andorra', N'Europe/Astrakhan', N'Europe/Athens', N'Europe/Belfast', N'Europe/Belgrade', N'Europe/Berlin', N'Europe/Bratislava', N'Europe/Brussels', N'Europe/Bucharest', N'Europe/Budapest', N'Europe/Busingen', N'Europe/Chisinau', N'Europe/Copenhagen', N'Europe/Dublin', N'Europe/Gibraltar', N'Europe/Guernsey', N'Europe/Helsinki', N'Europe/Isle_of_Man', N'Europe/Istanbul', N'Europe/Jersey', N'Europe/Kaliningrad', N'Europe/Kiev', N'Europe/Kirov', N'Europe/Lisbon', N'Europe/Ljubljana', N'Europe/London', N'Europe/Luxembourg', N'Europe/Madrid', N'Europe/Malta', N'Europe/Mariehamn', N'Europe/Minsk', N'Europe/Monaco', N'Europe/Moscow', N'Europe/Nicosia', N'Europe/Oslo', N'Europe/Paris', N'Europe/Podgorica', N'Europe/Prague', N'Europe/Riga', N'Europe/Rome', N'Europe/Samara', N'Europe/San_Marino', N'Europe/Sarajevo', N'Europe/Saratov', N'Europe/Simferopol', N'Europe/Skopje', N'Europe/Sofia', N'Europe/Stockholm', N'Europe/Tallinn', N'Europe/Tirane', N'Europe/Tiraspol', N'Europe/Ulyanovsk', N'Europe/Uzhgorod', N'Europe/Vaduz', N'Europe/Vatican', N'Europe/Vienna', N'Europe/Vilnius', N'Europe/Volgograd', N'Europe/Warsaw', N'Europe/Zagreb', N'Europe/Zaporozhye', N'Europe/Zurich', N'GB', N'GB-Eire', N'GMT', N'GMT+0', N'GMT-0', N'GMT0', N'Greenwich', N'HST', N'Hongkong', N'Iceland', N'Indian/Antananarivo', N'Indian/Chagos', N'Indian/Christmas', N'Indian/Cocos', N'Indian/Comoro', N'Indian/Kerguelen', N'Indian/Mahe', N'Indian/Maldives', N'Indian/Mauritius', N'Indian/Mayotte', N'Indian/Reunion', N'Iran', N'Israel', N'Jamaica', N'Japan', N'Kwajalein', N'Libya', N'MET', N'MST', N'MST7MDT', N'Mexico/BajaNorte', N'Mexico/BajaSur', N'Mexico/General', N'NZ', N'NZ-CHAT', N'Navajo', N'PRC', N'PST8PDT', N'Pacific/Apia', N'Pacific/Auckland', N'Pacific/Bougainville', N'Pacific/Chatham', N'Pacific/Chuuk', N'Pacific/Easter', N'Pacific/Efate', N'Pacific/Enderbury', N'Pacific/Fakaofo', N'Pacific/Fiji', N'Pacific/Funafuti', N'Pacific/Galapagos', N'Pacific/Gambier', N'Pacific/Guadalcanal', N'Pacific/Guam', N'Pacific/Honolulu', N'Pacific/Johnston', N'Pacific/Kiritimati', N'Pacific/Kosrae', N'Pacific/Kwajalein', N'Pacific/Majuro', N'Pacific/Marquesas', N'Pacific/Midway', N'Pacific/Nauru', N'Pacific/Niue', N'Pacific/Norfolk', N'Pacific/Noumea', N'Pacific/Pago_Pago', N'Pacific/Palau', N'Pacific/Pitcairn', N'Pacific/Pohnpei', N'Pacific/Ponape', N'Pacific/Port_Moresby', N'Pacific/Rarotonga', N'Pacific/Saipan', N'Pacific/Samoa', N'Pacific/Tahiti', N'Pacific/Tarawa', N'Pacific/Tongatapu', N'Pacific/Truk', N'Pacific/Wake', N'Pacific/Wallis', N'Pacific/Yap', N'Poland', N'Portugal', N'ROC', N'ROK', N'Singapore', N'Turkey', N'UCT', N'US/Alaska', N'US/Aleutian', N'US/Arizona', N'US/Central', N'US/East-Indiana', N'US/Eastern', N'US/Hawaii', N'US/Indiana-Starke', N'US/Michigan', N'US/Mountain', N'US/Pacific', N'US/Pacific-New', N'US/Samoa', N'UTC', N'Universal', N'W-SU', N'WET', N'Zulu')), [clipTemplateId] NVARCHAR(255) NULL, [shopId] NVARCHAR(255) NULL, [createdAt] DATETIMEOFFSET NOT NULL, [updatedAt] DATETIMEOFFSET NOT NULL, PRIMARY KEY ([id]), FOREIGN KEY ([clipTemplateId]) REFERENCES [ClipTemplate] ([id]) ON DELETE NO ACTION, FOREIGN KEY ([shopId]) REFERENCES [Shop] ([id]) ON DELETE NO ACTION);
Same format as the other queries, except it's got a list of timezones. I can see the previous queries in the audit log, but not this one.
I've run SELECT OBJECT_ID('[Client]', 'U') and it returns 1218103380, so the query shouldn't actually be doing anything, given the IF at the top?
Reproducable every time against multiple Azure SQL databases in different development environments with the node app. Other SQL clients send the query with success. App last worked yesterday. No app or infra changes since.
When putting the query into some online SQL validators like https://www.eversql.com/sql-syntax-check-validator/, it says the query is invalid, but MS SQL Server Management Studio thinks it's fine, and I can run it without issue.
Sequelize Config:
database: {
database: process.env.DATABASE_NAME,
dialect: "mssql",
host: process.env.DATABASE_HOST,
loggingEnabled: false,
password: process.env.DATABASE_PASSWORD,
pool: {
acquire: 30000,
idle: 30000,
max: 10,
min: 0,
},
port: parseInt(process.env.DATABASE_PORT, 10) || 1433,
transactionTimeout: parseInt(process.env.DB_TRANSACTION_TIMEOUT, 10) || 30000,
username: process.env.DATABASE_USERNAME,
}
It seems 10.17.0 broke my DB connection for long queries. Rolling back to Node v10.16.3 has resolved the issue.
On start up, my Node program was pulling the latest 10.x version. It started using Node v10.17.0, released 21st October 2019, which was when the issue started.
Here is a solution for timeout error while running long queries.
connection timeout or already too many clients

Not able to replicate CouchDB databases with admin and self signed SSL certificate

My local and remote CouchDB SSL server configuration
cert_file /etc/couchdb/cert/couchdb.pem
key_file /etc/couchdb/cert/privkey.pem
port 6984
ssl_certificate_max_depth 1
verify_ssl_certificates false
Whenever i create a document in _replicator database of my local couch db like this
{
"_id": "181361e45e43f9363612b9dec1005a78",
"_rev": "6-87e9881fd3fecb26e906bb38b8e00a2f",
"source": "https://adminname:pass#localhostName:6984/DatabaseFromReplicate/",
"target": "https://adminname:pass#remotehostName:6984/DatabaseToReplicate/",
"continuous": true,
"owner": "Admin Name"
}
My document gets revised to
{
"_id": "181361e45e43f9363612b9dec1005a78",
"_rev": "7-87e9881fd3fecb26e906bb38b8e00a2f",
"source": "https://adminname:pass#localhostName:6984/DatabaseFromReplicate/",
"target": "https://adminname:pass#remotehostName:6984/DatabaseToReplicate/",
"continuous": true,
"owner": "Admin Name",
"_replication_state": "error",
"_replication_state_time": "2018-05-02T07:23:07-06:00",
"_replication_state_reason": "timeout",
"_replication_id": "f236397473a906778fd8edbe63dbe59d"
}
DatabaseFromReplicate is present at local couchDb and DatabaseToReplicate is present at Remote couchDb
My CouchDB log shows:
Retrying HEAD request to https://admin:*****#remotehost:6984/DatabaseToReplicate/ in 0.25 seconds due to error {conn_failed,{error,closed}}
Update:
If the document created in _replicator database target key value is changed from https to http and port number from 6984 to 5984 then the replication starts in my remote database on https with 6984 port why is that.
My couchdb config for local and remote:
For replicator
connection_timeout 30000
db _replicator
http_connections 20
max_replication_retry_count 10
retries_per_request 10
socket_options [{keepalive, true}, {nodelay, false}]
ssl_certificate_max_depth 3
verify_ssl_certificates false
worker_batch_size 500
worker_processes 4
For httpd
allow_jsonp false
authentication_handlers {couch_httpd_oauth, oauth_authentication_handler}, {couch_httpd_auth, cookie_authentication_handler}, {couch_httpd_auth, default_authentication_handler}
bind_address 0.0.0.0
default_handler {couch_httpd_db, handle_request}
enable_cors false
log_max_chunk_size 1000000
port 5984
secure_rewrites true
socket_options [{recbuf, 262144}, {sndbuf, 262144}]
vhost_global_handlers _utils, _uuids, _session, _oauth, _users
For couch_httpd_auth
allow_persistent_cookies false
auth_cache_size 50
authentication_db _users
authentication_redirect /_utils/session.html
iterations 10
require_valid_user true
secret ***************************
timeout 600
For couch_httpd_oauth
use_users_db false
For ssl
cert_file /etc/couchdb/cert/couchdb.pem
key_file /etc/couchdb/cert/privkey.pem
port 6984
ssl_certificate_max_depth 1
verify_ssl_certificates false
My _users database just contains the design document.
My couchDb version is 1.6.1
In the document you put into /_replicator you have:
"source": "https://adminname:pass#host:6984/DatabaseFromReplicate/",
"target": "https://adminname:pass#host:6984/DatabaseToReplicate/",
For both source and target you're using host, shouldn't it be something like localHost for source and remoteHost for target? I mean, shouldn't they be different?
I'm not sure, maybe that's just a typo in your question.

Resources