I already have my Solr configured and running in the port 8983. Initially i indexed all the data in the MongoDB using data import handler. But now I realized that for every update and new insertion to automatically index we need a Mongo connector . I followed these links: Mongo Connector and Usage-with-Solr.
I am getting struck at the point
python mongo_connector.py -m localhost:27017 -t http://localhost:8983/solr
It shows error
python: can't open file 'mongo_connector.py': [Errno 2] No such file or directory
how to integrate a collection named food with the mongodb collection testfood such that new insertions and update automatically reflects in solr.
Related
I had a node application running on EC2 managed by Elastic Beanstalk.
The elastic beanstalk removed an instance and recreated new one due to some issue.
I had mongo store its db in a separate Elastic Block Store volume, and did re-attach the volume, mounted etc..
However when I tried to start mongodb using systemctl, I got various errors.
I tried --repair, chown the data directory to mongod and it finally worked, but now the user db was gone and the application re-created it and all collections are empty, but I do see large collection-x-xxxxxxx.wt and index-x-xxxxx.wt files in the data directory.
What am I doing wrong ?
Is there any way to recover the data.
PS: I did try the --repair before I saw the warning about how it would remove all corrupted data
I was able to restore from the collection-x-xxxxx.wt files....
In my 'corrupted' mongo data directory, there was a WiredTiger.wt.orig file and WiredTiger.wt file.
If I try to start mongod by removing the 'orig' extension, mongod will not start and start showing errors like WiredTiger.wt read error: failed to read 4096 bytes at offset 73728: WT_ERROR: non-specific WiredTiger error.
Searching for restoring a corrupted 'WiredTiger' file, I came across this medium article about repairing MongoDB after a corrupted WiredTiger file.
Steps from article as I followed them :
Stop mongod. (the one with empty collections)
Point mongod to a new data directory
Start mongod and create new db, and new collections with the same names as the ones from corrupted mongo db.
Insert atleast one dummy record into each of these collection.
Find the names of the collection*.wt files in this new location using db.<insert-collectionName>.stats(), look in the uri property of the output.
Stop mongod.
Copy over collection-x-xxxxx.wt from corrupted directory to the new directory and rename them to the corresponding ones from step 5.
7.1. i.e. Say, if your collection named 'testCollection' had the wt collection file name as collection-1-1111111.wt in corrupted directory and the name as collection-6.6666666.wt in new directory, you will have to copy the 'collection-1-1111111.wt' into the new directory and rename it to collection-6.6666666.wt
7.2. To find the collection wt file name of say 'testCollection', you can open the collection-x.xxxx.wt files in a text editor and scroll past the 'gibberish' to see your actual data matching the ones from 'testCollection'. (mine is not encrypted at rest).
Repeat the copy - rename step for all collections you have.
Run repair in new db path with --repair switch, you can see mongo fixing stuff in logs.
Start db.
Once done, verify the collections, mongodump from new db and mongorestore to fresh db and recreate indexes.
That article was a god sent, I cant believe it worked. Thank you Ido Ozeri from Medium
Use-case: I am trying to write data from a nodejs process running locally (on a docker container) to my locally running postgres server (no docker container). The nodejs process is able to connect to the server (setting the address to host.docker.internal solved that problem) however, when I attempt a simple "SELECT * FROM contact LIMIT 1" query, this error is returned:
{"type":"postgres error","request":"SELECT * FROM contact",
"error":{
"name":"error","length":106,
"severity":"ERROR",
"code":"42P01",
"position":"15",
"file":"parse_relation.c",
"line":"1376",
"routine":"parserOpenTable"}}
The relation error suggests the table is not found-- I created this table using a postgres client (postico) and have been able to successfully query the table's contents with other pg clients as well
I see multiple posts are suggesting running the sequelize db:migrate command, but would this be the right solution here?
I did not create a model nor a migration, and created the table directly in the table. Is there something else I may be overlooking that is producing this error?
I have made a cluster on mongodb website, whitelisted IP adress, added the user and connected my application, but it is not giving me 'Standard URI connection' option (maybe their website got updated) instead it is giving me one with srv. So when I used it in mongoose.connect() function it is giving me error:"Invalid Uri,Must begin with mongodb://"...
I have been working to connect to mongodb for so long but failing every time
You have created cluster on MongoDB Atlas? Right.
If using mongo application:
change nodejs version to 2.12 or later from connect your application screen.
If using mongo shell:
On the "Connect to Cluster0" page, select I have mongo shell installed, then select mongo shell version "3.4 or earlier". You will get the desired format.
I'm trying to connect postgres database from sequelize (node.js).But sequelize throws error like ERROR: database "public" does not exist.
The database url is given below:
postgres://postgres:root#localhost:5432/public
The show db result is given below:
I have modified the database url as follows:
postgres://postgres:root#localhost:5432/postgres, where postgres is valid database.Please find the attached image below:
database and schema in postgresql are not the same object .. you should provide the database name not the schema name
you could get the list of the database using
psql -U pgadmin -l
The postgres database is a default database meant for use by users, utilities and third party applications.
If you don't specify a database while connecting to postgres (using, day, psql), it will connect to postgres DB.
It is possible that you connected to postgres without specifying the DB and created the tables. Hence they were created in postgres DB.
References:
https://www.postgresql.org/docs/15/app-initdb.html
https://stackoverflow.com/a/2411860/12242023
I have elasticsearch(v6.4.0) running on windows 10 machine and also a python(v3.6.0) application with angular 5 frontend and mongodb as backend.I want to use elasticsearch with python so that I can send data from UI to insert into mongodb as well as create index in elasticsearch with same data.How to achieve this?
I succeeded to connect to elasticsearch server from python.But stuck at creating index and querying indexed data.
Please help.
Regards,
Vidyashree
You can do indexing using
es = ElasticSearch()
es.index('indexName', 'type', 'indexQuery').
And searching from:
es.search(index=indexName',doc_type= type,body= searchQuery)