I have a database and I want to truncate all records, I know it is possible to just add a _deleted key to every document or call db.delete() on CouchDB-python library. I am using the delete of couchdb-python but it does not seem to work when I fetch all the documents and then call .delete on each document excluding design documents.
Here is my code.
docs = get_db().view('_all_docs', include_docs=True)
for i in docs:
if not(i['id'].startswith('_')):
get_db().delete(i)
This is the error. Because the result from _all_docs is returning a id instead _id.
File "C:\Users\User\AppData\Local\Programs\Python\Python36-32\lib\site-packages\couchdb\client.py", line 625, in delete
if doc['_id'] is None:
KeyError: '_id'
My question is how do I fetch all documents that returns _id instead of just the id? Or is there any way around this?
In couchdb-python a view query returns a list of couchdb.client.Row objects, not a list of the docs. You need to pass an attribute doc to that delete, i.e. get_db().delete(i['doc']).
From performance perspective, however, it's better to use bulk api. With couchdb-python it should look something like this:
rows = get_db().view('_all_docs', include_docs=True)
docs = []
for row in rows:
if row['id'].startswith('_'):
continue
doc = row['doc']
doc['_deleted'] = True
docs.append(doc)
get_db().update(docs)
Deleting documents from CouchDB you can create in two step:
create a view (with filtering the documents you want to delete)
use the view to delete all documents using the view
I have written a tool for this.
Related
I'm just new to ArangoDB, so could you give me any tips how I can perform the following update?
I have the document collection, each document has a attribute seen. I want to update the existing collection from a csv-file in the following way: if in the file there is a line with _key that already is in the collection, I want to sum the seen value from csv-file and from the collection and replace the value in the collection by the sum; if there is no document with such _key, I just want to add it.
As far as I know, it is a little bit too much for the arangoimp tool, as it has an option either to replace or ignore dublicates.
How would you do that?
I would be grateful for any ideas.
As you say, it's too much to ask from the arangoimp tool. You could use it to update existing records via _key and replace the seen attribute and create new documents it the _key does not exist yet. But it doesn't support to add logic that would sum up the seen values.
However, you can import your CSV with arangoimp to a temporary collection and use an AQL query to do that. Let temp be that temporary collection and coll your main collection:
FOR doc IN temp
UPSERT { _key: doc._key }
INSERT doc
UPDATE { seen: OLD.seen + doc.seen }
IN coll
You could optionally REPLACE instead of UPDATE or MERGE() the existing document with attributes from the imported document if needed, or remove the temp documents at the end.
In my app I need a function that removes docs and giving me the removed doc fields. The docs for the driver are not clear about the callback of deleteMany. It returns a result that is deleteWriteOpResultObject. I don't know how to get the deleted doc fields from that.
You may use findAndModify operation with remove: true and fields: {your_fields} if you need to get and delete a single document. In case you have to remove and select multiple records, you may execute findAndModify multiple times or call find, wait the results and remove them all by IDs.
I want to delete all documents where foo equals x. Seems like a pretty basic operation, but I just can't figure it out.
I know how to delete an individual document, but that's not good enough - I may have to delete a few thousand at a time.
How do I bulk delete documents in CouchDB?
I don't know if it's the right way but make a view that exposes the foo field, query the view for the doc._ids of all your documents that you want to delete, and make a bulk update against all your documents. So two (ideally) calls to couch.
http://comments.gmane.org/gmane.comp.db.couchdb.user/11222
Has a similar way to go about it.
Do a bulk update on all the documents you want to delete and update doc._deleted=true following the example in Bulk deletion of documents
It's quite easy with bulk delete: https://wiki.apache.org/couchdb/HTTP_Bulk_Document_API
Just POST to _all_docs a list of JSONs that look like:
{"_id": "0", "_rev": "1-62657917", "_deleted": true}
I also needed something to handle that and, since there was nothing at the time, I decided to make my own implementation.
You can find it here.
Update
Since it was very helpful to me and in order to protect myself from mistakes, I added a backup/restore feature to this tool that can now be found on version 0.2
I tried a somewhat long method to delete documents. I first created a view called map_fun that called the documents i wanted to get deleted. I then iterated through the view and stored the keys of allt he documents and used del db['_id'] to delete them
map_fun = function(doc){
if (doc.doc_type == 'classic'){
emit(doc._id, doc)
}}
deldoclist = []
for row in db.query(map_fun):
deldoclist.append(row.key)
for item in deldoclist:
del db[item]
I have a Tags collection which contains documents of the following structure:
{
word:"movie", //tag word
count:1 //count of times tag word has been used
}
I am given an array of new tags that need to be added/updated in the Tags collection:
["music","movie","book"]
I can update the counts all Tags currently existing in the tags collection by using the following query:
db.Tags.update({word:{$in:["music","movies","books"]}}, {$inc:{count:1}}), true, true);
While this is an effective strategy to update, I am unable to see which tag values were not found in the collection, and setting the upsert flag to true did not create new documents for the unfound tags.
This is where I am stuck, how should I handle the bulk insert of "new" values into the Tags collection?
Is there any other way I could better utilize the update so that it does upsert the new tag values?
(Note: I am using Node.js with mongoose, solutions using mongoose/node-mongo-native would be nice but not necessary)
Thanks ahead
The concept of using upsert and the $in operator simultaneously is incongruous. This simply will not work as there is no way to different between upsert if *any* in and upsert if *none* in.
In this case, MongoDB is doing the version you don't want it to do. But you can't make it change behaviour.
I would suggest simply issuing three consecutive writes by looping through the array of tags. I know that's it's annoying and it has a bad code smell, but that's just how MongoDB works.
How to get Post with Comments Count in single query with CouchDB?
I can use map-reduce to build standalone view [{key: post_id, value: comments_count}] but then I had to hit DB twice - one query to get the post, another to get comments_count.
There's also another way (Rails does this) - count comments manually, on the application server and save it in comment_count attribute of the post. But then we need to update the whole post document every time a new comment added or deleted.
It seems to me that CouchDB is not tuned for such a way, unlike RDBMS when we can update only the comment_count attribute in CouchDB we are forced to update the whole post document.
Maybe there's another way to do it?
Thanks.
The view's return json includes the document count as 'total_rows', so you don't need to compute anything yourself, just emit all the documents you want counted.
{"total_rows":3,"offset":0,"rows":[
{"id":...,"key":...,value:doc1},
{"id":...,"key":...,value:doc2},
{"id":...,"key":...,value:doc3}]
}