Remove string in file on linux - linux

I am looking to remove the _id field and its value which changes:
{ "_id" : { "$oid" : "54da1bee58743hd23947f493" }, "name":"david", "age":"33"}
{ "_id" : { "$oid" : "5422222222222345d9f1f493" }, "name":"Dove", "age":"33"}
{ "_id" : { "$oid" : "54da1be57a4b727669f1f493" }, "name":"man", "age":"23"}
outcome:
{"name":"david","age":"33"}
{"name":"Dove", "age":"33"}
{"name":"man", "age":"23"}
I would like to use sed or any other command.

It is easy with jq.
jq -c 'del(._id)' input.txt
O/P:
{"name":"david","age":"33"}
{"name":"Dove","age":"33"}
{"name":"man","age":"23"}

Related

$merge, $match and $update in one aggregate query

I have data in a collection ex:"jobs". I am trying to copy specific data from "jobs" after every 2 hours to a new collection (which may not exist initially) and also add a new key to the copied data.
I have been trying with this query to copy the data:
db.getCollection("jobs").aggregate([{ $match: { "job_name": "UploadFile", "created_datetime" : {"$gte":"2021-08-18 12:00:00"} } },{"$merge":{into: {coll : "reports"}}}])
But after this, the count in "reports" collection is 0. Also, how can I update the documents (with an extract key "report_name") without using an extra updateMany() query?
The data in jobs collection is as shown:
{
"_id" : ObjectId("60fa8e8283dc22799134dc6f"),
"job_id" : "408a5654-9a89-4c15-82b4-b0dc894b19d7",
"job_name" : "UploadFile",
"data" : {
"path" : "share://LOCALNAS/Screenshot from 2021-07-23 10-34-34.png",
"file_name" : "Screenshot from 2021-07-23 10-34-34.png",
"parent_path" : "share://LOCALNAS",
"size" : 97710,
"md5sum" : "",
"file_uid" : "c4411f10-a745-48d0-a55d-164707b7d6c2",
"version_id" : "c3dfd31a-80ba-4de0-9115-2d9b778bcf02",
"session_id" : "c4411f10-a745-48d0-a55d-164707b7d6c2",
"resource_name" : "Screenshot from 2021-07-23 10-34-34.png",
"metadata" : {
"metadata" : {
"description" : "",
"tag_ids" : [ ]
},
"category_id" : "60eed9ea33c690a0dfc89b41",
"custom_metadata" : [ ]
},
"upload_token" : "upload_token_c5043927484e",
"upload_url" : "/mnt/share_LOCALNAS",
"vfs_action_handler_id" : "91be4282a9ad5067642cdadb75278230",
"element_type" : "file"
},
"user_id" : "60f6c507d4ba6ee28aee5723",
"node_id" : "syeda",
"state" : "COMPLETED",
"priority" : 2,
"resource_name" : "Screenshot from 2021-07-23 10-34-34.png",
"group_id" : "upload_group_0babf8b7ce0b",
"status_info" : {
"progress" : 100,
"status_msg" : "Upload Completed."
},
"error_code" : "",
"error_message" : "",
"created_datetime" : ISODate("2021-07-23T15:10:18.506Z"),
"modified_datetime" : ISODate("2021-07-23T15:10:18.506Z"),
"schema_version" : "1.0.0",
}
Your $match stage contains a condition which takes created_datetime as string while in your sample data it is an ISODate. Such condtion won't return any document, try:
{
$match: {
"job_name": "UploadFile",
"created_datetime": {
"$gte": ISODate("2021-07-01T12:00:00.000Z")
}
}
}
Mongo Playground

$add,$subtract aggregation-framework in mongodb

Hi i am mentioning the sample data
///collection - test////
{
"_id" : {
"date" : ISODate("2020-02-11T17:00:00Z"),
"userId" : ObjectId("5e43e5cdc11f750864f46820"),
"adminId" : ObjectId("5e43de778b57693cd46859eb")
},
"outstanding" : 212.39999999999998,
"totalBill" : 342.4,
"totalPayment" : 130
}
{
"_id" : {
"date" : ISODate("2020-02-11T17:00:00Z"),
"userId" : ObjectId("5e43e73169fe1e3fc07eb7c5"),
"adminId" : ObjectId("5e43de778b57693cd46859eb")
},
"outstanding" : 797.8399999999999,
"totalBill" : 797.8399999999999,
"totalPayment" : 0
}
I need to structure a query which does following things-
I need to calculate the actualOutstanding:[(totalBill+outstanding)-totalPayment],
I need to save this actualOutstanding in the same collection & in the same document according to {"_id" : {"date","userId", "adminId" }}
NOTE: userId is different in both the documents.
Introduced in Mongo version 4.2+ pipelined updates, meaning we can now use aggregate expressions to update documents.
db.collection.updateOne(
{
"adminId" : ObjectId("5e43de778b57693cd46859eb")
'_id."userId" : ObjectId("5e43e73169fe1e3fc07eb7c5"),
'_id.date': ISODate("2020-02-11T18:30:00Z"),
},
[
{ '$set': {
actualOutstanding: {
$subtract:[ {$add: ['$totalBill','$outstanding']},'$totalPayment']
}
} }
]);
For any other Mongo version you have to split it into 2 actions, first query and calculate then update the document with the calculation.

How to query data from mongodb collection by just the first 3 numbers

I have a collection in my mongodb that is called pnumber and looks like this when I do db.pnumber.find() mongo shell
{ "_id" : ObjectId("57696e03b9c614e1143c759e"), "pnumber" : NumberLong("5037816575") }
{ "_id" : ObjectId("57696e03b9c614e1143c759f"), "pnumber" : NumberLong("5087368074") }
{ "_id" : ObjectId("57696e03b9c614e1143c75a0"), "pnumber" : NumberLong("8055019263") }
{ "_id" : ObjectId("57696e03b9c614e1143c75a1"), "pnumber" : NumberLong("2153334813") }
{ "_id" : ObjectId("57696e03b9c614e1143c75a2"), "pnumber" : NumberLong("5592842086") }
{ "_id" : ObjectId("57696e03b9c614e1143c75a3"), "pnumber" : NumberLong("4356696916") }
{ "_id" : ObjectId("57696e03b9c614e1143c75a4"), "pnumber" : NumberLong("9545935012") }
{ "_id" : ObjectId("57696e03b9c614e1143c75a5"), "pnumber" : NumberLong("8083661362") }
{ "_id" : ObjectId("57696e03b9c614e1143c75a6"), "pnumber" : NumberLong("6065240853") }
{ "_id" : ObjectId("57696e03b9c614e1143c75a7"), "pnumber" : NumberLong("4236768328") }
{ "_id" : ObjectId("57696e03b9c614e1143c75a8"), "pnumber" : NumberLong("9705354839") }
I am using a mean stack application that is connected to this database through mongoose.
I would like to have an input field with a search button that when a user enters the area code that a list pops up from the collection of all the entry's that match that area code and have a download button that creates a csv with all of those entries.
I have:
var csv = require('express-csv');
in my app.js file.
Any info would be much appreciated.
db.pnumber.find( { pnumber: /^123/ } )
this results with all response that start with the value 123.
if you tie the form with this, the response should be the results you are looking for.
if you would like to match strings, use $regex operator
db.getCollection('blah').find({'phoneNumber':{'$regex': /^phoneNumber/}})
if you would like to match numbers, use $where operator.
db.getCollection('blah').find({$where: "/^970*/.test(this.phoneNumber)"})

How to combine Elasticsearch highlighting and query types?

If I do a search like:
{
"query" : {
"bool" : {
"should" : [
{"match" : { "FullName" : "MiddleName1" } },
{"match_phrase" : { "FullName" : "FirstName2 LastName2" } }
]
}
}
}
I would get documents like:
{
...
"_id": "1",
...
"FullName" : "FirstName1 MiddleName1 LastName1",
...
}
{
...
"_id": "2"
...
"FullName" : "FirstName2 LastName2",
...
}
I know highlighting shows what parts of the documents caused retrievals, but not what query type.
I can also do a multi query then merge the results.
Is there a way do a single query and find out what query type fields and tokens that caused certain documents to be retrieved?
Ideal result:
{
"id" : "1",
"clause" : "match",
"field" : "FullName",
"tokens" : ["MiddleName1"]
}
{
"id" : "2",
"clause" : "match_phrase",
"field" : "FullName",
"tokens" : ["FirstName2", "LastName2"]
}

MongoDB remove the lowest score, node.js

I am trying to remove the lowest homework score.
I tried this,
var a = db.students.find({"scores.type":"homework"}, {"scores.$":1}).sort({"scores.score":1})
but how can I remove this set of data?
I have 200 pieces of similar data below.
{
"_id" : 148,
"name" : "Carli Belvins",
"scores" : [
{
"type" : "exam",
"score" : 84.4361816750119
},
{
"type" : "quiz",
"score" : 1.702113040528119
},
{
"type" : "homework",
"score" : 22.47397850465176
},
{
"type" : "homework",
"score" : 88.48032660881387
}
]
}
you are trying to remove an element but the statement you provided is just to find it.
Use db.students.remove(<query>) instead. Full documentation here

Resources