Bunyan logger.info error in Restful API - node.js

I'm working on a Restful API and I'm logging all the important steps with bunyan, including the requests. I'm having two problems with logging:
My first problem is that when I log them, all my objects, instead of appearing like independent objects, appear in the msg field, like strings.
Here is my code to log the requests:
var logger = bunyan.createLogger({
name: 'main',
streams: [{
level: 'info',
path: './logs/requests.log'
}]
});
logRequest = function(request){
logger.info("Request started.", {id: request.id}, {method: request.method});
};
and when I see the request.log file it appears like this (I've just added some tabs to make it more comfortable to see) :
{
"name": "logger",
"hostname": "LLAS",
"pid": 7700,
"level": 30,
"msg":"Request started. { id: '1428527975041:LLAS:7700:i898o4l5:10000'{ method:'post' } ",
"time":"2015-04-08T21:19:35.055Z",
"v":0
}
So my problem is that "msg" field, I want to see the "id" and "method" like other fields instead of a string. I.E. :
{
"name": "logger",
"hostname": "LLAS",
"pid": 7700,
"level": 30,
"msg":"Request started.",
"id": '1428527975041:LLAS:7700:i898o4l5:10000',
"method": 'post',
"time":"2015-04-08T21:19:35.055Z",
"v":0
}
How can I solve my problem?
And my second problem is: When I do more than one log in the same file,it writes the JSON in the same line, instead of a new line, like this:
{"name":"logger",...,"v":0}{"name":"logger",...,"v":0}
Instead of this:
{"name":"logger",...,"v":0}
{"name":"logger",...,"v":0}
And I can't work later with those objects in the same line, also it is hard to read and mantain in that way.
Anyone know why is this happening?

My first problem is that when I log them, all my objects, instead of
appearing like independent objects, appear in the "msg" field, like
strings.
That's because you are passing more than one object. You can only pass one object as the first parameter for logging, all other parameters will be considered as msg. From the bunyan source code:
/**
* The functions below log a record at a specific level.
*
* Usages:
* log.<level>() -> boolean is-trace-enabled
* log.<level>(<Error> err, [<string> msg, ...])
* log.<level>(<string> msg, ...)
* log.<level>(<object> fields, <string> msg, ...)
*
* where <level> is the lowercase version of the log level. E.g.:
*
* log.info()
* ....
*/
So if you pass all your parameters in one object, it will work properly:
var request = { id: "abc", method: "GET" }; // dummy request object
logger.info({id: request.id, method: request.method}, "Request started.");
The result is (pretty printed):
{
"name": "main",
"hostname": "Victors-MacBook-Pro.local",
"pid": 2848,
"level": 30,
"id": "abc",
"method": "GET",
"msg": "Request started.",
"time": "2015-04-08T23:25:37.967Z",
"v": 0
}
And my second problem is: When I do more than one log in the same
file,it writes the JSON in the same line, instead of a new line.
Are you on Windows? If you are, maybe the problem is that bunyan may be using the UNIX style carriage return (\n) and not the Windows style (\r\n). Try using a text editor that supports UNIX style carriage returns (like notepad++ or sublime text, for example).

Related

IBM Bluemix Discovery - query parameter

I have created a Discovery service on my bluemix account. I want to query my documents from a nodejs application.
I have built a query with some aggregation, tested it using the bluemix online tool and it's working well.
Now when I query the collection from my code, whatever my parameters are, I always receive all of my documents with the enriched text and so on. I think I am missing how to send the query attributes to the service (like filters and aggregations).
Here is my code:
var queryParams = {
query:'CHLOE RICHARDS',
return:'title',
count:1,
aggregations:'nested(enriched_text.entities).filter(enriched_text.entities.type:Person).term(enriched_text.entities.text, count:5)'
};
discovery.query({environment_id:that.environment_id, collection_id:that.collection_id, query_options:queryParams }, function(error, data) {
if(error){
console.error(error);
reject(error);
}
else{
console.log(JSON.stringify(data, null, 2));
resolve(data.matching_results);
}
});
And the result is always:
{
"matching_results": 28,
"results": [
{
"id": "fe5e2a38e6cccfbd97dbdd0c33c9c8fd",
"score": 1,
"extracted_metadata": {
"publicationdate": "2016-01-05",
"sha1": "28434b0a7e2a94dd62cabe9b5a82e98766584dd412",
"author": "Richardson, Heather S",
"filename": "whatever.docx",
"file_type": "word",
"title": "no title"
},
"text": "......
Independantly of the value of the query_optionparameter. Can you help me?
EDIT
Instead of the query_options:queryParams, I have used query:"text:CHLOE RICHARDS" and it's working well. Now my problem still remains to find the right parameter format to add the aggregations I want
EDIT 2
So I have looked at IBM's example on Github more carefully, and the parameters are now formatted like this:
const queryParams = {
count: 5,
return: 'title,enrichedTitle.text',
query: '"CHLOE RICHARDS"',
aggregations: [ 'nested(enriched_text.entities).filter(enriched_text.entities.type:Person).term(enriched_text.entities.text, count:5)' ],
environment_id: '1111111111',
collection_id: '11111111111'
};
It works well if I use only the query attribute. Now if I only use the aggregations one, all the documents are sent back as a result (which is understandable) but I have no aggregation part, so I can not access the list of proper name in my documents.
Your query does not look right. I you are going to use query then you will need to construct a query search like text:"CHLOE RICHARDS"
If you want to perform a natural language query then you should be setting the parameter natural_language_query.

Marklogic Node.js API: How to get the document where an embedded triple lives?

I tried to insert the following test document:
db.documents.write(
{
uri: "/test/doc1.json",
contentType: "application/json",
collections: "test",
content: {
name : "Peter",
hobby: "Sleeping",
other: "Some other info",
"triple": {
"subject": {   
"datatype": "http://example.com/name/",  
"value": "Peter"   
},   
"predicate": {     
"datatype": "http://example.com/relation/",  
"value": "livesin"   
},   
"object": {     
"datatype": "http://example.com/location/",  
"value": "Paris"   
}
  }
}
}
).
result(function(response){
console.log("Done loading");
});
Then I queried as follows:
var query = [
'SELECT ?s ?p ?o' ,
'WHERE { ?s ?p ?o }' ,
];
db.graphs.sparql('application/sparql-results+json', query.join('\n')
).result(function (result) {
console.log(JSON.stringify(result, null, 2));
}, function(error) {
console.log(JSON.stringify(error, null, 2));
});
The results showed me the values of the triple, but what if I also want to get the entire document where the triple was embedded? Is it also possible to filter by other fields in the document?
There isn't a way to retrieve the document that contains the result of a SPARQL query, because those results may not be a triple that exists within a particular document (instead, it returns a "solution" consisting of 1 or more values).
If you know you are looking for a particular triple, and you want the document that holds that triple, I would normally say to use a cts:triple-range-query; however, I don't see a way to do that through the Node.js API (or through REST, for that matter). With that in mind, I see two choices:
insert a triple that includes the document's URI as the subject or object, then make a request for that document (as #grtjn suggested)
make a REST API extension (using either JavaScript or XQuery) that calls cts:search with cts:triple-range-query as part of the query; call that extension from Node
I'd recommend doing it in two stages:
Run a sparql that will return document uris.
Run a document search to return those documents, optionally further constrained with extra criteria.
For this you will need to embed triples in your documents listing the document uri of the documents themselves.
HTH!

How telegram bot can get file_id of uploaded file?

In telegram API documentation I see: "You can either pass a file_id as String to resend a photo that is already on the Telegram servers", but I can't find ways to get file_id of uploaded file. How can I get it?
Its depended to your content_types ,for example:
Video:
message.video.file_id
Audio:
message.audio.file_id
Photo:
message.photo[2].file_id
For more see this link.
This is the easiest way I've found to do it.
Upload your file to any chat and forward the message to #RawDataBot. It will return something like this:
{
"update_id": 754677603,
"message": {
"message_id": 403656,
"from": {
"id": xxx,
"is_bot": false,
"first_name": "xxx",
"username": "xxx",
"language_code": "en"
},
"chat": {
"id": xxx,
"first_name": "xxx",
"username": "xxx",
"type": "private"
},
"date": 1589342513,
"forward_from": {
"id": xxx,
"is_bot": false,
"first_name": "xxx",
"username": "xxx",
"language_code": "en"
},
"forward_date": 1589342184,
"document": {
"file_name": "filename.pdf",
"mime_type": "application/pdf",
"file_id": "This_Is_The_Thing_You_Need",
"file_unique_id": "notthis",
"file_size": 123605
}
}
}
What you need is the string under file_id. Once you have copied that, you can simply the following code to send the message.
context.bot.sendDocument(chat_id=update.effective_chat.id,
document = "Your_FILE_ID_HERE")
Depending on the method (File type) which you chose to send a file, after sending a file to Telegram a response is returned. For example if you send a MP3 file to Telegram using sendAudio method, Telegram returns an Audio object which contains the file ID.
Source: https://core.telegram.org/bots/api#audio
In addition to the answers above, you can log Updates that comes to your bot, Either from https://api.telegram.org/bot'.BOT_TOKEN.'/getUpdates or throw updates that come in your application. there you will find a Json property like below:
{
"update_id" = 1111111,
"message" =
{
"message_id" = 1111111,
"from" =
{
"id" = 111111,
...
}
"chat" =
{
"id" = 111111,
...
}
"date" = 111111,
"photo" =
{
{
"file_id" = HERE IS YOU FILE ID 1,
"file_size" => XXXX,
"width" => XX,
"height" => XX,
}
}
}
}
Say you receive a Message with an array of PhotoSize
https://core.telegram.org/bots/api#photosize
As you can see, there's a file_id, you can use this to send a photo through sendPhoto.
If we assume Update is an object, with in it a Message object, which in turn provides a Chat object with in it a id of the chat where the initial message came from and an array of PhotoSize (excuse me for using PHP here, but that's my main language...)
$update->message->photo is how you can access the array.
Use some kind of For loop to iterate over the items, or just access the first one if the array isn't bigger than 1.
After that, you can use the result(s) to extract the file_id and send it as a string via sendPhoto's photo parameter and the Chat ID via the chat_id parameter.
I hope this helped!
P.S. Here is a diagram of my current implementation of the API, i hope it brings some clarity to you!
if you use PHP:
you can write this line for full size:
$file_id = $updates['message']['photo'][1]['file_id'];
and this line for thumb:
$file_id = $updates['message']['photo'][0]['file_id'];
According to the latest docs (v20.0a6) plenty of classes have been changed. I have found that the easiest way to get started with files is using the effective_attachment property.
async def handle_file(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
media_item = await context.bot.get_file(update.message.effective_attachment[0].file_id)
media_url = media_item.file_path
For declaring the handler there have also been changes to filters, here is a simple way to declare it:
application.add_handler(MessageHandler(filters.ATTACHMENT, handle_file))

nodejs async: multiple dependant HTTP API calls

I'm working on a project that involves making multiple HTTP GET requests to different APIs, each requiring information from the last. I'm trying to avoid nested-callaback-and-counter-hell, and have been trying to get it working with the async module.
This is what I need to do: I have an array of 1..n course identifiers (['2014/summer/iat/100/d100', '2014/spring/bisc/372/d100']). For each course in the array, I need to fetch its course outline via a HTTP GET.
The resulting outline looks something like this:
{
"info": {
"nodePath": "2014/spring/bisc/372/d100",
"number": "372",
"section": "D100",
"title": "Special Topics in Biology",
"term": "Spring 2014",
"description": "Selected topics in areas not currently offered...",
"name": "BISC 372 D100",
"dept": "BISC",
},
"instructor": [
{
"lastName": "Smith",
"commonName": "Frank",
"phone": "1 555 555-1234",
"email": "franksmith#school.edu",
"name": "Frank Smith",
"roleCode": "PI"
},
{
"lastName": "Doe",
"commonName": "John",
"phone": "1 555 555-9876",
"email": "johndoe#school.edu",
"name": "John Doe",
"roleCode": "PI"
}
]
}
(a bunch of non-relevant fields omitted)
Each outline object may contain an instructor property which is an array of 0..n instructor objects for the course. For each member of the instructor array, I need to then call another API to get additional data. When that call returns, I need to insert it into the right instructor object.
Finally, when everything is done, the data gets passed to a template for express to render and return to the client.
I've tried getting this working using async and had some success with async.waterfall when doing a proof-of-concept with only getting one of the instructor profiles (e.g. not looping over the array, just getting instructor[0]). The async module's docs are comprehensive, but pretty dense and I'm having a hard time determining what I actually need to do. I had a Frankenstein combination of various nested async calls which still didn't work.
I don't really care how I accomplish the task - flow-control, promises, magic pixie dust, whatever. Any hints greatly appreciated.
Using Q for promises, you can probably do something like this:
return Q
.all(course_ids.map(function(course) {
return HTTP.GET(course); // Assuming this returns a promise
}))
.then(function(course_data) {
var instructors = [];
course_data.forEach(function(course) {
var p = Q
.all(course.instructor.map(function(instructor) {
return HTTP.GET(instructor.id);
}))
.then(function(instructors) {
course.instructors_data = instructors;
return course;
});
promises.push(p);
});
return Q.all(promises);
});
Will resolve with an array containing the courses, each of which contains
an array of instructor data in its instructors_data value.
You could use async.each(), which would do the API requests in parallel (assuming there is no concurrent API request limits on the server side, if that is the case, use async.eachLimit() instead):
async.each(instructors, function(instructor, callback) {
// call API here, store result on `instructor`,
// and call `callback` when done
}, function(err){
if (err)
console.log('An error occurred while processing instructors');
else
console.log('All instructors have been processed successfully');
});

Express.JS url parameters aren't parsed

I have the following route configured
app.put('/v1/users/:uid', function(req, res){
res.send(req.route);
});
When sending a PUT request to http://localhost:3000/v1/users/blablabla
I get the following output back
{
"path": "/v1/users/:uid",
"method": "put",
"callbacks": [
null
],
"keys": [
{
"name": "uid",
"optional": false
}
],
"regexp": {},
"params": []
}
As you see the params array seems to be empty instead of having the value "blablabla". But the "uid" key appears in keys, which I don't really know what to make of.
Would appreciate any suggestions.
OK, the trick is that Express uses a sparse array to parse the params.
When you pass it to req.send, the array is converted with JSON.stringify. Here's what happens in a JS shell:
> var params = [];
> params['uid'] = 1;
> params;
[ uid: 1 ]
> JSON.stringify(params);
'[]'
What's happening is that adding a non-numeric to an array does not change its length:
> params.length
0
So the new value is ignored by JSON.stringify.
Well this is the weirdest thing I've seen.
When doing a console.log(req.params) or console.log(req.route.params) I get an empty array response ([]).
But when doing a console.log(req.params.uid) I get the value! Thats extremely weird but hey, it works :)
Cheers.

Resources