MDC Context getting copied to only one thread of ForkJoinPool - multithreading

I want to pass MDC context to threads of the ForkJoinPool as I need to print the requestId populated in the MDCContextMap in all the spawned threads logs for better debugging. I am using the following solution for this: How to use MDC with ForkJoinPool?.
I am using the pool as:
final ForkJoinPool mdcPool = new MdcForkJoinPool(30);
mdcPool.submit(() -> ruleset.getOperationList().parallelStream().forEach(operation -> {
log.info("Sample log line");
But from the logs it looks like MDC Context is getting copied to only one thread of the pool. I have also put some logs in the submit method of the MdcForkJoinPool and also in the beforeExecution method. Following are the logs where the requestID is getting printed only in one thread ("ForkJoinPool-1-worker-25") logs:
16 Oct 2018 15:05:32,570 [INFO] eb73e823-d154-11e8-871a-d138501061d6 (Coral Endpoint : 1) com.amazon.service.utils.MdcForkJoinPool: submit: Runnable
16 Oct 2018 15:05:32,571 [INFO] (ForkJoinPool-1-worker-25) com.amazon.service.utils.MdcForkJoinPool: Before Execution: {RequestId=eb73e823-d154-11e8-871a-d138501061d6}
16 Oct 2018 15:05:32,571 [INFO] (ForkJoinPool-1-worker-25) com.amazon.service.utils.MdcForkJoinPool: Before Execution: {}
16 Oct 2018 15:05:32,571 [INFO] (ForkJoinPool-1-worker-25) com.amazon.service.utils.MdcForkJoinPool: Setting mDC: {}
16 Oct 2018 15:05:32,574 [INFO] (ForkJoinPool-1-worker-4) com.amazon.service.activity.business.VariablesEvaluator: Sample log line
16 Oct 2018 15:05:32,574 [INFO] (ForkJoinPool-1-worker-29) com.amazon.service.activity.business.VariablesEvaluator: Sample log line
16 Oct 2018 15:05:32,575 [INFO] eb73e823-d154-11e8-871a-d138501061d6 (ForkJoinPool-1-worker-25) com.amazon.service.activity.business.VariablesEvaluator: Sample log line
16 Oct 2018 15:05:32,576 [INFO] (ForkJoinPool-1-worker-22) com.amazon.service.activity.business.VariablesEvaluator: Sample log line
16 Oct 2018 15:05:32,576 [INFO] (ForkJoinPool-1-worker-4) com.amazon.service.activity.business.VariablesEvaluator: some Other logs
16 Oct 2018 15:05:32,576 [INFO] eb73e823-d154-11e8-871a-d138501061d6 (ForkJoinPool-1-worker-25) com.amazon.service.activity.business.VariablesEvaluator: some Other logs
Am I not using the custom ForkJoinPool in the correct way?

Related

mod_pagespeed many "Unrecognized Script" messages in the console

I am running mod_pagespeed (Apache) on my server and it is functioning great, apart from many "info level" messages in the admin console log. The site is currently scoring 94/89 on GTMetrix and I'm at the point of trying to squeeze that last bit of juice from this wonderful module.
These messages refer to unrecognised scripts that mod_pagespeed was attempting to process. In most cases they contain json or text/html, i.e.:
[Wed, 29 Jan 2020 13:17:10 GMT] [Info] [16645] [https://www.example.com/comment/:271] Unrecognized script:'<script type='application/ld+json'></script> 271...295'
[Wed, 29 Jan 2020 13:17:10 GMT] [Info] [16645] [https://www.example.com/comment/:88] Unrecognized script:'<script type='application/ld+json' class='yoast-schema-graph yoast-schema-graph--main'></script> 88...88'
[Wed, 29 Jan 2020 13:13:01 GMT] [Info] [14352] [https://www.example.com/comment/:47] Unrecognized script:'<script type="text/html" id="wpb-modifications"></script> 47...47'
[Wed, 29 Jan 2020 13:13:01 GMT] [Info] [14352] [https://www.example.com/comment/:11] Unrecognized script:'<script type='application/ld+json'></script> 11...33'
[Wed, 29 Jan 2020 13:13:01 GMT] [Info] [14352] [https://www.example.com//:11] Unrecognized script:'<script type='application/ld+json' class='yoast-schema-graph yoast-schema-graph--main'></script> 11...11'
Does anyone here know of any way to configure Pagespeed to be able to recognise this format?
I've already fixed some issues with unrecognised image links within some tags:
ModPagespeedUrlValuedAttribute img data-lazy-src image
ModPagespeedUrlValuedAttribute a data-tolb-src image
And those assets are now being rewritten/compressed.
My main problem is that I don't know how to get Pagespeed to recognise these script tags.
Is there a way? (Is it even necessary for these type of script tags?)
Many thanks in advance.

BBB Palette not displayed on Node-Red on Beaglebone Black Wireless

My node-red installation does not display the nodes needed for accessing beaglebone IOs (node-red-node-beaglebone).
In my opinion error is actually caused because node-red-node-beaglebone loads octalbonescript which needs serialport, and something requests serialPort instead of serialport there.
I already tried the preinstalled, stable and lts nodejs version. Additionally npm#2 and npm#3 for installing the nodes. The .node-red folder was also already deleted by me and the node-red-node-beaglebone packaged got installed in the .node-red/node_modules folder started from scrach.
root#beaglebone:/etc# cat debian_version
8.7
root#beaglebone:~# npm -v
2.15.11
root#beaglebone:~# uname -a
Linux beaglebone 4.4.30-ti-r64 #1 SMP Fri Nov 4 21:23:33 UTC 2016 armv7l GNU/Linux
root#beaglebone:~# export AUTO_LOAD_CAPE=0 #optional
root#beaglebone:~# node-red-pi
1487183133432 Board Looking for connected device
15 Feb 19:25:34 - [info]
Welcome to Node-RED
===================
15 Feb 19:25:34 - [info] Node-RED version: v0.16.2
15 Feb 19:25:34 - [info] Node.js version: v6.9.5
15 Feb 19:25:34 - [info] Linux 4.4.30-ti-r64 arm LE
15 Feb 19:25:44 - [info] Loading palette nodes
15 Feb 19:25:57 - [warn] ------------------------------------------------------
15 Feb 19:25:57 - [warn] [bbb] ReferenceError: serialPort is not defined +seems to be the problem
15 Feb 19:25:57 - [warn] ------------------------------------------------------
15 Feb 19:25:57 - [info] Settings file : /root/.node-red/settings.js
15 Feb 19:25:57 - [info] User directory : /root/.node-red
15 Feb 19:25:57 - [info] Flows file : /root/.node-red/flows_beaglebone.json
15 Feb 19:25:57 - [info] Creating new flow file
15 Feb 19:25:57 - [debug] loaded flow revision: 513fd923d68021b8ee98fcb250470340
15 Feb 19:25:57 - [debug] red/runtime/nodes/credentials.load : no user key present
15 Feb 19:25:57 - [debug] red/runtime/nodes/credentials.load : using default key
15 Feb 19:25:57 - [info] Starting flows
15 Feb 19:25:57 - [info] Started flows
15 Feb 19:25:57 - [info] Server now running at http://127.0.0.1:1880/

Custom logging in couch.log from couchapp?

Is it possible to write in the couchdb server log (the one defined by default.ini or local.ini in [log]) from a couchapp? (But from somewhere else than a view)
If that's not possible, maybe there's a workaround which would allow to log successful or unsuccessful authentication attemps in the couchdb server log? I'd like to process this server side and would like to avoid logging all httpd activity and grepping for user logging patterns, which doesn't seem to be easy or pretty...
Cheers,
Jun
A year later I find that it was in fact possible to log from views (or lists or any Javascript Design Doc functions) using the log() function: http://docs.couchdb.org/en/1.6.1/query-server/javascript.html#log
log(message)
Log a message to the CouchDB log (at the INFO level).
Arguments:
message – Message to be logged
function(doc){
log('Procesing doc ' + doc['_id']);
emit(doc['_id'], null);
}
After the map function has run, the following line can be found in CouchDB logs (e.g. at /var/log/couchdb/couch.log):
[Sat, 03 Nov 2012 17:38:02 GMT] [info] [<0.7543.0>] OS Process #Port<0.3289> Log :: Processing doc 8d300b86622d67953d102165dbe99467
Who would have guessed :)
I'm pretty sure you can't write to couch.log from a view, it's a sandboxed system.
Getting a record of connections to the server is possible though. Here's a dump from my couch.log, with an HTTP error in there:
/
[Sat, 13 Sep 2014 08:18:57 GMT] [info] [<0.160.0>] Opening index for db: test idx: _design/ivet sig: "f6b64ef8593e23cac644c13b895b7607"
[Sat, 13 Sep 2014 08:18:57 GMT] [info] [<0.121.0>] 127.0.0.1 - - GET /test/_design/ivet/_view/medicationWHP/foobar?include_docs=true 200
[Sat, 13 Sep 2014 08:18:57 GMT] [info] [<0.121.0>] 127.0.0.1 - - GET /test/_design/ivet/_view/medicationWHP/foobar?include_docs=true 500
[Sat, 13 Sep 2014 08:18:57 GMT] [error] [<0.121.0>] httpd 500 error response:
{"error":"json_encode","reason":"{bad_term,{key,null}}"}
[Sat, 13 Sep 2014 08:19:05 GMT] [info] [<0.36.0>] Apache CouchDB has started on http://127.0.0.1:5984/
You can see it has the VERB PATH CODE format for each line, so you can filter that for whatever you need. (Unauthorized is 401) You can also access the log through /_log. Details on that are here:
http://docs.couchdb.org/en/latest/api/server/common.html#log
To get all that information, you'll need to have the log level set to info. You can do this at the config screen in futon.
To do it server-side, you'd probably need to use node.js or something like that. Just have it consume the /_log endpoint, and filter each line by the HTTP response code.

log4j - DailyRollingFileAppender, override subAppend()

I am new to log4j's DailyRollingFileAppender class and I would like to use this to perform daily rotation of the log file and at the same time would like to also manually modify the log file every time there is an event triggered to log event.
For example, I would like to always increment the value by one for "TOTAL COUNT:" inside the log file. How can I go about doing that?
Example of the log content:
07 Oct 2011 16:57:51 [INFO ] - Failed
07 Oct 2011 16:57:51 [WARN ] - Failed
07 Oct 2011 16:57:51 [ERROR] - Successful
07 Oct 2011 16:57:51 [FATAL] - Failed
07 Oct 2011 16:57:52 [DEBUG] - Successful
07 Oct 2011 16:57:52 [INFO ] - Failed
07 Oct 2011 16:57:52 [WARN ] - Failed
07 Oct 2011 16:57:52 [ERROR] - Successful
07 Oct 2011 16:57:52 [FATAL] - Failed
07 Oct 2011 16:57:53 [DEBUG] - Successful
07 Oct 2011 16:57:53 [INFO ] - Failed
07 Oct 2011 16:57:53 [WARN ] - Failed
07 Oct 2011 16:57:53 [ERROR] - Successful
07 Oct 2011 16:57:53 [FATAL] - Failed
07 Oct 2011 16:57:54 [DEBUG] – Successful
TOTAL COUNT: 15
It seems overriding subAppend() in the DailyRollingFileAppender is the way to go. You'll also have to be cautious when calling super.subAppend(), as the WriterAppender implements it like this:
protected void subAppend(LoggingEvent event) {
this.qw.write(this.layout.format(event));
// ...
}
and you don't want layout with your "TOTAL COUNT" line.
In my subAppend() I'd copy exactly what D.R.F.Appender does, adding:
line counting logic, if not present elsewhere,
a condition checking if a special line should be printed, then formatting and printing it directly to qw.

Cann't receive REMOTE host log messages with Chainsaw (localhost messages just OK)

I did a logger with programmatically configured SocketAppender for Chainsaw. I successfully receive messages in the SimpleReceiver of Chainsaw from code running on localhost, but from remote host i'm just getting like
63 Fri Sep 23 14:44:08 MSD
2011 INFO org.apache.log4j.chainsaw.messages.MessageCenter Connection
received from
my.host.com:50299 Chainsaw-WorkerThread log chainsaw
68 Fri Sep 23 15:12:22 MSD
2011 INFO org.apache.log4j.chainsaw.messages.MessageCenter Connection
lost! :: null Chainsaw-WorkerThread
log chainsaw 69 Fri Sep 23 15:12:22 MSD
2011 DEBUG org.apache.log4j.net.SocketReceiver accepted
socket Chainsaw-WorkerThread log chainsaw
70 Fri Sep 23 15:12:22 MSD
2011 DEBUG org.apache.log4j.net.SocketReceiver socket not null -
creating and starting
socketnode Chainsaw-WorkerThread log chainsaw
71 Fri Sep 23 15:12:22 MSD
2011 DEBUG org.apache.log4j.net.SocketReceiver w*aiting to accept
socket* Chainsaw-WorkerThread log chainsaw
but no tab with log messages from remote host. What's wrong? No firewall is running.
Tested with v2 and v2.1-trunk-today.
Solved yesterday. Log level in big app's plugin wasn't set correctly.

Resources