Silence Debug Logging in HTTPComponents - log4j

How do I silence the debug logging in http componenets?
09:23:22.145 [main] DEBUG o.a.h.c.protocol.RequestAddCookies - CookieSpec selected: default
09:23:22.145 [main] DEBUG o.a.h.c.protocol.RequestAuthCache - Auth cache not set in the context
09:23:22.145 [main] DEBUG o.a.h.i.c.PoolingHttpClientConnectionManager - Connection request: [route: {}->http://example.org:80][total kept alive: 1; route allocated: 1 of 100; total allocated: 1 of 200]
09:23:22.145 [main] DEBUG o.a.h.i.c.PoolingHttpClientConnectionManager - Connection leased: [id: 0][route: {}->http://diagnostics-uat.corp.apple.com:80][total kept alive: 0; route allocated: 1 of 100; total allocated: 1 of 200]
09:23:22.145 [main] DEBUG o.a.h.impl.execchain.MainClientExec - Executing request POST /evt/detect HTTP/1.1
09:23:22.145 [main] DEBUG o.a.h.impl.execchain.MainClientExec - Target auth state: UNCHALLENGED
09:23:22.145 [main] DEBUG o.a.h.impl.execchain.MainClientExec - Proxy auth state: UNCHALLENGED
09:23:22.145 [main] DEBUG org.apache.http.headers - http-outgoing-0 >> POST /evt/detect HTTP/1.1
09:23:22.145 [main] DEBUG org.apache.http.headers - http-outgoing-0 >> Content-Length: 92
I have tried a code based solution:
static {
System.setProperty("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.SimpleLog");
System.setProperty("org.apache.commons.logging.simplelog.log.org.apache.http", "warn");
}
public static void main(String[] args) throws Exception {
And this log4j.properties
log4j.rootLogger=INFO, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%5p [%c] %m%n
log4j.logger.org.apache.http=WARN
log4j.logger.org.apache.http.wire=WARN
But neither have worked. Help is appreciated, thanks!

You settings are correct possible causes may be another log4j config file is in the classpath and overriding your config. So I will recommended this.
- Ensure you log4j.properties file is in classpath . Use below command to see open files by a process (assuming unix)
lsof -p java-process-pid | grep log4j
Search for log4j.properties in your runtime folders and get rid of all duplicate files (if possible) - otherwise change them all.
Search for log4j.xml and do the same.
Sometimes I have seen third party jars that ship with a log4j.xml or log4j.properties so in worst case check for all jars using below command
jar -tvf a.jar | grep log4j

Programmatically, it can be set silent with:
(Apache HttpComponents 4.3.5)
import org.apache.log4j.Level;
import org.apache.log4j.Logger;
...
Logger.getLogger("org.apache.http") .setLevel(Level.WARN);
Logger.getLogger("org.apache.http.wire") .setLevel(Level.WARN);
Logger.getLogger("org.apache.http.headers").setLevel(Level.WARN);

Related

Can not generate nodejs-server with openapi-generator

How can I generate nodejs-server with OAS 2.0 as input with openapi-generator?
openapi-generator runs on two versions 3.3.4 and 4.0.0.
The results of my execution are listed below.
■in 3.3.4
java -jar openapi-generator-cli-3.3.4.jar generate -i petstore.json -g nodejs-server -o stub
[main] WARN o.o.c.ignore.CodegenIgnoreProcessor - Output directory does not exist, or is inaccessible. No file (.openapi-generator-ignore) will be evaluated.
[main] WARN o.o.c.languages.NodeJSServerCodegen -
=======================================================================================
Currently, Node.js server doesn't work as its dependency doesn't support OpenAPI Spec3.
For further details, see https://github.com/OpenAPITools/openapi-generator/issues/34
=======================================================================================
[main] INFO o.o.codegen.DefaultGenerator - Model Pets not generated since it's an alias to array (without property)
Exception in thread "main" java.lang.RuntimeException: Could not generate api file for 'Pets'
at org.openapitools.codegen.DefaultGenerator.generateApis(DefaultGenerator.java:651)
at org.openapitools.codegen.DefaultGenerator.generate(DefaultGenerator.java:891)
at org.openapitools.codegen.cmd.Generate.run(Generate.java:355)
at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:62)
Caused by: java.lang.IllegalArgumentException: character to be escaped is missing
at java.util.regex.Matcher.appendReplacement(Matcher.java:809)
at java.util.regex.Matcher.replaceAll(Matcher.java:955)
at java.lang.String.replaceAll(String.java:2223)
at org.openapitools.codegen.languages.NodeJSServerCodegen.apiFilename(NodeJSServerCodegen.java:192)
at org.openapitools.codegen.DefaultGenerator.generateApis(DefaultGenerator.java:595)
... 3 more
■in 4.0.0
java -jar openapi-generator-cli-4.0.0.jar generate -i petstore.json -g nodejs-server -o stub
[main] WARN o.o.c.ignore.CodegenIgnoreProcessor - Output directory does not exist, or is inaccessible. No file (.openapi-generator-ignore) will be evaluated.
[main] INFO o.o.codegen.DefaultGenerator - OpenAPI Generator: nodejs-server (server)
[main] INFO o.o.codegen.DefaultGenerator - Generator 'nodejs-server' is considered stable.
[main] WARN o.o.c.languages.NodeJSServerCodegen -
=======================================================================================
Currently, Node.js server doesn't work as its dependency doesn't support OpenAPI Spec3.
For further details, see https://github.com/OpenAPITools/openapi-generator/issues/34
=======================================================================================
[main] INFO o.o.codegen.DefaultGenerator - Model Pets not generated since it's an alias to array (without property) and `generateAliasAsModel` is set to false (default)
Exception in thread "main" java.lang.RuntimeException: Could not generate api file for 'Pets'
at org.openapitools.codegen.DefaultGenerator.generateApis(DefaultGenerator.java:666)
at org.openapitools.codegen.DefaultGenerator.generate(DefaultGenerator.java:922)
at org.openapitools.codegen.cmd.Generate.run(Generate.java:396)
at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:60)
Caused by: java.lang.IllegalArgumentException: character to be escaped is missing
at java.util.regex.Matcher.appendReplacement(Matcher.java:809)
at java.util.regex.Matcher.replaceAll(Matcher.java:955)
at java.lang.String.replaceAll(String.java:2223)
at org.openapitools.codegen.languages.NodeJSServerCodegen.apiFilename(NodeJSServerCodegen.java:181)
at org.openapitools.codegen.DefaultGenerator.generateApis(DefaultGenerator.java:611)
... 3 more
The input OAS uses the following.
https://raw.githubusercontent.com/OAI/OpenAPI-Specification/master/examples/v2.0/json/petstore.json
The execution environment is as follows.
・windows10
・java 1.8.0_202
It works for me:
java -jar modules/openapi-generator-cli/target/openapi-generator-cli.jar generate -g nodejs-server -i https://raw.githubusercontent.com/OAI/OpenAPI-Specification/master/examples/v2.0/json/petstore.json -o /tmp/nodejs-server
[main] INFO o.o.codegen.DefaultGenerator - OpenAPI Generator: nodejs-server (server)
[main] INFO o.o.codegen.DefaultGenerator - Generator 'nodejs-server' is considered stable.
[main] WARN o.o.c.languages.NodeJSServerCodegen -
=======================================================================================
Currently, Node.js server doesn't work as its dependency doesn't support OpenAPI Spec3.
For further details, see https://github.com/OpenAPITools/openapi-generator/issues/34
=======================================================================================
[main] INFO o.o.codegen.DefaultCodegen - Skipped overwriting README.md as the file already exists in /tmp/java2//README.md
[main] INFO o.o.codegen.DefaultGenerator - Model Pets not generated since it's an alias to array (without property) and `generateAliasAsModel` is set to false (default)
[main] INFO o.o.codegen.AbstractGenerator - writing file /tmp/java2/service/PetsService.js
[main] INFO o.o.codegen.AbstractGenerator - writing file /tmp/java2/controllers/Pets.js
[main] INFO o.o.codegen.AbstractGenerator - writing file /tmp/java2/utils/writer.js
[main] INFO o.o.codegen.AbstractGenerator - writing file /tmp/java2/api/openapi.yaml
[main] INFO o.o.codegen.AbstractGenerator - writing file /tmp/java2/index.js
[main] INFO o.o.codegen.AbstractGenerator - writing file /tmp/java2/package.json
[main] INFO o.o.codegen.AbstractGenerator - writing file /tmp/java2/.openapi-generator/VERSION
But as mentioned in the warning, the nodejs-server generator no longer works as expected as one of its dependencies does not support OpenAPI spec v3.
Please refer to https://github.com/OpenAPITools/openapi-generator/issues/2828 for the latest development of creating a new NodeJS Express generator.
UPDATE (2019/09): we've added a new nodejs-express-server generator. Please refer to https://twitter.com/oas_generator/status/1160000504455319553 for more information.

How to disable log4j default logging?

I am using restless API for my webservice implementation. I see that following lines are being printed on console for every call to my webservice:
Aug 5, 2016 12:30:09 PM org.restlet.engine.log.LogFilter afterHandle
INFO: 2016-08-05 12:30:09 172.23.4.200 - 172.23.7.44 8080 GET /abcservice/xyz - 200 86 0 21 http://localhost
There are so many calls to my webservice and as a result my logs (tomcat catalina.out) are going crazy. I want to disable this logging.
I have configured the log4j settings in log4j.xml How can I disable this logging.
You can set the root logger level to OFF (instead of WARN, DEBUG, INFO, etc)
e.g. log4j.rootLogger = OFF
But, I will recommended you to keep the level to WARN. You will not get the INFO logs but it will let you know about Warnings, Error and Fatal in your applications.
Priority of the logging levels:
**
ALL < DEBUG < INFO < WARN < ERROR < FATAL < OFF
**

Log4j filter by package

I would like to have log4j (v1.2.17) send ...
... to fileAppender
INFO (and above) from all packages
... to console
if from "my.project" then INFO (and above)
else WARN (and above) for all other packages
How can I do this? I'd prefer a properties file if possible, but would switch to XML if necessary.
I've tried combinations of loggers, non-additivity, Threshold, & LevelMatchFilter but can't figure it out.
This can definitely be done. Below you will find some example code and log4j properties that will do what you want.
Here is the first example class:
package my;
import org.apache.log4j.Logger;
public class Example1 {
private static final Logger logger = Logger.getLogger(Example1.class);
public static void main(String[] args){
logger.debug("debug from my.Example1 - should display nowhere!");
logger.info("info from my.Example1 - should display in log file only!");
logger.warn("warning from my.Example1 - should display in console and log file!");
logger.error("error from my.Example1 - should display in console and log file!");
logger.fatal("fatal from my.Example1 - should display in console and log file!");
}
}
Here is the second example class:
package my.project;
import org.apache.log4j.Logger;
public class Example2 {
private static final Logger logger = Logger.getLogger(Example2.class);
public static void main(String[] args){
logger.debug("debug from my.project.Example2 - should display nowhere!");
logger.info("info from my.project.Example2 - should display in console and log file!");
logger.warn("warning from my.project.Example2 - should display in console and log file!");
logger.error("error from my.project.Example2 - should display in console and log file!");
logger.fatal("fatal from my.project.Example2 - should display in console and log file!");
}
}
Here is the log4j.properties file:
# The root logger will accept INFO messages or higher and send them to the log file
# and to a console appender that filters out any messages below WARN level
log4j.rootLogger=INFO, R, warnStdout
# Configure a console appender that will be used for messages of any level
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
# Pattern to output the caller's file name and line number.
log4j.appender.stdout.layout.ConversionPattern=%5p [%t] (%F:%L) - %m%n
#This will be used to print WARN level or higher messages to console
log4j.appender.warnStdout=org.apache.log4j.ConsoleAppender
log4j.appender.warnStdout.layout=org.apache.log4j.PatternLayout
log4j.appender.warnStdout.Threshold=WARN
# Pattern to output the caller's file name and line number.
log4j.appender.warnStdout.layout.ConversionPattern=%5p [%t] (%F:%L) - %m%n
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=test.log
log4j.appender.R.MaxFileSize=100KB
# Keep one backup file
log4j.appender.R.MaxBackupIndex=1
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%p %t %c - %m%n
# Classes in the my.project package will accept messages of INFO level or higher
# and send those messages to the console and to the log file
log4j.logger.my.project=INFO, stdout, R
# Need to set additivity to false or else both the my.project and root loggers
# will accept messages from classes in package my.project
log4j.additivity.my.project=false
Here is the console output after running Example1 followed immediately by Example2:
WARN [main] (Example1.java:11) - warning from my.Example1 - should display in console and log file!
ERROR [main] (Example1.java:12) - error from my.Example1 - should display in console and log file!
FATAL [main] (Example1.java:13) - fatal from my.Example1 - should display in console and log file!
INFO [main] (Example2.java:11) - info from my.project.Example2 - should display in console and log file!
WARN [main] (Example2.java:12) - warning from my.project.Example2 - should display in console and log file!
ERROR [main] (Example2.java:13) - error from my.project.Example2 - should display in console and log file!
FATAL [main] (Example2.java:14) - fatal from my.project.Example2 - should display in console and log file!
Here is the test.log file content after running Example1 followed by Example2:
INFO main my.Example1 - info from my.Example1 - should display in log file only!
WARN main my.Example1 - warning from my.Example1 - should display in console and log file!
ERROR main my.Example1 - error from my.Example1 - should display in console and log file!
FATAL main my.Example1 - fatal from my.Example1 - should display in console and log file!
INFO main my.project.Example2 - info from my.project.Example2 - should display in console and log file!
WARN main my.project.Example2 - warning from my.project.Example2 - should display in console and log file!
ERROR main my.project.Example2 - error from my.project.Example2 - should display in console and log file!
FATAL main my.project.Example2 - fatal from my.project.Example2 - should display in console and log file!

Rexster refuses to start with extension but does not display errors

I have a small Rexster/Titan cluster using Cassandra. A Rexster extension is used to query the graph. I did some benchmarking and did start and stop Rexster/Titan many times. But now I run into a strange issue: Rexster refuses to start but does not display any error message.
I tried to figure out what is causing this and reduced the cluster to a single node 192.168.0.4.
If I remove my extension Rexster manages to start up.
# console output
Forking Cassandra...
Running `nodetool statusthrift`..... OK
(returned exit status 0 and printed string "running").
Forking Titan + Rexster...
Connecting to Titan + Rexster (127.0.0.1:8184)...... OK
(connected to 127.0.0.1:8184).
Run rexster-console.sh to connect.
but when I place my extension uber JAR in the ext folder Rexster refuses to start.
# console output
Forking Cassandra...
Running `nodetool statusthrift`..... OK
(returned exit status 0 and printed string "running").
Forking Titan + Rexster...
Connecting to Titan + Rexster (127.0.0.1:8184)............................
timeout exceeded (60 seconds): could not connect to 127.0.0.1:8184
See /var/lib/titan/bin/../log/rexstitan.log for Rexster log output.
If I now check rexstitan.log, as suggested by the console output, I can not find any error message.
# rexstitan.log
0 [main] INFO com.tinkerpop.rexster.Application - .:Welcome to Rexster:.
73 [main] INFO com.tinkerpop.rexster.server.RexsterProperties -
Using [/var/lib/titan/rexhome/../conf/rexster-cassandra-cluster.xml]
as configuration source.
78 [main] INFO com.tinkerpop.rexster.Application - Rexster is watching
[/var/lib/titan/rexhome/../conf/rexster-cassandra-cluster.xml] for change.
244 [main] INFO com.netflix.astyanax.connectionpool.impl.ConnectionPoolMBeanManager -
Registering mbean: com.netflix.MonitoredResources:type=ASTYANAX,
name=ClusterTitanConnectionPool,ServiceType=connectionpool
252 [main] INFO com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor -
AddHost: 192.168.0.4
537 [main] INFO com.netflix.astyanax.connectionpool.impl.ConnectionPoolMBeanManager -
Registering mbean: com.netflix.MonitoredResources:type=ASTYANAX,
name=KeyspaceTitanConnectionPool,ServiceType=connectionpool
538 [main] INFO com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor -
AddHost: 192.168.0.4
1951 [main] INFO com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration -
Set cluster.partition=false from store features
1971 [main] INFO com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration -
Set default timestamp provider MICRO
2019 [main] INFO com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration -
Generated unique-instance-id=7f0000012902-node1
2045 [main] INFO com.netflix.astyanax.connectionpool.impl.ConnectionPoolMBeanManager -
Registering mbean: com.netflix.MonitoredResources:type=ASTYANAX,
name=ClusterTitanConnectionPool,ServiceType=connectionpool
2046 [main] INFO com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor -
AddHost: 192.168.0.4
2053 [main] INFO com.netflix.astyanax.connectionpool.impl.ConnectionPoolMBeanManager -
Registering mbean: com.netflix.MonitoredResources:type=ASTYANAX,
name=KeyspaceTitanConnectionPool,ServiceType=connectionpool
2054 [main] INFO com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor -
AddHost: 192.168.0.4
2228 [main] INFO com.thinkaurelius.titan.diskstorage.Backend -
Initiated backend operations thread pool of size 4
6619 [main] INFO com.thinkaurelius.titan.diskstorage.log.kcvs.KCVSLog -
Loaded unidentified ReadMarker start time Timepoint[1423479705116000 μs]
into com.thinkaurelius.titan.diskstorage.log.kcvs.KCVSLog$MessagePuller#212f3ff1
6625 [main] INFO com.tinkerpop.rexster.RexsterApplicationGraph -
Graph [graph] - configured with allowable namespace [*:*]
The only entry that looks strange to me is the one concerning the log:
6619 [main] INFO com.thinkaurelius.titan.diskstorage.log.kcvs.KCVSLog -
Loaded unidentified ReadMarker start time Timepoint[1423479705116000 μs]
into com.thinkaurelius.titan.diskstorage.log.kcvs.KCVSLog$MessagePuller#212f3ff1
My exception uses the logger for debugging. You can see the instantiation an usage on github: https://github.com/sebschlicht/titan-graphity-kribble/blob/master/src/main/java/de/uniko/sebschlicht/titan/extensions/GraphityExtension.java#L22
Though Rexster failed to start there is a process with the PID displayed in the console but curl fails to connect to Rexster:
$ curl 192.168.0.4:8182
curl: (7) Failed to connect to 192.168.0.4 port 8182: Connection refused
Why doesn't Rexster throw an exception? How can I debug this situation?
edit:
I removed any log messages in my code. I removed all exceptions that may be thrown during startup. Still Rexster refuses to start with my extension and the only hint in the log files is the unidentified read marker. I have to clue what prevents Rexster from starting.
The log message is nothing to worry about.
After rebuilding the application in another project step-by-step Rexster is now able to start with the extension. During this rebuild I noticed two situations, that can cause the behaviour described:
Missing dependency
If your project depends on a second project you might use Maven to inject it as a dependency. However, if you use
mvn clean package
to build the extension's JAR file it does not contain this dependency by default. You need to use a Maven plugin (e.g. maven-shade-plugin) to create a shaded JAR that contains all the dependencies your extension needs. Set the dependency scope to provided for all Titan/Rexster/Blueprints related dependencies. Use the shaded uber-JAR to deploy the extension to Rexster.
However, this was not new to me and should not have caused the problem in my case. There might be more situations that cause this problem or maybe there was a problem with Maven that messed up the shaded JAR. Feel free to browse the commit on github to catch this voodoo.
Missing extension
Another cause of this behaviour is a missing extension.
If you specify an extension in the com.tinkerpop.rexster.extension.RexsterExtension resource file, that is not present on startup, Rexster does neither log nor throw an exception, but refuses to start.

Embed Apache ZooKeeper in Jetty generates Debug log message

I am using Jetty 8 and ZooKeeper 3.4.5. When I connect to ZooKeeper, jetty keep generating DEBUG level message. How to suppress it?
16:54:56.757 [main-SendThread(127.0.0.1:2181)] DEBUG o.a.z.client.ZooKeeperSaslClient - Could not retrieve login configuration: java.lang.SecurityException: Unable to locate a login configuration
16:54:56.758 [main-SendThread(127.0.0.1:2181)] DEBUG o.a.z.client.ZooKeeperSaslClient - Could not retrieve login configuration: java.lang.SecurityException: Unable to locate a login configuration
16:54:56.758 [main-SendThread(127.0.0.1:2181)] DEBUG o.a.z.client.ZooKeeperSaslClient - Could not retrieve login configuration: java.lang.SecurityException: Unable to locate a login configuration
16:54:56.759 [main-SendThread(127.0.0.1:2181)] DEBUG o.a.z.client.ZooKeeperSaslClient - Could not retrieve login configuration: java.lang.SecurityException: Unable to locate a login configuration
16:54:56.760 [main-SendThread(127.0.0.1:2181)] DEBUG o.a.z.client.ZooKeeperSaslClient - Could not retrieve login configuration: java.lang.SecurityException: Unable to locate a login configuration
16:54:56.760 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for sessionid: 0x13ddc6e9a2b000c after 3ms
16:54:56.760 [main-SendThread(127.0.0.1:2181)] DEBUG o.a.z.client.ZooKeeperSaslClient - Could not retrieve login configuration: java.lang.SecurityException: Unable to locate a login configuration
16:54:58.424 [main-SendThread(127.0.0.1:2181)] DEBUG o.a.z.client.ZooKeeperSaslClient - Could not retrieve login configuration: java.lang.SecurityException: Unable to locate a login configuration
16:54:58.425 [main-SendThread(127.0.0.1:2181)] DEBUG o.a.z.client.ZooKeeperSaslClient - Could not retrieve login configuration: java.lang.SecurityException: Unable to locate a login configuration
16:54:58.425 [main-SendThread(127.0.0.1:2181)] DEBUG o.a.z.client.ZooKeeperSaslClient - Could not retrieve login configuration: java.lang.SecurityException: Unable to locate a login configuration
16:54:58.426 [main-SendThread(127.0.0.1:2181)] DEBUG o.a.z.client.ZooKeeperSaslClient - Could not retrieve login configuration: java.lang.SecurityException: Unable to locate a login configuration
16:54:58.427 [main-SendThread(127.0.0.1:2181)] DEBUG o.a.z.client.ZooKeeperSaslClient - Could not retrieve login configuration: java.lang.SecurityException: Unable to locate a login configuration
16:54:58.427 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for sessionid: 0x13ddc6e9a2b000c after 2ms
I added log4j.properties under src folder:
# Set root logger level to DEBUG and its only appender to A1.
log4j.rootLogger=DEBUG, A1
# A1 is set to be a ConsoleAppender.
log4j.appender.A1=org.apache.log4j.ConsoleAppender
# A1 uses PatternLayout.
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
log4j.appender.A1.layout.ConversionPattern=%-4r - [%t] %-5p %c %x - %m%n
log4j.logger.org.apache.zookeeper.ZooKeeper=INFO
Still no luck.
I have checked again my included library, and I found out that LogBack was included. I think JVM load logback instead of log4j. Once I supply logback.xml, the problem is solved.
Something like this in config should help:
<logger name="com.apache.zookeeper" level="info">
<!-- appenders -->
</logger>

Resources