Search pattern in awk - linux

I want to create a script such that it checks the conditions:
Port string should be present in the file and if yes then it's value should be 20000.
The file should have a mention of sslkey, sslCert, ssl_cipher (The values against these strings/keys can be anything).
The attempt was made:
$ awk '/port|sslKey|sslCert|ssl_cipher/ {print $2,$3}' pkg.conf
port 20000
sslKey /usr/product/plat/etc/ssl/server.pem
sslCert /usr/product/plat/etc/ssl/server.cert
ssl_cipher ECDH+AES128:ECDH+AESGCM:ECDH+AES256:DH+AES:DH+AESGCM:DH+AES256:RSA+AES:RSA+AESGCM:!aNULL:!RC4:!MD5:!DSS:!3DES
The problem with the above command is even if one of the strings 'port|sslKey|sslCert|ssl_cipher' is missing even then it runs.
Could this be achieved using only a few lines of awk.
If any of the string/condition is missing then the output should display that condition and also the conditions those are met.

Considering Your details in questions here are input examples:
Correct:
$ cat pkg.conf
port 20000
sslKey /usr/product/plat/etc/ssl/server.pem
sslCert /usr/product/plat/etc/ssl/server.cert
ssl_cipher ECDH+AES128:ECDH+AESGCM:ECDH+AES256:DH+AES:DH+AESGCM:DH+AES256:RSA+AES:RSA+AESGCM:!aNULL:!RC4:!MD5:!DSS:!3DES
Wrong (port 18000 and missing sslCert):
$ cat pkg_wrong.conf
port 18000
sslKey /usr/product/plat/etc/ssl/server.pem
ssl_cipher ECDH+AES128:ECDH+AESGCM:ECDH+AES256:DH+AES:DH+AESGCM:DH+AES256:RSA+AES:RSA+AESGCM:!aNULL:!RC4:!MD5:!DSS:!3DES
AWK solution:
Correct pkg.conf. Returns 0 and outputs nothing:
$ awk '
/^port [0-9]+$/ { if ( $2 == 20000 ) isport=1; port=$2; }
/^sslKey .+$/ { issslKey=1; sslKey=$2; }
/^sslCert .+$/ { issslCert=1; sslCert=$2; }
/^ssl_cipher .+$/ { isssl_cipher=1; ssl_cipher=$2; }
END { if (isport && issslKey && issslCert && isssl_cipher) exit(0);
else { print("port " port); print("sslKey " sslKey); print("sslCert " sslCert); print("ssl_cipher " ssl_cipher); exit(1); }
}
' pkg.conf
Wrong pkg_wrong.conf (port 18000 and missing sslCert):
$ awk '
/^port [0-9]+$/ { if ( $2 == 20000 ) isport=1; port=$2; }
/^sslKey .+$/ { issslKey=1; sslKey=$2; }
/^sslCert .+$/ { issslCert=1; sslCert=$2; }
/^ssl_cipher .+$/ { isssl_cipher=1; ssl_cipher=$2; }
END { if (isport && issslKey && issslCert && isssl_cipher) exit(0);
else { print("port " port); print("sslKey " sslKey); print("sslCert " sslCert); print("ssl_cipher " ssl_cipher); exit(1); }
}
' pkg_wrong.conf
port 18000
sslKey /usr/product/plat/etc/ssl/server.pem
sslCert
ssl_cipher ECDH+AES128:ECDH+AESGCM:ECDH+AES256:DH+AES:DH+AESGCM:DH+AES256:RSA+AES:RSA+AESGCM:!aNULL:!RC4:!MD5:!DSS:!3DES

Related

Find count in the grep command in linux

I have one log file of mongodb. I want to display all output of grep command in greater than or less than given value"protocol:op_msg 523ms"
sed -n '/2022-09-15T12:26/,/2022-09-15T14:03/p' mongod.log| grep "op_msg 523ms"
output of log file :
7391:2022-11-22T09:23:23.047-0500 I COMMAND [conn26] command test.test appName: "MongoDB Shell" command: find { find: "test", filter: { creationDate: new Date(1663252936409) }, lsid: { id: UUID("7c1bb40c-5e99-4281-9351-893e3d23261d") }, $clusterTime: { clusterTime: Timestamp(1669126970, 1), signature: { hash: BinData(0, B141BFD0978167F8C023DFB4AB32BBB117B3CD80), keyId: 7136078726260850692 } }, $db: "test" } planSummary: COLLSCAN keysExamined:0 docsExamined:337738 cursorExhausted:1 numYields:2640 nreturned:1 queryHash:6F9DC23E planCacheKey:6F9DC23E reslen:304 locks:{ ReplicationStateTransition: { acquireCount: { w: 2641 } }, Global: { acquireCount: { r: 2641 } }, Database: { acquireCount: { r: 2641 } }, Collection: { acquireCount: { r: 2641 } }, Mutex: { acquireCount: { r: 1 } } } storage:{ data: { bytesRead: 28615999, timeReadingMicros: 288402 } } protocol:op_msg 523ms
I have tried below command , but this command is only giving exact value. I need to find all query of log file which is greater than 100ms.
sed -n '/2022-09-15T12:26/,/2022-09-15T14:03/p' mongod.log| grep "op_msg 523ms"
Option 1) You can use grep bash onliner like this to filter mongo queries in specific execution time range:
grep -P '\d+ms' /log/mongodb/mongos.log | while read LINE; do querytime="$(echo "$LINE" | grep -oP '\d+ms' | grep -oP '\d+')"; if [ "$querytime" -gt 6000 ]&&[ "$querytime" -lt 7000 ]; then echo "$LINE"; fi; done
Explained:
Filter all log lines having Number+"ms" at the end.
Loop over the queries to extract execution time in querytime variable
Check if $querytime between 6000ms and 7000ms
If $queritime in the specified range print the current $LINE
Option 2) You can use the --slow MS, --fast MS options from mlogfilter from the mtools package where you can do something like:
mlogfilter mongod.log --slow 7000 --fast 6000
If all of the lines of interest have protocol:op_msg as the penultimate column, this becomes pretty trivial:
awk '$(NF-1) ~ /protocol:op_msg/ && $NF > 100 { print $NF }'
Note that this completely ignores the units when doing the comparison and will also print lines in which the final column is 250h or 250ns, but that's easy enough to filter for. With a more accurate problem description, a more precise solution is certainly available.

Can I avoid this subshell in a POSIX sh script?

I am trying to comprehend how, if even it can be done, can I avoid subshell?
Is this the only way the code can be written or is there another way?
I tried to use braces { ... }, but it won't pass shellcheck and won't run.
is_running_interactively ()
# test if file descriptor 0 = standard input is connected to the terminal
{
[ -t 0 ]
}
is_tput_available ()
# check if tput coloring is available
{
command -v tput > /dev/null 2>&1 &&
tput bold > /dev/null 2>&1 &&
tput setaf 1 > /dev/null 2>&1
}
some_other_function ()
# so far unfinished function
{
# is this a subshell? if so, can I avoid it somehow?
( is_running_interactively && is_tput_available ) || # <-- HERE
{
printf '%b' "${2}"
return
}
...
}
It is a compound-list, and yes those commands are run in a subshell. To avoid it, use curly braces instead of parentheses:
{ is_running_interactively && is_tput_available; } || ...

How to get a response of file printing job to user from printer using CUPS in linux

I am trying to print a file in remote server through CUPS command , User needs to know the status of job status. How to get a response from that. Here is my code :
#!/usr/bin/perl
my $response = system("lpr -P laserJet123 -o raw -T test_womargin abc.txt");
print $response;
$? returns the response status of system command.
system("lpr -P laserJet123 -o raw -T test_womargin abc.txt");
if ($? == -1) {
print "failed to execute: $!\n";
}
elsif ($? & 127) {
printf "Job failed with signal %d, %s coredump\n",
($? & 127), ($? & 128) ? 'with' : 'without';
}
else {
printf "Job exited with value %d\n", $? >> 8;
}

CHECK_GEARMAN CRITICAL - function 'BulkEmail' is not registered in the server

I am using the nagios to monitor gearman and getting error "CRITICAL - function 'xxx' is not registered in the server"
Script that nagios execute to check the gearman is like
#!/usr/bin/env perl
# taken from: gearmand-0.24/libgearman-server/server.c:974
# function->function_name, function->job_total,
# function->job_running, function->worker_count);
#
# this code give following result with gearadmin --status
#
# FunctionName job_total job_running worker_count
# AdsUpdateCountersFunction 0 0 4
use strict;
use warnings;
use Nagios::Plugin;
my $VERSION="0.2.1";
my $np;
$np = Nagios::Plugin->new(usage => "Usage: %s -f|--flist <func1[:threshold1],..,funcN[:thresholdN]> [--host|-H <host>] [--port|-p <port>] [ -c|--critworkers=<threshold> ] [ -w|--warnworkers=<threshold>] [-?|--usage] [-V|--version] [-h|--help] [-v|--verbose] [-t|--timeout=<timeout>]",
version => $VERSION,
blurb => 'This plugin checks a gearman job server, expecting that every function in function-list arg is registered by at least one worker, and expecting that job_total is not too much high.',
license => "Brought to you AS IS, WITHOUT WARRANTY, under GPL. (C) Remi Paulmier <remi.paulmier\#gmail.com>",
shortname => "CHECK_GEARMAN",
);
$np->add_arg(spec => 'flist|f=s',
help => q(Check for the functions listed in STRING, separated by comma. If optional threshold is given (separated by :), check that waiting jobs for this particular function are not exceeding that value),
required => 1,
);
$np->add_arg(spec => 'host|H=s',
help => q(Check the host indicated in STRING),
required => 0,
default => 'localhost',
);
$np->add_arg(spec => 'port|p=i',
help => q(Use the TCP port indicated in INTEGER),
required => 0,
default => 4730,
);
$np->add_arg(spec => 'critworkers|c=i',
help => q(Exit with CRITICAL status if fewer than INTEGER workers have registered a particular function),
required => 0,
default => 1,
);
$np->add_arg(spec => 'warnworkers|w=i',
help => q(Exit with WARNING status if fewer than INTEGER workers have registered a particular function),
required => 0,
default => 4,
);
$np->getopts;
my $ng = $np->opts;
# manage timeout
alarm $ng->timeout;
my $runtime = {'status' => OK,
'message' => "Everything OK",
};
# host & port
my $host = $ng->get('host');
my $port = $ng->get('port');
# verbosity
my $verbose = $ng->get('verbose');# look for gearadmin, use nc if not found
my #paths = grep { -x "$_/gearadmin" } split /:/, $ENV{PATH};
my $cmd = "gearadmin --status -h $host -p $port";
if (#paths == 0) {
print STDERR "gearadmin not found, using nc\n" if ($verbose != 0);
# $cmd = "echo status | nc -w 1 $host $port";
$cmd = "echo status | nc -i 1 -w 1 $host $port";
}
foreach (`$cmd 2>/dev/null | grep -v '^\\.'`) {
chomp;
my ($fname, $job_total, $job_running, $worker_count) =
split /[[:space:]]+/;
$runtime->{'funcs'}{"$fname"} = {job_total => $job_total,
job_running => $job_running,
worker_count => $worker_count };
# print "$fname : $runtime->{'funcs'}{\"$fname\"}{'worker_count'}\n";
}
# get function list
my #flist = split /,/, $ng->get('flist');
foreach (#flist) {
my ($fname, $fthreshold);
if (/\:/) {
($fname, $fthreshold) = split /:/;
} else {
($fname, $fthreshold) = ($_, -1);
}
# print "defined for $fname: $runtime->{'funcs'}{\"$fname\"}{'worker_count'}\n";
# if (defined($runtime->{'funcs'}{"$fname"})) {
# print "$fname is defined\n";
# } else {
# print "$fname is NOT defined\n";
# }
if (!defined($runtime->{'funcs'}{"$fname"}) &&
$runtime->{'status'} <= CRITICAL) {
($runtime->{'status'}, $runtime->{'message'}) =
(CRITICAL, "function '$fname' is not registered in the server");
} else {
if ($runtime->{'funcs'}{"$fname"}{'worker_count'} <
$ng->get('critworkers') && $runtime->{'status'} <= CRITICAL) {
($runtime->{'status'}, $runtime->{'message'}) =
(CRITICAL,
"less than " .$ng->get('critworkers').
" workers were found having function '$fname' registered.");
}
if ($runtime->{'funcs'}{"$fname"}{'worker_count'} <
$ng->get('warnworkers') && $runtime->{'status'} <= WARNING) {
($runtime->{'status'}, $runtime->{'message'}) =
(WARNING,
"less than " .$ng->get('warnworkers').
" workers were found having function '$fname' registered.");
}
if ($runtime->{'funcs'}{"$fname"}{'job_total'} > $fthreshold
&& $fthreshold != -1 && $runtime->{'status'}<=WARNING) {
($runtime->{'status'}, $runtime->{'message'}) =
(WARNING,
$runtime->{'funcs'}{"$fname"}{'job_total'}.
" jobs for $fname exceeds threshold $fthreshold");
}
}
}
$np->nagios_exit($runtime->{'status'}, $runtime->{'message'});
When the script is executed simply by command line it says "everything ok"
But in nagios it shows error "CRITICAL - function 'xxx' is not registered in the server"
Thanks in advance
After spending long time on this, finally got the answer all that have to do is.
yum install nc
nc is what that was missing from the system.
With Regards,
Bankat Vikhe
Not easy to say but it could be related to your script not being executable as embedded Perl.
Try with # nagios: -epn at the beginning of the script.
#!/usr/bin/env perl
# nagios: -epn
use strict;
use warnings;
Be sure to check all the hints in the Perl Plugins section of the Nagios Plugin Development Guidelines

using external redis server for testing tcl scripts

I am running Ubuntu 11.10.
i am trying to run TCL test scripts using external redis server.
using the following :
sb#sb-laptop:~/Redis/redis$ tclsh tests/test_helper.tcl --host 192.168.1.130 --port 6379
Getting the following error :
Testing unit/type/list
[exception]: Executing test client: couldn't open socket: connection refused.
couldn't open socket: connection refused
while executing
"socket $server $port"
(procedure "redis" line 2)
invoked from within
"redis $::host $::port"
(procedure "start_server" line 9)
invoked from within
"start_server {tags {"protocol"}} {
test "Handle an empty query" {
reconnect
r write "\r\n"
r flush
assert_equal "P..."
(file "tests/unit/protocol.tcl" line 1)
invoked from within
"source $path"
(procedure "execute_tests" line 4)
invoked from within
"execute_tests $data"
(procedure "test_client_main" line 9)
invoked from within
"test_client_main $::test_server_port "
the redis.conf is set to default binding, but it is commented out.
If this is possible, what i am doing wrong?
Additional Information:
Below is the tcl code that is responsible for starting the server
proc start_server {options {code undefined}} {
# If we are runnign against an external server, we just push the
# host/port pair in the stack the first time
if {$::external} {
if {[llength $::servers] == 0} {
set srv {}
dict set srv "host" $::host
dict set srv "port" $::port
set client [redis $::host $::port]
dict set srv "client" $client
$client select 9
# append the server to the stack
lappend ::servers $srv
}
uplevel 1 $code
return
}
# setup defaults
set baseconfig "default.conf"
set overrides {}
set tags {}
# parse options
foreach {option value} $options {
switch $option {
"config" {
set baseconfig $value }
"overrides" {
set overrides $value }
"tags" {
set tags $value
set ::tags [concat $::tags $value] }
default {
error "Unknown option $option" }
}
}
set data [split [exec cat "tests/assets/$baseconfig"] "\n"]
set config {}
foreach line $data {
if {[string length $line] > 0 && [string index $line 0] ne "#"} {
set elements [split $line " "]
set directive [lrange $elements 0 0]
set arguments [lrange $elements 1 end]
dict set config $directive $arguments
}
}
# use a different directory every time a server is started
dict set config dir [tmpdir server]
# start every server on a different port
set ::port [find_available_port [expr {$::port+1}]]
dict set config port $::port
# apply overrides from global space and arguments
foreach {directive arguments} [concat $::global_overrides $overrides] {
dict set config $directive $arguments
}
# write new configuration to temporary file
set config_file [tmpfile redis.conf]
set fp [open $config_file w+]
foreach directive [dict keys $config] {
puts -nonewline $fp "$directive "
puts $fp [dict get $config $directive]
}
close $fp
set stdout [format "%s/%s" [dict get $config "dir"] "stdout"]
set stderr [format "%s/%s" [dict get $config "dir"] "stderr"]
if {$::valgrind} {
exec valgrind --suppressions=src/valgrind.sup src/redis-server $config_file > $stdout 2> $stderr &
} else {
exec src/redis-server $config_file > $stdout 2> $stderr &
}
# check that the server actually started
# ugly but tries to be as fast as possible...
set retrynum 100
set serverisup 0
if {$::verbose} {
puts -nonewline "=== ($tags) Starting server ${::host}:${::port} "
}
after 10
if {$code ne "undefined"} {
while {[incr retrynum -1]} {
catch {
if {[ping_server $::host $::port]} {
set serverisup 1
}
}
if {$serverisup} break
after 50
}
} else {
set serverisup 1
}
if {$::verbose} {
puts ""
}
if {!$serverisup} {
error_and_quit $config_file [exec cat $stderr]
}
# find out the pid
while {![info exists pid]} {
regexp {\[(\d+)\]} [exec cat $stdout] _ pid
after 100
}
# setup properties to be able to initialize a client object
set host $::host
set port $::port
if {[dict exists $config bind]} { set host [dict get $config bind] }
if {[dict exists $config port]} { set port [dict get $config port] }
# setup config dict
dict set srv "config_file" $config_file
dict set srv "config" $config
dict set srv "pid" $pid
dict set srv "host" $host
dict set srv "port" $port
dict set srv "stdout" $stdout
dict set srv "stderr" $stderr
# if a block of code is supplied, we wait for the server to become
# available, create a client object and kill the server afterwards
if {$code ne "undefined"} {
set line [exec head -n1 $stdout]
if {[string match {*already in use*} $line]} {
error_and_quit $config_file $line
}
while 1 {
# check that the server actually started and is ready for connections
if {[exec cat $stdout | grep "ready to accept" | wc -l] > 0} {
break
}
after 10
}
# append the server to the stack
lappend ::servers $srv
# connect client (after server dict is put on the stack)
reconnect
# execute provided block
set num_tests $::num_tests
if {[catch { uplevel 1 $code } error]} {
set backtrace $::errorInfo
# Kill the server without checking for leaks
dict set srv "skipleaks" 1
kill_server $srv
# Print warnings from log
puts [format "\nLogged warnings (pid %d):" [dict get $srv "pid"]]
set warnings [warnings_from_file [dict get $srv "stdout"]]
if {[string length $warnings] > 0} {
puts "$warnings"
} else {
puts "(none)"
}
puts ""
error $error $backtrace
}
# Don't do the leak check when no tests were run
if {$num_tests == $::num_tests} {
dict set srv "skipleaks" 1
}
# pop the server object
set ::servers [lrange $::servers 0 end-1]
set ::tags [lrange $::tags 0 end-[llength $tags]]
kill_server $srv
} else {
set ::tags [lrange $::tags 0 end-[llength $tags]]
set _ $srv
}
}
Either there's nothing listening on host 192.168.1.130, port 6379 (well, at a guess) or your firewall configuration is blocking the connection. Impossible to say which, since all the code is really seeing is “the connection didn't work; something said ‘no’…”.

Resources