When installing apache kudu in docker by executing the below command set:
KUDU_QUICKSTART_IP=$(ifconfig | grep "inet " | grep -Fv 127.0.0.1 | awk '{print $2}' | tail -1)
I get below error:
tail: option used in invalid context -- 1
How to avoid this?
Related
I am trying to use/understand xargs for printing various details for some pods I have running in kubernetes across different namespaces. E.g. this command gives:
$ kubectl get pods -A | grep Error | awk '{print $2 " -n=" $1}'
my-pod-kf8xch6y-qc6ms-k6ww2 -n=my-ns
my-pod-kf8xlg64-g0ss7-mdv1x -n=my-ns
my-pod-kldslg64-polf7-msdw3 -n=another-ns
which is correct/expected.
When I add xargs to the above command I get:
$ kubectl get pods -A | grep Error | awk '{print $2 " -n=" $1}' | xargs kubectl $1 get pod $0 -oyaml | grep phase
Error from server (NotFound): pods "bash" not found
phase: Failed
phase: Failed
Which is actually the expected output but I don't understand
Error from server (NotFound): pods "bash" not found
why is bash passed to xargs?
We suggest to avoid xargs complications and combine maximum into single awk script.
kubectl get pods -A | awk '/Error/{system("kubectl get pod "$2" -n="$1" -oyaml")}'|grep phase
Or get even more detailed results:
kubectl get pods -A | awk '/Error/{system("kubectl get pod "$2" -n="$1" -oyaml")}'|grep -E "(^ name|^ phase)
I am trying to combine the following commands:
docker ps | grep track
that will give me
6b86b28a27b0 dev/jobservice/worker-jobtracking:3.5.0-SNAPSHOT "/tini -- /startup/s…" 25 seconds ago Up 2 seconds (health: starting)
jobservice_jobTrackingWorker_1
So then, I grab the id and use it in the next request as:
docker logs 6b8 | grep -A 3 'info'
So far, the easiest way I could find was to send those commands separately, but i wonder if there would be a simple way to do it.
I think that the main issue here is that I am trying to find the name of the container based on part of its name.
So, to resume, I would like to find and store the id of a container based on its name then use it to explore its logs.
Thanks!
Perhaps there are cleaner ways to do it, but this works.
To get the ID of a partially matching container name:
$ docker ps --format "{{.ID}} {{.Names}}" | grep "partial" | cut -d " " -f1
Then you can use it in another bash command:
$ docker logs $(docker ps --format "{{.ID}} {{.Names}}" | grep "partial" | cut -d " " -f1)
Or wrap it in a function:
$ function dlog() { docker logs $(docker ps --format "{{.ID}} {{.Names}}" | grep "$1" | cut -d " " -f1); }
Which can then be used as:
$ dlog partial
In a nutshell the pure bash approach to achieve what you want:
With sudo:
sudo docker ps | grep -i track - | awk '{print $1}' | head -1 | xargs sudo docker logs
Without sudo:
docker ps | grep -i track - | awk '{print $1}' | head -1 | xargs docker logs
Now let's break it down...
Let's see what containers I have running in my laptop for the Elixir programming language:
command:
sudo docker ps | grep -i elixir -
output:
0a19c6e305a2 exadra37/phoenix-dev:1.5.3_elixir-1.10.3_erlang-23.0.2_git "iex -S mix phx.serv…" 7 days ago Up 7 days 127.0.0.1:2000-2001->2000-2001/tcp Projects_todo-tasks_app
65ef527065a8 exadra37/st3-3211-elixir:latest "bash" 7 days ago Up 7 days SUBL3_1600981599
232d8cfe04d5 exadra37/phoenix-dev:1.5.3_elixir-1.10.3_erlang-23.0.2_git "mix phx.server" 8 days ago Up 8 days 127.0.0.1:4000-4001->4000-4001/tcp Staging_todo-tasks_app
Now let's find their ids:
command:
sudo docker ps | grep -i elixir - | awk '{print $1}'
output:
0a19c6e305a2
65ef527065a8
232d8cfe04d5
Let's get the first container ID:
command:
sudo docker ps | grep -i elixir - | awk '{print $1}' | head -1
NOTE: replace head -1 with head -2 to get the second line in the output...
output:
0a19c6e305a2
Let's see the logs for the first container in the list
command:
sudo docker ps | grep -i elixir - | awk '{print $1}' | head -1 | xargs sudo docker logs
NOTE: replace head -1 with tail -1 to get the logs for the last container in the list.
output:
[info] | module=WebIt.Live.Calendar.Socket function=mount/1 line=14 | Mount Calendar for date: 2020-09-30 23:29:38.229174Z
[debug] | module=Tzdata.ReleaseUpdater function=poll_for_update/0 line=40 | Tzdata polling for update.
[debug] | module=Tzdata.ReleaseUpdater function=poll_for_update/0 line=44 | Tzdata polling shows the loaded tz database is up to date.
Combining the different replies, I used:
function dlog() { docker ps | grep -i track - | awk '{print $1}' | head -1 | xargs docker logs | grep -i -A 4 "$2";}
to get the best of both worlds. So I can have a function that will have me type 4 letters instead of 2 commands and with no case sensitivity
I can then use dlog keyword to get my logs.
I hardcoded track and -A 4 as I usually use that query but I could have passed them as arguments to add on modularity (my goal here was really simplicity)
Thanks for your help!
The following local command on host xyz provides the following correct output
taskset -p `ps -ef | grep ripit | grep -v grep| awk '{print \$2}'`
pid 21352's current affinity mask: 1
When I run the following command and ssh to xyz host I also get correct output
ssh xyz "ps -ef | grep ripit | grep -v grep |awk '{print \$2}'"
21352
However When I add the taskset command and run remotely on host xyz host i get this incorrect output.
ssh xyz "taskset -p `ps -ef | grep ripit | grep -v grep | awk '{print \$2}'`"
sched_getaffinity: No such process
failed to get pid 27599's affinity
bash: line 1: 32127: command not found
I tried many different single and double quote combination and I used escape character all over the place to no avail. Can anyone help?
Thanks
I haven't tested with your exact commands, but
ssh host 'lsof -p $(pgrep program)'
worked for me
For running commands remotely:
#!/bin/bash
SCRIPT='
#Your commands
'
sshpass -p<pass> ssh -o 'StrictHostKeyChecking no' -p <port> user#host "$SCRIPT"
When I add the taskset command and run remotely on host xyz host
ssh xyz "taskset -p `ps -ef | grep ripit | grep -v grep | awk '{print \$2}'`"
Here, the command substitution between `` is executed on the local host and yields a local process ID - no wonder that there is No such process on the remote host. If you escape the backquotes like
ssh xyz "taskset -p \`ps -ef | grep ripit | grep -v grep | awk '{print \$2}'\`"
the command substitution is executed on the remote host and yields the correct process ID.
Running Apache and Jboss on Linux, sometimes my server halts unexpectedly saying that the problem was Too Many Open Files.
I know that we might set a higher limit for nproc and nofile at /etc/security/limits.conf to fix the open files problem, but I am trying to get better output, such as using watch to monitor them in real-time.
With this command line I can see how many open files per PID:
lsof -u apache | awk '{print $2}' | sort | uniq -c | sort -n
Output (Column 1 is # of open files for the user apache):
1 PID
1335 13880
1389 13897
1392 13882
If I could just add the watch command it would be enough, but the code below isn't working:
watch lsof -u apache | awk '{print $2}' | sort | uniq -c | sort -n
You should put the command insides quotes like this:
watch 'lsof -u apache | awk '\''{print $2}'\'' | sort | uniq -c | sort -n'
or you can put the command into a shell script like test.sh and then use watch.
chmod +x test.sh
watch ./test.sh
This command will tell you how many files Apache has opened:
ps -A x |grep apache | awk '{print $1}' | xargs -I '{}' ls /proc/{}/fd | wc -l
You may have to run it as root in order to access the process fd directory. This sounds like you've got a web application which isn't closing its file descriptors. I would focus my efforts on that area.
I am looking for a way to log and graphically display cpu and RAM usage of linux processes over time. Since I couldn't find a simple tool to so (I tried zabbix and munin but installation failed) I started writing a shell script to do so
The script file parses the output of top command through awk and logs into a csv file. It
Figures out the pid of the processes through ps command
Uses top and awk to log cpu and memory usage.
Here is how the script looks like
#!/bin/sh
#A script to log the cpu and memory usage of linux processes namely - redis, logstash, elasticsearch and kibana
REDIS_PID=$(ps -ef | grep redis | grep -v grep | awk '{print $2}')
LOGSTASH_PID=$(ps -ef | grep logstash | grep -v grep | awk '{print $2}')
ELASTICSEARCH_PID=$(ps -ef | grep elasticsearch | grep -v grep | awk '{print $2}')
KIBANA_PID=$(ps -ef | grep kibana | grep -v grep | awk '{print $2}')
LOG_FILE=/var/log/user/usage.log
echo $LOG_FILE
top -b | awk -v redis="$REDIS_PID" -v logstash="$LOGSTASH_PID" '/redis|logstash/ {print $1","$9","$10","$12}'
How do I
Print the resource usage for multiple processes. Specifying multiple
variables in the awk pattern is not working. It prints the usage for
the first pid (redis in the above script)
Print current timestamp when printing the resource details (through date +"%T")
Print the process name along with the resource usage. Redis, Logstash, ElasticSearch or Kibana in the above case
Redirect the above commands output to a log file. I tried > $LOG_FILE but it didn't work.
Thoughts/Inputs?
Thanks in advance.
To figure out PIDs you can simplify your script greatly using pgrep:
REDIS_PID=$(pgrep -f redis)
LOGSTASH_PID=$(pgrep -f logstash)
ELASTICSEARCH_PID=$(pgrep -f elasticsearch)
KIBANA_PID=$(pgrep -f kibana)
EDIT: Sorry had to leave for some work and couldn't provide the full answer.
In order to capture top's output use following script:
while :; do
top -n 1 -b | awk -v redis="$REDIS_PID" -v logstash="$LOGSTASH_PID"
'$1 == redis || $1 == logstash {print $1","$9","$10","$12}' >> $LOG_FILE
sleep 3
done