I have an impala query in a file and I want to open a screen for this query.
The query is like
select *
from db.test
where date >20201101
I will run the file with impala-shell -f test.sql
Also is it possible to generate a file with the execution time of the screen after?
Yes, this is possible using some more shell scripting.
impala-shell -k --ssl -i xx:100 -q "select * from table;" >logfile.txt 2>&1
grep "Fetched" logfile.txt | cut -d " " -f 5 > Run_info.txt #for Select,Create
grep "Modified" logfile.txt | cut -d " " -f 5 >>Run_info.txt #for Insert
Output will be 0.11s
You can use this to run all your scripts in loop. If you want you can see how many records fetched/modified too from logfile. You just have to tweak the grep command.
Related
I have use command to get response time :
curl -s -w "%{time_total}\n" -o /dev/null https://www.ziffi.com/suggestadoc/js/ds.ziffi.https.v308.js
and I also need download time of this below mentioned js file link so used wget command to download this file but i get multiple parameter out put. I just need download time from it
$ wget --output-document=/dev/null https://www.ziffi.com/suggestadoc/js/ds.ziffi.https.v307.js
please suggest
I think what you are looking for is this:
wget --output-document=/dev/null https://www.ziffi.com/suggestadoc/js/ds.ziffi.https.v307.js 2>&1 >/dev/null | grep = | awk '{print $5}' | sed 's/^.*\=//'
Explanation:
2>&1 >/dev/null | --> Makes sure stderr gets piped instead of stdout
grep = --> select the line that contains the '=' symbol
sed 's/^.*\=//' --> deletes everything from linestart to the = symbol
I am trying to capture the output of "sort -c" in linux. I tried redirecting it to a file, used tee command but both did not helped. Any suggestions ?
For example:
roor>cat db | sort -c
sort: -:319: disorder: 1842251880: aa bb bc dd ee
Following failed to give me output
roor>cat db | sort -c > fileName
roor>cat db | sort -c |tee fileName
Sample file:
>cat file
111 aa as sdasd
222 sadf dzfasf af
333 sada gvsdgf h hgfhfghfg
444 asdfafasfa gsdgsdg sgsg
222 asdasd fasdfaf asdasdasd
root>cat file |sort -c
sort: -:5: disorder: 222 asdasd fasdfaf asdasdasd
8>sort -c db 2> fileName
sort: extra operand `2' not allowed with -c
0>sort -c < file 2> result1.txt
sort: open failed: 2: No such file or directory
ANY ALTERNATE TO SORT -C would ALSO WORK FOR ME!!
If sort -c is producing an error, it sends that error to "standard error" (stderr), not to "standard output" (stdout).
In shell, you need to use special redirects to capture standard error.
sort -c inputfile > /path/to/stdout.txt 2> /path/to/stderr.txt
These two output streams are called "file descriptors", and you can alternately redirect one of them to the other:
sort -c inputfile > /path/to/combined.txt 2>&1
You can read more about how these work at tldp.org, in the Bash reference manual, the bash-hackers wiki and the Bash FAQ. Happy reading! :-D
other good alternative for '2>' is STDERR pipe
|&
cat db | sort -c -h |& tee >fileName
Some time it is very suitable when present STDIN, for example:
TIMEFORMAT=%R;for i in `seq 1 20` ; do time kubectl get pods -l app=pod >/dev/null ; done |& sort -u -h
or
TIMEFORMAT=%R;for i in `seq 1 20` ; do time kubectl get pods >>log1 ; done |& sort -u -h
sort -c has no output as you might expect:
[root#home:~] cat /etc/services | sort -c
sort: -:2: disorder: #
As described by the manpage, the -c argument simply checks whether a given file or input is sorted or not.
If you're trying to catch the message from the command, try redirecting the error stream (2), not the standard output (1):
cat file | sort -c 2>result.txt
sort -conly checks if the input is sorted. It does not performs any sorting.
See: man sort
Remove -c to sort the lines.
PS: It gives the "disorder" error because the file of yours isn't already sorted. On line 5, "222" appears after "444" on the previous line.
EDIT: I think I misunderstood.
To redirect the error to a file you must use 2>.
So, the command would become: roor>cat db | sort -c 2> fileName
EDIT2: You can simply use: sort -c db 2> fileName
Hi I have a shell script that inserts disk status of linux servers in SQL server. Before inserting the SQL command, the program executes and ends properly. However, when I inserted the SQL command, the .sh file never goes to the next line to execute. It is always in a loop. Kindly help
#!/bin/bash
#Functions here
insert() {
echo "--INSERT FUNCTION--"
echo "$1"
fsname=$1
fs=$(echo "${fsname: -3}")
sqlcmd -S <ipadd> -U <user> -P <pass> -d tech_admin -Q "EXEC insertDiskStatus $fs"
sleep 1
}
echo "TEST"
cd ~/Documents
pwd
df -Ph --exclude-type=tmpfs --exclude-type=ext3 --block-size=GB | column -t | sed 1d > diskspace.log
filename=diskspace.log
while read -r line
do
this=$line
fs=$(echo $line | awk '{print $1}')
insert $fs
done < "$filename"
I had the same problem. In my bash script, I have sqlcmd in a while loop and the loop would continue running the first item in the list over and over and not continue on to the next item. As mentioned in the comments, adding this < /dev/null to the end of sqlcmd line will cause it to get out of stuck spot in the loop and carry on to the next tiem.
I'm completely new to Netezza. I've connected to Netezza server through a putty access and need to run an nzsql command in the Linux terminal but when I give nzsql, it says command not found. Can someone tell me how to get started with nzsql and execute queries ?
Thanks in advance
You need to install NzClient to run nzsql from staging machine, Please read following link -
http://bajajvarun.blogspot.in/2014/02/install-netezza-client-on-ubuntu.html
Most likely the nzsql command is not on your path.
http://pic.dhe.ibm.com/infocenter/ntz/v7r0m3/index.jsp?topic=%2Fcom.ibm.nz.adm.doc%2Fr_sysadm_nzsql_command.html indicates the location of the commands, so if you are on the Netezza host the command is expected to be in /nz/kit/bin.
Does typing "/nz/kit/bin/nzsql" find the command? If so, add that directory to your path. If not, check with someone who can run the command to see what "which nzsql" shows, and add that directory to your path.
If you want the nzsql commands then try something like:
nzsql -host -d -u -pw -c -c "select * from tablename" -o /root/home/outputfilename.txt;
nzsql -host -d -u -pw -c "select * from tablename" -F "|" -A -q -t | gzip > /root/home/outputfilename.txt.gz;
nzsql -host -d -u -pw -c 'insert into tablename values (1 ,2 )' -o /root/home/outputfilename.txt;
http://dwbitechguru.blogspot.ca/2014/11/extract-load-migrate-filesdata-from.html
or use them from unix scripts:
# Unix Script to Drop & Truncate a Netezza tables
#!/bin/sh
# enter your database name and table name in below
dbname=exampledb
tblname=exampletbl
echo "Dropping table $i"
# use below line to drop a table
nzsql $dbanme -c "drop table $tblname"
# use below line to truncate a table
nzsql $dbanme -c "truncate table $tblname"
http://dwbitechguru.blogspot.ca/2014/12/how-to-create-unix-script-to-drop.html
I'm using tsql (installed with FreeTDS) and I'm wondering if there's a way to run a SQL script in tsql from the command line and get the result in a text file.
For example, in psql I can do:
psql -U username -C "COPY 'SELECT * FROM some_table' to 'out.csv' with csv header"
Or:
psql -U username -C "\i script.sql"
And in script.sql do:
\o out.csv
SELECT * FROM some_table;
Is there a way for doing this in tsql? I have read the linux man page and search everywhere but I just don't find a way.
I think, you can try "bsqldb", see http://linux.die.net/man/1/bsqldb
I really didn't find how to do this and I'm starting to thinks it just can't be done in tsql. However I solved for my specific problem redirecting the stdin and stdout. I use a bash script like this:
tsql connection parameters < tsql_input.sql > tsql_output.csv
lines=`cat tsql_output.csv | wc -l`
# The output includes the header info of tsql and the "(n number of rows)" message
# so I use a combination of head and tail to select the lines I need
headvar=$((lines-2))
tailvar=$((lines-8))
head -$headvar tsql_output.csv | tail -$tailvar tsql_output.csv > new_output_file.csv
And that saves just the query result on 'new_output_file.csv'
freebcp "select * from mytable where a=b" queryout sample.csv -U anoop -P mypassword -S hostname.com -D dbname -t , -c
This command woks like a charm. Kudos to FreeTDS...
tsql -U username -P password -p 1433 > out.csv <<EOS SELECT * FROM some_table; go EOS