I am trying to get values from postgresql DB using the bash command line : There is an issue trying to get run a select statement from the table.
For example if I execute this select statement, it return successful and gives the value
psql -U postgres -d postgres -p 5432 -t -c "select count(*) from sampledata.sif_work where servicerequesttype='CreatepostgresCase'"
However when I tried to add more where statement either hardcoded or variables to the WHERE statement, I got this error :
ERROR: operator does not exist: character varying <> integer
LINE 1: ...questtype='CreatepostgresCase' and applicationerrorcode!=25 and a...
^
HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
The script :
#!/bin/bash
errorCodeSuccess=0
errorCodeFailure=30
sampleDbUser=postgres
sampleDBPort=5432
appErrorCodeFailure=25
#hardcoded
psql -U postgres -d postgres -p 5432 -t -c "select count(*) from sampledata.sif_work where servicerequesttype='CreatepostgresCase' and applicationerrorcode=25 and pxcreatedatetime>current_date"
#variables used
psql -U "${sampleDbUser}" -d postgres -p "${sampleDBPort}" -t -c "select count(*) from sampledata.sif_work where servicerequesttype='CreatepostgresCase' and applicationerrorcode!="${appErrorCodeFailure}" and applicationerrorcode!="${errorCodeSuccess}" and pxcreatedatetime>current_date"
Any reason why even though I hardcoded the value, it is still throwing error. Any reason ?
PostgreSQL understands 25 as an integer literal but '25' will be interpreted as a text literal/string constant, which would work with your character varying type column.
You could add the single quote ' before you close and after you open the double quotes ", but you also don't need to close that double-quoted string at all - bash evaluates $ expressions in double quotes:
errorCodeSuccess=0
errorCodeFailure=30
sampleDbUser=postgres
sampleDBPort=5432
appErrorCodeFailure=25
#hardcoded
psql -U postgres -d postgres -p 5432 -t \
-c "select count(*)
from sampledata.sif_work
where servicerequesttype='CreatepostgresCase'
and applicationerrorcode='25'--single quotes indicate a text literal
and pxcreatedatetime>current_date"
#variables used
psql -U "${sampleDbUser}" -d postgres -p "${sampleDBPort}" -t \
-c "select count(*)
from sampledata.sif_work
where servicerequesttype='CreatepostgresCase'
and applicationerrorcode!='${appErrorCodeFailure}'
and applicationerrorcode!='${errorCodeSuccess}'
and pxcreatedatetime>current_date; "
You already knew you can safely use single quotes within a double-quoted string, looking at servicerequesttype='CreatepostgresCase'.
You can also make the single quotes a part of the value:
#already doesn't work:
errorCodeSuccess=0
#same effect:
errorCodeSuccess='0'
#this will be interpreted as a column named "0":
errorCodeSuccess='"0"'
#"0" would be a valid name, but I doubt you have one or want one
#this will work:
errorCodeSuccess="'0'"
errorCodeFailure="'30'"
sampleDbUser=postgres
sampleDBPort=5432
psql -U "${sampleDbUser}" -d postgres -p "${sampleDBPort}" -t \
-c "select count(*)
from sampledata.sif_work
where servicerequesttype='CreatepostgresCase'
and applicationerrorcode != ${appErrorCodeFailure}
and applicationerrorcode != ${errorCodeSuccess}
and pxcreatedatetime>current_date; "
Keep in mind that it's always unsafe to construct queries this way - both in terms of security and convenience. You could start improving this with psql -v.
I'm writing a script to send SQL output in mail, but it is not executing successfully and is not generating the output I want.
The query generates two columns with multiple rows. How can I generate the output in table format as below?
Below is my code:
#!/bin/bash
ORACLE_HOME= **PATH
export ORACLE_HOME
PATH=$PATH:$ORACLE_HOME/bin
export PATH
TNS_ADMIN= ** PATH
export TNS_ADMIN
today=$(date +%d-%m-%Y)
output=$(sqlplus -S user/pass#service <<EOF
set heading off;
SELECT distinct list_name ,max(captured_dttm) as Last_received FROM db.table1
group by list_name having max(captured_dttm) <= trunc(sysdate - interval '2' hour);
EOF)
if [ -z "$output" ];
then
echo"its fine"
exit
else
echo "
Dear All,
Kindly check we've not received the list for last 2 hour : $output
Regards,
Team" | mailx -S smtp=XX.XX.X.XX:XX -s "URGENT! Please check list FOR $today" user#abc.com
fi
When using a here document, the closing string can't be followed by anything but a newline. Move the closing parenthesis to the next line:
output=$(sqlplus -S user/pass#service <<EOF
...
EOF
)
I have a table named "myTable" in my PostgreSQL database which has 4 columns - id, non_keyword_columnA, non_keyword_columnB and group.
Structure of the table is as follows:
Column | Type | Modifiers
--------------------+------------+------------------------------------------
id | integer | not null default nextval('myTable_id_seq'::regclass)
non_keyword_columnA | integer |
non_keyword_columnB | integer |
group | integer | not null
Foreign-key constraints:
"tablename_group_fkey" FOREIGN KEY ("group") REFERENCES groups(id)
I want to insert data into this table using shell and i am using following code to do it:
sudo /usr/sbin/chroot environment_path_here su - username -c "psql -A -z -c \"INSERT INTO myTable (non_keyword_columnA ,non_keyword_columnB ,"group") VALUES (1,2,(SELECT id from groups WHERE name='someGroupName'));\""
I am not an expert in Databases but I understand that group is a keyword which can be used in psql queries if used with double quotes which i did in above script.
But receive error as
ERROR: syntax error at or near "group"
LINE 1: ...RT INTO myTable(entity,machine,group) VAL...
^`
If I enter the environment manually and then execute the psql query then query executes successfully and row gets inserted but nothing is working through shell script.
I have tried various permutations and combinations to try as escape group keyword by using these combinations:
sudo /usr/sbin/chroot environment_path_here su - username -c "psql -A -z -c \"INSERT INTO myTable (non_keyword_columnA ,non_keyword_columnB ,\""group"\") VALUES (231,3355,(SELECT id from groups WHERE name='releventGroupName'));\""
sudo /usr/sbin/chroot environment_path_here su - username -c "psql -A -z -c \"INSERT INTO myTable (non_keyword_columnA ,non_keyword_columnB ,"\"group\"") VALUES (231,3355,(SELECT id from groups WHERE name='releventGroupName'));\""
But none of them have worked till now. I am not an expert in shell either so it may be possible that I might be making some really silly mistake here. Any help would be appreciated.
Use a here-document as standard input for your psql.
(Untested:)
#!/bin/sh
(sudo /usr/sbin/chroot environment_path_here su - username -c 'psql -A -z' ) <<OMG
INSERT INTO myTable (non_keyword_columnA ,non_keyword_columnB ,"group")
SELECT 1,2,id
FROM groups
WHERE name = 'someGroupName'
;
OMG
see also...
Need to extract the below query data along with header in csv file using shell script.
Below is the query.
SELECT SourceIdentifier,SourceFileName,ProfitCentre2,PlantCode,
tax_retur ReturnPeriod,document_number DocumentNumber,TO_CHAR(invoice_generation_date,'YYYY-MM-DD')
Docume,Original,customer_name CustomerName,NVL(sns_pos,new_state_code)POS,PortCode,NEW_HSN_CODE HSNorSAC,(SGSATE+UTGSATE) Stat,(SGS+UT)StateUT,Userde FROM arbor.INV_REPO_FINA WHERE UPPER(document_type)='INV' AND UPPER(backout_flag)='VALID' AND new_gst_id_new IS NOT NULL AND new_charges<>0 AND taxable_adj=0
UNION
SELECT SourceIdentifier,SourceFileName,ProfitCentre2,PlantCode,
tax_retur ReturnPeriod,document_number DocumentNumber,TO_CHAR(invoice_generation_date,'YYYY-MM-DD')
Docume,Original,customer_name CustomerName,NVL(sns_pos,new_state_code)POS,PortCode, NEW_HSN_CODE HSNorSAC,(SGSATE+UTGSATE) Stat,(SGS+UTG)StateUT,Userde FROM arbor.INV_REPO_FINA WHERE UPPER(document_type)='INV' AND UPPER(backout_flag)='VALID' AND new_gst_id_new IS NOT NULL AND new_charges<>0 AND taxable_adj<>0
Could please let me know if below approach to fetch data using shell script is correct and script is correct.
#!/bin/bash
file="output.csv"
sqlplus -s username/password#Oracle_SID << EOF
SPOOL $file
select 'SourceIdentifier','SourceFileName','ProfitCentre2','PlantCode',
'tax_retur ReturnPeriod','document_number DocumentNumber','TO_CHAR(invoice_generation_date,'YYYY-MM-DD') Docume','Original','customer_name CustomerName','NVL(sns_pos,new_state_code)POS','PortCode','NEW_HSN_CODE HSNorSAC','(SGSATE+UTGSATE) Stat','(SGS+UT)StateUT','Userde' from dual
Union all
select 'TO_CHAR(SourceIdentifier)','TO_CHAR(SourceFileName)','TO_CHAR(ProfitCentre2)','TO_CHAR(PlantCode)',
'TO_CHAR(tax_retur ReturnPeriod)','TO_CHAR(document_number DocumentNumber)','TO_CHAR(invoice_generation_date,'YYYY-MM-DD')
Docume','TO_CHAR(Original)','TO_CHAR(customer_name CustomerName)','TO_CHAR(NVL(sns_pos,new_state_code)POS)','TO_CHAR(PortCode)','TO_CHAR(NEW_HSN_CODE HSNorSAC)','TO_CHAR((SGSATE+UTGSATE) Stat)','TO_CHAR((SGS+UT)StateUT)','TO_CHAR(Userde)' from
(SELECT SourceIdentifier,SourceFileName,ProfitCentre2,PlantCode,
tax_retur ReturnPeriod,document_number DocumentNumber,TO_CHAR(invoice_generation_date,'YYYY-MM-DD')
Docume,Original,customer_name CustomerName,NVL(sns_pos,new_state_code)POS,PortCode,NEW_HSN_CODE HSNorSAC,(SGSATE+UTGSATE) Stat,(SGS+UT)StateUT,Userde FROM arbor.INV_REPO_FINA WHERE UPPER(document_type)='INV' AND UPPER(backout_flag)='VALID' AND new_gst_id_new IS NOT NULL AND new_charges<>0 AND taxable_adj=0
UNION
SELECT SourceIdentifier,SourceFileName,ProfitCentre2,PlantCode,
tax_retur ReturnPeriod,document_number DocumentNumber,TO_CHAR(invoice_generation_date,'YYYY-MM-DD')
Docume,Original,customer_name CustomerName,NVL(sns_pos,new_state_code)POS,PortCode, NEW_HSN_CODE HSNorSAC,(SGSATE+UTGSATE) Stat,(SGS+UTG)StateUT,Userde FROM arbor.INV_REPO_FINA WHERE UPPER(document_type)='INV' AND UPPER(backout_flag)='VALID' AND new_gst_id_new IS NOT NULL AND new_charges<>0 AND taxable_adj<>0)
SPOOL OFF
EXIT
EOF
In short: the ; is missing from the end of the select statement.
Some unrequested advice:
I think spool will put extra stuff into your file (at least some new lines), a redirect is better, further the first line is not db-related:
echo "SourceIdentifier;SourceFileName;ProfitCentre2..." > $file
I recommend to generate the csv format right in the select query, later it will be more headache (you can escape there what you want):
$query = "select SourceIdentifier || ';' || SourceFileName || ';' || ProfitCentre2 ... ;"
So querying the DB (I think capital -S is the right one) plus for the formatting of the records (and maybe you want to format your columns too):
sqlplus -S username/password#Oracle_SID >> $file << EOF
set linesize 32767 pagesize 0 heading off
$query
EOF
For me this one is working but one empty line before first query and second query is coming. Empty line remove using awk command
#!/bin/bash
FILE="A.csv"
$ORACLE_HOME/bin/sqlplus -s username/password#Oracle_SID<<EOF
SET PAGESIZE 50000 COLSEP "," LINESIZE 20000 FEEDBACK OFF HEADING off
SPOOL $FILE
select 'TYPE_OF_CALL_V','SWITCH_CALL_TYPE_V','RECORD_TYPE_V','TARF_TYPE_V' from dual;
SELECT TYPE_OF_CALL_V,SWITCH_CALL_TYPE_V,RECORD_TYPE_V,TARF_TYPE_V FROM TABLE;
SPOOL OFF
EXIT
EOF
awk 'NF > 0' $FILE > out.txt
mv out.txt $FILE
I am using below script :
script_home=/home/insetl/ppp_scripts/prod/VSAVS
date_var=`date +'%Y%m%d' -d "0 days ago"`
date_var1=`date +'%Y%m%d' -d "1 days ago"`
lftp<<END_SCRIPT
open sftp://213.239.205.74
user sftpvuclip LW?T8e62
set xfer:clobber on
get Vuclip_C2B_TRX-${date_var}0030.csv
bye
END_SCRIPT
sudo grep -v '^ACC' $script_home/Vuclip_C2B_TRX-${date_var}0030.csv
/home/insetl/ppp_scripts/prod/VSAVS/Vuclip_C2B_TRX-${date_var}.csv
scp $script_home/Vuclip_C2B_TRX-${date_var}.csv $TARGET_HOST:/tmp/
psql -h $TARGET_HOST -d $TARGET_DB -p 5432 << ENDSQL
delete from vsavs_offline_revenue_dump where update_date like '${date_var2}%';
copy vsavs_offline_revenue_dump from '/tmp/Vuclip_C2B_TRX-${date_var}.csv' with delimiter as ',';
update fact_channel_daily_aggr a
set revenue_from_file = (select sum(substr(amount,3)::float) from vsavs_offline_revenue_dump where to_char(to_timestamp(update_date, 'dd-mm-yyyy' ),'YYYYMMDD') = ${date_var1})
where product_sk = 110030 and
a.date_id=${date_var1};
ENDSQL
The script runs completely fine when run manually. However the LFTP/SFTP part of the script doesnt run when ran through crontab. Please advise the solution for the same.
Output :
/home/insetl/ppp_scripts/prod/VSAVS/sftp_pull_in.sh: line 9: 08/02/2017: No such file or directory
20170209
/home/insetl/ppp_scripts/prod/VSAVS
grep: /home/insetl/ppp_scripts/prod/VSAVS/Vuclip_C2B_TRX-201702090030.csv: No such file or directory
DELETE 0
COPY 0
UPDATE 20