freeTDS bash: Executing sql queries in Microsoft SQL server - linux

I am able to connect to a Microsoft SQL Server 2008 instance via a Mint Linux VM using freeTSD and command line to execute sql statements on it. Now I want automate this in a bash script. I am able to successfully login in my bash script:
TDSVER=8.0 tsql -H servername -p 1433 -D dbadmin -U domain\\Administrator -P password
I then have my SQL query:
USE dbname GO delete from schema.tableA where ID > 5 GO delete from schema.tableB where ID > 5 GO delete from schema.tableC where ID > 5 GO exit
This works when doing manually via freeTSD command line, but not when I put in bash file. I followed this post: freeTSD & bash.
Here is my bash script sample:
echo "USE dbname GO delete from schema.tableA where userid > 5 go delete from schema.tableB where userid > 5 go delete from schema.tableC where ID > 5 GO exit" > tempfile | TDSVER=8.0 tsql -H servername -p 1433 -D dbname -U domain\\Administrator -P password < tempfile
the output of the bash script is:
locale is "en_US.UTF-8"
locale charset is "UTF-8"
Default database being set to sbdb
1> 2> 3> 4> 5> 6> 7> 8>
and then the rest of my script is executed.
Can someone give me a step by step answer to my problem ?

I'm not sure how your sample can work at all.
Here is my bash script sample:
echo "USE dbname .... exit" > tempfile | TDSVER=8.0 tsql -H servername -p 1433 -D dbname -U domain\\Administrator -P password < tempfile
# ------------------------------------^^^^ ---- pipe char?
Try using a ';' char.
echo "USE dbname .... exit" > tempfile ; TDSVER=8.0 tsql -H servername -p 1433 -D dbname -U domain\\Administrator -P password < tempfile
# ------------------------------------^^^^ ---- semi-colon
Better yet, use shell's "here documents".
TDSVER=8.0 tsql -H servername -p 1433 -D dbname -U domain\\Administrator -P password <<EOS
USE dbname
GO
delete from schema.tableA where userid > 5
go
delete from schema.tableB where userid > 5
go
delete from schema.tableC where ID > 5
GO
exit
EOS
IHTH.
Current command line input:
echo "delete from table where userid > 5
go
delete from table where userid > 5
go
delete from table where ID > 5
GO
exit" < /tmp/tempfile; TDSDUMP=/tmp/freetds.log TDSVER=8.0 tsql -H servername -p 1433 -D dbname -U Administrator -P password <<EOS

Old thread but this seemed to work..
printf "use mydbname\ngo\nselect * from mytable\ngo\nexit\n"|tsql -I freetds.conf -S profileName -U user -P 'password'
1> 2> 1> 2> ID stringtest integertest
1 test 50
2 teststring2 60
3 test3 70
(3 rows affected)

try
echo "USE dbname\n GO\n delete from schema.tableA where ID > 5\n GO\n delete from schema.tableB userid > 5\n go\n delete from schema.tableC where ID > 5\n GO\n exit\n"
the rest of this string is stuff that maybe works
and try
echo "USE dbname;\n delete from schema.tableA where ID > 5;\n delete from schema.tableB userid > 5;\n delete from schema.tableC where ID > 5;\n exit\n"
and try
echo "USE dbname; delete from schema.tableA where ID > 5; delete from schema.tableB userid > 5; delete from schema.tableC where ID > 5; exit"
if you are using odbc, i recommend the second trial.
if you are sending commands to sql with a "go" word as sql sentences separator, maybe the first one is better.
maybe the third one... who knows... only trial and error can tell...

Related

No operator matches the given name and argument types. You might need to add explicit

I am trying to get values from postgresql DB using the bash command line : There is an issue trying to get run a select statement from the table.
For example if I execute this select statement, it return successful and gives the value
psql -U postgres -d postgres -p 5432 -t -c "select count(*) from sampledata.sif_work where servicerequesttype='CreatepostgresCase'"
However when I tried to add more where statement either hardcoded or variables to the WHERE statement, I got this error :
ERROR: operator does not exist: character varying <> integer
LINE 1: ...questtype='CreatepostgresCase' and applicationerrorcode!=25 and a...
^
HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
The script :
#!/bin/bash
errorCodeSuccess=0
errorCodeFailure=30
sampleDbUser=postgres
sampleDBPort=5432
appErrorCodeFailure=25
#hardcoded
psql -U postgres -d postgres -p 5432 -t -c "select count(*) from sampledata.sif_work where servicerequesttype='CreatepostgresCase' and applicationerrorcode=25 and pxcreatedatetime>current_date"
#variables used
psql -U "${sampleDbUser}" -d postgres -p "${sampleDBPort}" -t -c "select count(*) from sampledata.sif_work where servicerequesttype='CreatepostgresCase' and applicationerrorcode!="${appErrorCodeFailure}" and applicationerrorcode!="${errorCodeSuccess}" and pxcreatedatetime>current_date"
Any reason why even though I hardcoded the value, it is still throwing error. Any reason ?
PostgreSQL understands 25 as an integer literal but '25' will be interpreted as a text literal/string constant, which would work with your character varying type column.
You could add the single quote ' before you close and after you open the double quotes ", but you also don't need to close that double-quoted string at all - bash evaluates $ expressions in double quotes:
errorCodeSuccess=0
errorCodeFailure=30
sampleDbUser=postgres
sampleDBPort=5432
appErrorCodeFailure=25
#hardcoded
psql -U postgres -d postgres -p 5432 -t \
-c "select count(*)
from sampledata.sif_work
where servicerequesttype='CreatepostgresCase'
and applicationerrorcode='25'--single quotes indicate a text literal
and pxcreatedatetime>current_date"
#variables used
psql -U "${sampleDbUser}" -d postgres -p "${sampleDBPort}" -t \
-c "select count(*)
from sampledata.sif_work
where servicerequesttype='CreatepostgresCase'
and applicationerrorcode!='${appErrorCodeFailure}'
and applicationerrorcode!='${errorCodeSuccess}'
and pxcreatedatetime>current_date; "
You already knew you can safely use single quotes within a double-quoted string, looking at servicerequesttype='CreatepostgresCase'.
You can also make the single quotes a part of the value:
#already doesn't work:
errorCodeSuccess=0
#same effect:
errorCodeSuccess='0'
#this will be interpreted as a column named "0":
errorCodeSuccess='"0"'
#"0" would be a valid name, but I doubt you have one or want one
#this will work:
errorCodeSuccess="'0'"
errorCodeFailure="'30'"
sampleDbUser=postgres
sampleDBPort=5432
psql -U "${sampleDbUser}" -d postgres -p "${sampleDBPort}" -t \
-c "select count(*)
from sampledata.sif_work
where servicerequesttype='CreatepostgresCase'
and applicationerrorcode != ${appErrorCodeFailure}
and applicationerrorcode != ${errorCodeSuccess}
and pxcreatedatetime>current_date; "
Keep in mind that it's always unsafe to construct queries this way - both in terms of security and convenience. You could start improving this with psql -v.

Bash script with multiline heredoc doesn't output anything

I'm writing a script to send SQL output in mail, but it is not executing successfully and is not generating the output I want.
The query generates two columns with multiple rows. How can I generate the output in table format as below?
Below is my code:
#!/bin/bash
ORACLE_HOME= **PATH
export ORACLE_HOME
PATH=$PATH:$ORACLE_HOME/bin
export PATH
TNS_ADMIN= ** PATH
export TNS_ADMIN
today=$(date +%d-%m-%Y)
output=$(sqlplus -S user/pass#service <<EOF
set heading off;
SELECT distinct list_name ,max(captured_dttm) as Last_received FROM db.table1
group by list_name having max(captured_dttm) <= trunc(sysdate - interval '2' hour);
EOF)
if [ -z "$output" ];
then
echo"its fine"
exit
else
echo "
Dear All,
Kindly check we've not received the list for last 2 hour : $output
Regards,
Team" | mailx -S smtp=XX.XX.X.XX:XX -s "URGENT! Please check list FOR $today" user#abc.com
fi
When using a here document, the closing string can't be followed by anything but a newline. Move the closing parenthesis to the next line:
output=$(sqlplus -S user/pass#service <<EOF
...
EOF
)

PostgreSQL - Inserting into a columns whose name is a keyword using shell script

I have a table named "myTable" in my PostgreSQL database which has 4 columns - id, non_keyword_columnA, non_keyword_columnB and group.
Structure of the table is as follows:
Column | Type | Modifiers
--------------------+------------+------------------------------------------
id | integer | not null default nextval('myTable_id_seq'::regclass)
non_keyword_columnA | integer |
non_keyword_columnB | integer |
group | integer | not null
Foreign-key constraints:
"tablename_group_fkey" FOREIGN KEY ("group") REFERENCES groups(id)
I want to insert data into this table using shell and i am using following code to do it:
sudo /usr/sbin/chroot environment_path_here su - username -c "psql -A -z -c \"INSERT INTO myTable (non_keyword_columnA ,non_keyword_columnB ,"group") VALUES (1,2,(SELECT id from groups WHERE name='someGroupName'));\""
I am not an expert in Databases but I understand that group is a keyword which can be used in psql queries if used with double quotes which i did in above script.
But receive error as
ERROR: syntax error at or near "group"
LINE 1: ...RT INTO myTable(entity,machine,group) VAL...
^`
If I enter the environment manually and then execute the psql query then query executes successfully and row gets inserted but nothing is working through shell script.
I have tried various permutations and combinations to try as escape group keyword by using these combinations:
sudo /usr/sbin/chroot environment_path_here su - username -c "psql -A -z -c \"INSERT INTO myTable (non_keyword_columnA ,non_keyword_columnB ,\""group"\") VALUES (231,3355,(SELECT id from groups WHERE name='releventGroupName'));\""
sudo /usr/sbin/chroot environment_path_here su - username -c "psql -A -z -c \"INSERT INTO myTable (non_keyword_columnA ,non_keyword_columnB ,"\"group\"") VALUES (231,3355,(SELECT id from groups WHERE name='releventGroupName'));\""
But none of them have worked till now. I am not an expert in shell either so it may be possible that I might be making some really silly mistake here. Any help would be appreciated.
Use a here-document as standard input for your psql.
(Untested:)
#!/bin/sh
(sudo /usr/sbin/chroot environment_path_here su - username -c 'psql -A -z' ) <<OMG
INSERT INTO myTable (non_keyword_columnA ,non_keyword_columnB ,"group")
SELECT 1,2,id
FROM groups
WHERE name = 'someGroupName'
;
OMG
see also...

Shell script to fetch sql query data in csv file

Need to extract the below query data along with header in csv file using shell script.
Below is the query.
SELECT SourceIdentifier,SourceFileName,ProfitCentre2,PlantCode,
tax_retur ReturnPeriod,document_number DocumentNumber,TO_CHAR(invoice_generation_date,'YYYY-MM-DD')
Docume,Original,customer_name CustomerName,NVL(sns_pos,new_state_code)POS,PortCode,NEW_HSN_CODE HSNorSAC,(SGSATE+UTGSATE) Stat,(SGS+UT)StateUT,Userde FROM arbor.INV_REPO_FINA WHERE UPPER(document_type)='INV' AND UPPER(backout_flag)='VALID' AND new_gst_id_new IS NOT NULL AND new_charges<>0 AND taxable_adj=0
UNION
SELECT SourceIdentifier,SourceFileName,ProfitCentre2,PlantCode,
tax_retur ReturnPeriod,document_number DocumentNumber,TO_CHAR(invoice_generation_date,'YYYY-MM-DD')
Docume,Original,customer_name CustomerName,NVL(sns_pos,new_state_code)POS,PortCode, NEW_HSN_CODE HSNorSAC,(SGSATE+UTGSATE) Stat,(SGS+UTG)StateUT,Userde FROM arbor.INV_REPO_FINA WHERE UPPER(document_type)='INV' AND UPPER(backout_flag)='VALID' AND new_gst_id_new IS NOT NULL AND new_charges<>0 AND taxable_adj<>0
Could please let me know if below approach to fetch data using shell script is correct and script is correct.
#!/bin/bash
file="output.csv"
sqlplus -s username/password#Oracle_SID << EOF
SPOOL $file
select 'SourceIdentifier','SourceFileName','ProfitCentre2','PlantCode',
'tax_retur ReturnPeriod','document_number DocumentNumber','TO_CHAR(invoice_generation_date,'YYYY-MM-DD') Docume','Original','customer_name CustomerName','NVL(sns_pos,new_state_code)POS','PortCode','NEW_HSN_CODE HSNorSAC','(SGSATE+UTGSATE) Stat','(SGS+UT)StateUT','Userde' from dual
Union all
select 'TO_CHAR(SourceIdentifier)','TO_CHAR(SourceFileName)','TO_CHAR(ProfitCentre2)','TO_CHAR(PlantCode)',
'TO_CHAR(tax_retur ReturnPeriod)','TO_CHAR(document_number DocumentNumber)','TO_CHAR(invoice_generation_date,'YYYY-MM-DD')
Docume','TO_CHAR(Original)','TO_CHAR(customer_name CustomerName)','TO_CHAR(NVL(sns_pos,new_state_code)POS)','TO_CHAR(PortCode)','TO_CHAR(NEW_HSN_CODE HSNorSAC)','TO_CHAR((SGSATE+UTGSATE) Stat)','TO_CHAR((SGS+UT)StateUT)','TO_CHAR(Userde)' from
(SELECT SourceIdentifier,SourceFileName,ProfitCentre2,PlantCode,
tax_retur ReturnPeriod,document_number DocumentNumber,TO_CHAR(invoice_generation_date,'YYYY-MM-DD')
Docume,Original,customer_name CustomerName,NVL(sns_pos,new_state_code)POS,PortCode,NEW_HSN_CODE HSNorSAC,(SGSATE+UTGSATE) Stat,(SGS+UT)StateUT,Userde FROM arbor.INV_REPO_FINA WHERE UPPER(document_type)='INV' AND UPPER(backout_flag)='VALID' AND new_gst_id_new IS NOT NULL AND new_charges<>0 AND taxable_adj=0
UNION
SELECT SourceIdentifier,SourceFileName,ProfitCentre2,PlantCode,
tax_retur ReturnPeriod,document_number DocumentNumber,TO_CHAR(invoice_generation_date,'YYYY-MM-DD')
Docume,Original,customer_name CustomerName,NVL(sns_pos,new_state_code)POS,PortCode, NEW_HSN_CODE HSNorSAC,(SGSATE+UTGSATE) Stat,(SGS+UTG)StateUT,Userde FROM arbor.INV_REPO_FINA WHERE UPPER(document_type)='INV' AND UPPER(backout_flag)='VALID' AND new_gst_id_new IS NOT NULL AND new_charges<>0 AND taxable_adj<>0)
SPOOL OFF
EXIT
EOF
In short: the ; is missing from the end of the select statement.
Some unrequested advice:
I think spool will put extra stuff into your file (at least some new lines), a redirect is better, further the first line is not db-related:
echo "SourceIdentifier;SourceFileName;ProfitCentre2..." > $file
I recommend to generate the csv format right in the select query, later it will be more headache (you can escape there what you want):
$query = "select SourceIdentifier || ';' || SourceFileName || ';' || ProfitCentre2 ... ;"
So querying the DB (I think capital -S is the right one) plus for the formatting of the records (and maybe you want to format your columns too):
sqlplus -S username/password#Oracle_SID >> $file << EOF
set linesize 32767 pagesize 0 heading off
$query
EOF
For me this one is working but one empty line before first query and second query is coming. Empty line remove using awk command
#!/bin/bash
FILE="A.csv"
$ORACLE_HOME/bin/sqlplus -s username/password#Oracle_SID<<EOF
SET PAGESIZE 50000 COLSEP "," LINESIZE 20000 FEEDBACK OFF HEADING off
SPOOL $FILE
select 'TYPE_OF_CALL_V','SWITCH_CALL_TYPE_V','RECORD_TYPE_V','TARF_TYPE_V' from dual;
SELECT TYPE_OF_CALL_V,SWITCH_CALL_TYPE_V,RECORD_TYPE_V,TARF_TYPE_V FROM TABLE;
SPOOL OFF
EXIT
EOF
awk 'NF > 0' $FILE > out.txt
mv out.txt $FILE

LFTP and SFTP not working using crontab

I am using below script :
script_home=/home/insetl/ppp_scripts/prod/VSAVS
date_var=`date +'%Y%m%d' -d "0 days ago"`
date_var1=`date +'%Y%m%d' -d "1 days ago"`
lftp<<END_SCRIPT
open sftp://213.239.205.74
user sftpvuclip LW?T8e62
set xfer:clobber on
get Vuclip_C2B_TRX-${date_var}0030.csv
bye
END_SCRIPT
sudo grep -v '^ACC' $script_home/Vuclip_C2B_TRX-${date_var}0030.csv
/home/insetl/ppp_scripts/prod/VSAVS/Vuclip_C2B_TRX-${date_var}.csv
scp $script_home/Vuclip_C2B_TRX-${date_var}.csv $TARGET_HOST:/tmp/
psql -h $TARGET_HOST -d $TARGET_DB -p 5432 << ENDSQL
delete from vsavs_offline_revenue_dump where update_date like '${date_var2}%';
copy vsavs_offline_revenue_dump from '/tmp/Vuclip_C2B_TRX-${date_var}.csv' with delimiter as ',';
update fact_channel_daily_aggr a
set revenue_from_file = (select sum(substr(amount,3)::float) from vsavs_offline_revenue_dump where to_char(to_timestamp(update_date, 'dd-mm-yyyy' ),'YYYYMMDD') = ${date_var1})
where product_sk = 110030 and
a.date_id=${date_var1};
ENDSQL
The script runs completely fine when run manually. However the LFTP/SFTP part of the script doesnt run when ran through crontab. Please advise the solution for the same.
Output :
/home/insetl/ppp_scripts/prod/VSAVS/sftp_pull_in.sh: line 9: 08/02/2017: No such file or directory
20170209
/home/insetl/ppp_scripts/prod/VSAVS
grep: /home/insetl/ppp_scripts/prod/VSAVS/Vuclip_C2B_TRX-201702090030.csv: No such file or directory
DELETE 0
COPY 0
UPDATE 20

Resources