SQLPLUS formatting output in excel - linux

Below is the code to retrieve data from DB and output it in excel
sqlplus -s ${uidpwd} <<EOF
SET PAGESIZE 50000
SET FEEDBACK OFF
SET MARKUP HTML ON
SET NUM 24
SPOOL Extract.xls
whenever oserror exit 9
whenever sqlerror exit sql.sqlcode
#${SQL_SCRIPT}
SET SERVEROUTPUT ON
SET ECHO OFF
SET VERIFY OFF
SPOOL OFF
SET MARKUP HTML OFF
quit;
EOF
But what i see in excel report is all numbers in excel are with exponential like "5.30056E+15"
i tried below options, but none of them worked
SET NUMBERFORMAT '9999999999999999999999'
SET NUMWIDTH 20
in SQL i tried below
to_char(conf_num, '999999999999999999999999')
SELECT ' '||conf_num "Confirmation Number"
Also i have exec > ${LOGPATH}/log.``date +"%y%m%d"``.log 2>&1 in my script, Query out put is getting written in to this log file, how to avoid that? It should have only error logs.

Related

Bash script with multiline heredoc doesn't output anything

I'm writing a script to send SQL output in mail, but it is not executing successfully and is not generating the output I want.
The query generates two columns with multiple rows. How can I generate the output in table format as below?
Below is my code:
#!/bin/bash
ORACLE_HOME= **PATH
export ORACLE_HOME
PATH=$PATH:$ORACLE_HOME/bin
export PATH
TNS_ADMIN= ** PATH
export TNS_ADMIN
today=$(date +%d-%m-%Y)
output=$(sqlplus -S user/pass#service <<EOF
set heading off;
SELECT distinct list_name ,max(captured_dttm) as Last_received FROM db.table1
group by list_name having max(captured_dttm) <= trunc(sysdate - interval '2' hour);
EOF)
if [ -z "$output" ];
then
echo"its fine"
exit
else
echo "
Dear All,
Kindly check we've not received the list for last 2 hour : $output
Regards,
Team" | mailx -S smtp=XX.XX.X.XX:XX -s "URGENT! Please check list FOR $today" user#abc.com
fi
When using a here document, the closing string can't be followed by anything but a newline. Move the closing parenthesis to the next line:
output=$(sqlplus -S user/pass#service <<EOF
...
EOF
)

Excel file generation using SQL query, SPOOL command in Shell script

I have to generate excel file from tables from Oracle Database. The code is working fine but however the column names are not coming completely, there are just coming as the length of there data length.
I want complete header/column names
The column names are coming like this
ST STAT_TYPE_DESC ST S NXT_STATME DELAY_DAYS ANN_PREM_LIM_LOW ANN_PREM_LIM_HI CONTRIB_HIST_LEN EVENT_DO C P
But I want is complete names of the columns, for example ST is STATEMENT_TYPE_ID
#!/bin/ksh
FILE="A.csv"
sqlplus -s lifelite/lifelite#dv10 <<EOF
SPOOL $FILE
SET HEADING ON
SET HEADSEP OFF
SET FEEDBACK OFF
SET LINESIZE 250
SET PAGESIZE 5000 embedded ON
SET COLSEP ","
SELECT * FROM TLLI_01150STATTYPE;
EOF
SPOOL OFF
EXIT 0
Before your select add a new one with the column names:
SELECT 'STATEMENT_TYPE_ID' || ',' || 'STAT_TYPE_DESC' || ',' || ... FROM dual;
And set heading off

split pipe separated string fetched using SQL

My shell script executes a SQL to fetch the data in the following format:
JOB_ID_001|[PROD] This is a mail subject one ${application_date}|a#example.com,b#example.com
JOB_ID_002|[PROD] This is a mail subject two ${application_date}|c#example.com,b#example.com
I want to split this pipe-separated string, but the output looks very odd as follows:
JOB_ID_001[0]
JOB_ID_001[1]
JOB_ID_001[2]
This[0]
This[1]
This[2]
is[0]
is[1]
is[2]
a[0]
a[1]
a[2]
mail[0]
mail[1]
mail[2]
subject[0]
subject[1]
subject[2]
one[0]
one[1]
one[2]
${application_date}[0]
${application_date}[1]
${application_date}[2]
example.com,b#example.com[0]
example.com,b#example.com[1]
example.com,b#example.com[2]
JOB_ID_002[0]
JOB_ID_002[1]
JOB_ID_002[2]
This[0]
This[1]
This[2]
is[0]
is[1]
is[2]
a[0]
a[1]
a[2]
mail[0]
mail[1]
mail[2]
subject[0]
subject[1]
subject[2]
two[0]
two[1]
two[2]
${application_date}[0]
${application_date}[1]
${application_date}[2]
ple.com,b#example.com[0]
ple.com,b#example.com[1]
ple.com,b#example.com[2]
My desired output is:
JOB_ID_001
[PROD] This is a mail subject one ${application_date}
a#example.com,b#example.com
JOB_ID_002
[PROD] This is a mail subject two ${application_date}
c#example.com,b#example.com
So that I can continue with those strings.
My shell script is as follows:
email_configs=(`sqlplus -silent $DB_CONN <<-EOF
whenever sqlerror exit 1 oserror exit oscode
set heading off feedback off echo off verify off pagesize 0
$sql_subject_of_mail;
exit;
EOF`)
for i in "${!email_configs[#]}"
do
email_config=${email_configs[i]}
IFS='|' read -r -a email_config_array <<< "$email_config"
job_id=$email_config_array[0]
subject_of_mail=$email_config_array[1]
to_mail_id=$email_config_array[2]
echo $job_id
echo $subject_of_mail
echo $to_mail_id
done
I checked some alternate solutions from this page, but in the output ${application_date} part is missing or there is some other problem.
Can anyone have an idea about my mistake?
The $email_configs array is not being set correctly. It's using spaces as the array delimiters, not newlines.
Rather than set an array, read the output of sqlplus in a loop.
while IFS='|' read -r job_id subject_of_mail to_mail_id
do
echo "$job_id"
echo "$subject_of_mail"
echo "$to_mail_id"
done < <(sqlplus -silent $DB_CONN <<-EOF
whenever sqlerror exit 1 oserror exit oscode
set heading off feedback off echo off verify off pagesize 0
$sql_subject_of_mail;
exit;
EOF
)

Shell SQLPLUS prints CLOB data twice

I have the below code in my ksh shell script file and when i run this i get the output printed twice in the file. here the FILE_DATA is of type CLOB
sqlplus -s ${uidpwd} > ${DATA_FILE_NAME} << EOF
set head off
SET pagesize 0
SET LINESIZE 32767
SET LONGCHUNKSIZE 32767
SET LONG 320000000
SET RECSEP OFF
set SERVEROUTPUT off
SET ECHO off
SET TRIMOUT ON
SET TRIMSPOOL ON
set termout off
set feedback off verify off heading off newpage 0
whenever OSERROR EXIT 9
whenever SQLERROR EXIT sql.SQLCODE
SELECT FILE_DATA FROM FILES WHERE ID='208';
/
exit;
EOF
Data:
"Client_id,Client_id_type,Client_Segment
1,LOCAL_ID,Individual"

Shell script to fetch sql query data in csv file

Need to extract the below query data along with header in csv file using shell script.
Below is the query.
SELECT SourceIdentifier,SourceFileName,ProfitCentre2,PlantCode,
tax_retur ReturnPeriod,document_number DocumentNumber,TO_CHAR(invoice_generation_date,'YYYY-MM-DD')
Docume,Original,customer_name CustomerName,NVL(sns_pos,new_state_code)POS,PortCode,NEW_HSN_CODE HSNorSAC,(SGSATE+UTGSATE) Stat,(SGS+UT)StateUT,Userde FROM arbor.INV_REPO_FINA WHERE UPPER(document_type)='INV' AND UPPER(backout_flag)='VALID' AND new_gst_id_new IS NOT NULL AND new_charges<>0 AND taxable_adj=0
UNION
SELECT SourceIdentifier,SourceFileName,ProfitCentre2,PlantCode,
tax_retur ReturnPeriod,document_number DocumentNumber,TO_CHAR(invoice_generation_date,'YYYY-MM-DD')
Docume,Original,customer_name CustomerName,NVL(sns_pos,new_state_code)POS,PortCode, NEW_HSN_CODE HSNorSAC,(SGSATE+UTGSATE) Stat,(SGS+UTG)StateUT,Userde FROM arbor.INV_REPO_FINA WHERE UPPER(document_type)='INV' AND UPPER(backout_flag)='VALID' AND new_gst_id_new IS NOT NULL AND new_charges<>0 AND taxable_adj<>0
Could please let me know if below approach to fetch data using shell script is correct and script is correct.
#!/bin/bash
file="output.csv"
sqlplus -s username/password#Oracle_SID << EOF
SPOOL $file
select 'SourceIdentifier','SourceFileName','ProfitCentre2','PlantCode',
'tax_retur ReturnPeriod','document_number DocumentNumber','TO_CHAR(invoice_generation_date,'YYYY-MM-DD') Docume','Original','customer_name CustomerName','NVL(sns_pos,new_state_code)POS','PortCode','NEW_HSN_CODE HSNorSAC','(SGSATE+UTGSATE) Stat','(SGS+UT)StateUT','Userde' from dual
Union all
select 'TO_CHAR(SourceIdentifier)','TO_CHAR(SourceFileName)','TO_CHAR(ProfitCentre2)','TO_CHAR(PlantCode)',
'TO_CHAR(tax_retur ReturnPeriod)','TO_CHAR(document_number DocumentNumber)','TO_CHAR(invoice_generation_date,'YYYY-MM-DD')
Docume','TO_CHAR(Original)','TO_CHAR(customer_name CustomerName)','TO_CHAR(NVL(sns_pos,new_state_code)POS)','TO_CHAR(PortCode)','TO_CHAR(NEW_HSN_CODE HSNorSAC)','TO_CHAR((SGSATE+UTGSATE) Stat)','TO_CHAR((SGS+UT)StateUT)','TO_CHAR(Userde)' from
(SELECT SourceIdentifier,SourceFileName,ProfitCentre2,PlantCode,
tax_retur ReturnPeriod,document_number DocumentNumber,TO_CHAR(invoice_generation_date,'YYYY-MM-DD')
Docume,Original,customer_name CustomerName,NVL(sns_pos,new_state_code)POS,PortCode,NEW_HSN_CODE HSNorSAC,(SGSATE+UTGSATE) Stat,(SGS+UT)StateUT,Userde FROM arbor.INV_REPO_FINA WHERE UPPER(document_type)='INV' AND UPPER(backout_flag)='VALID' AND new_gst_id_new IS NOT NULL AND new_charges<>0 AND taxable_adj=0
UNION
SELECT SourceIdentifier,SourceFileName,ProfitCentre2,PlantCode,
tax_retur ReturnPeriod,document_number DocumentNumber,TO_CHAR(invoice_generation_date,'YYYY-MM-DD')
Docume,Original,customer_name CustomerName,NVL(sns_pos,new_state_code)POS,PortCode, NEW_HSN_CODE HSNorSAC,(SGSATE+UTGSATE) Stat,(SGS+UTG)StateUT,Userde FROM arbor.INV_REPO_FINA WHERE UPPER(document_type)='INV' AND UPPER(backout_flag)='VALID' AND new_gst_id_new IS NOT NULL AND new_charges<>0 AND taxable_adj<>0)
SPOOL OFF
EXIT
EOF
In short: the ; is missing from the end of the select statement.
Some unrequested advice:
I think spool will put extra stuff into your file (at least some new lines), a redirect is better, further the first line is not db-related:
echo "SourceIdentifier;SourceFileName;ProfitCentre2..." > $file
I recommend to generate the csv format right in the select query, later it will be more headache (you can escape there what you want):
$query = "select SourceIdentifier || ';' || SourceFileName || ';' || ProfitCentre2 ... ;"
So querying the DB (I think capital -S is the right one) plus for the formatting of the records (and maybe you want to format your columns too):
sqlplus -S username/password#Oracle_SID >> $file << EOF
set linesize 32767 pagesize 0 heading off
$query
EOF
For me this one is working but one empty line before first query and second query is coming. Empty line remove using awk command
#!/bin/bash
FILE="A.csv"
$ORACLE_HOME/bin/sqlplus -s username/password#Oracle_SID<<EOF
SET PAGESIZE 50000 COLSEP "," LINESIZE 20000 FEEDBACK OFF HEADING off
SPOOL $FILE
select 'TYPE_OF_CALL_V','SWITCH_CALL_TYPE_V','RECORD_TYPE_V','TARF_TYPE_V' from dual;
SELECT TYPE_OF_CALL_V,SWITCH_CALL_TYPE_V,RECORD_TYPE_V,TARF_TYPE_V FROM TABLE;
SPOOL OFF
EXIT
EOF
awk 'NF > 0' $FILE > out.txt
mv out.txt $FILE

Resources