Excel Query - invalid number of parameter and invalid descriptor index errors - excel

I work in financial deparment of a company and want to extract data from our server (SQL) to put together reports for Board.
So I have several excel file where I use excel query to retreive data and make presentations.
I have been upgrading my querys and hit an obstacle with this one.
It was working fine with this bit of code:
SELECT
INTERNAL_REFERENCE as ref,
CMP_CODE AS CMP_CODE,
COUNTERPARTY_CODE AS BANK_CODE,
TRANSACTION_CODE AS TRANS_CODE,
CAST(CONVERT(varchar(10), AMO_END_DATE, 110) AS datetime) AS DATE,
SUM([AMORTIZATION]) AS AMOUNT
FROM
[SAGE_MTC_FRP].[dbo].[LOAN_SCHEDULE] inner join [SAGE_MTC_FRP].[dbo]. [LOANS]
on [SAGE_MTC_FRP].[dbo].[LOAN_SCHEDULE].LOAN_ID=[SAGE_MTC_FRP].[dbo]. [LOANS].LOAN_ID
WHERE
(AMO_END_DATE>=?) AND (BOOK_DATE<?) AND
(TRANSACTION_CODE<>'CPCA' AND TRANSACTION_CODE<>'CPCF' AND TRANSACTION_CODE<>'RENT') AND
IS_DELETED=0 AND VERSION_NUMBER=1 AND CMP_CODE='MTG'
GROUP BY
INTERNAL_REFERENCE, CMP_CODE, COUNTERPARTY_CODE, TRANSACTION_CODE, AMO_END_DATE
But when I put all code with the union all, it blows up with these 2x errors:
problem:
invalid number of parameter
invalid descriptor index
Code:
SELECT * FROM
(
SELECT
INTERNAL_REFERENCE as ref,
CMP_CODE AS CMP_CODE,
COUNTERPARTY_CODE AS BANK_CODE,
TRANSACTION_CODE AS TRANS_CODE,
CAST(CONVERT(varchar(10), AMO_END_DATE, 110) AS datetime) AS DATE,
SUM([AMORTIZATION]) AS AMOUNT
FROM
[SAGE_MTC_FRP].[dbo].[LOAN_SCHEDULE] inner join [SAGE_MTC_FRP].[dbo]. [LOANS]
on [SAGE_MTC_FRP].[dbo].[LOAN_SCHEDULE].LOAN_ID=[SAGE_MTC_FRP].[dbo]. [LOANS].LOAN_ID
WHERE
(AMO_END_DATE>=?) AND (BOOK_DATE<?) AND
(TRANSACTION_CODE<>'CPCA' AND TRANSACTION_CODE<>'CPCF' AND TRANSACTION_CODE<>'RENT') AND
IS_DELETED=0 AND VERSION_NUMBER=1 AND CMP_CODE='MTG'
GROUP BY
INTERNAL_REFERENCE, CMP_CODE, COUNTERPARTY_CODE, TRANSACTION_CODE, AMO_END_DATE
UNION ALL
SELECT
CL_CODE as ref
,LEFT([ACC_CODE] , 3) AS CMP_CODE
,[COUNTERPARTY_CODE] AS BANK_CODE
,RIGHT([CL_DESCRIPTION] , 3) AS TRANS_CODE
,CAST(CONVERT(varchar(10), [END_DATE], 110) AS datetime) AS DATE
,[CL_AMOUNT] AS AMOUNT
FROM
[SAGE_MTC_FRP].[dbo].[CREDIT_LINES]
WHERE
(END_DATE>?) AND
RIGHT([CL_DESCRIPTION] , 3)='PPC'
) AS DATA
ORDER BY REF

If both queries run by there self, you could use something like:
DECLARE #result TABLE(
[ref] NVARCHAR(50),
[CMP_CODE] NVARCHAR(50),
[BANK_CODE] NVARCHAR(50),
[TRANS_CODE] NVARCHAR(50),
[Date] DATE,
[AMOUNT] DECIMAL(10,2)
)
INSERT INTO #result
( ref ,
CMP_CODE ,
BANK_CODE ,
TRANS_CODE ,
Date ,
AMOUNT
)
SELECT
INTERNAL_REFERENCE as ref,
CMP_CODE AS CMP_CODE,
COUNTERPARTY_CODE AS BANK_CODE,
TRANSACTION_CODE AS TRANS_CODE,
CAST(CONVERT(varchar(10), AMO_END_DATE, 110) AS datetime) AS DATE,
SUM([AMORTIZATION]) AS AMOUNT
FROM
[SAGE_MTC_FRP].[dbo].[LOAN_SCHEDULE] inner join [SAGE_MTC_FRP].[dbo]. [LOANS]
on [SAGE_MTC_FRP].[dbo].[LOAN_SCHEDULE].LOAN_ID=[SAGE_MTC_FRP].[dbo]. [LOANS].LOAN_ID
WHERE
(AMO_END_DATE>=?) AND (BOOK_DATE<?) AND
(TRANSACTION_CODE<>'CPCA' AND TRANSACTION_CODE<>'CPCF' AND TRANSACTION_CODE<>'RENT') AND
IS_DELETED=0 AND VERSION_NUMBER=1 AND CMP_CODE='MTG'
GROUP BY
INTERNAL_REFERENCE, CMP_CODE, COUNTERPARTY_CODE, TRANSACTION_CODE, AMO_END_DATE
INSERT INTO #result
( ref ,
CMP_CODE ,
BANK_CODE ,
TRANS_CODE ,
Date ,
AMOUNT
)
SELECT
CL_CODE as ref
,LEFT([ACC_CODE] , 3) AS CMP_CODE
,[COUNTERPARTY_CODE] AS BANK_CODE
,RIGHT([CL_DESCRIPTION] , 3) AS TRANS_CODE
,CAST(CONVERT(varchar(10), [END_DATE], 110) AS datetime) AS DATE
,[CL_AMOUNT] AS AMOUNT
FROM
[SAGE_MTC_FRP].[dbo].[CREDIT_LINES]
WHERE
(END_DATE>?) AND
RIGHT([CL_DESCRIPTION] , 3)='PPC'
SELECT * FROM #result ORDER BY ref

I realize this is ancient but ran across the problem today.
To test, I removed the parameter and found that it was a security issue. I granted the user execute permission to the SQL Procedure, and it worked without the parameter. When I re-introduced the parameter, it continued working.
I'm far from a security expert, so I'm sure there's a better way to grant access, but this got me past the problem for now. Feel free to suggest a better solution.

Related

cross apply an array of values recorded every 10 mins from a timestamp and generate their timestamps in stream analytics

I have the following stream analytics input:
{ "ID":"DEV-001-Test",
"TMSMUTC":"2021-10-14T14:00:00.000",
"MSGTYP":"TELEMETRY",
"THING":[
{
"TMSDUTC":"2021-10-14T13:00:00.000",
"DATA":[
{
"TAGID":"TAGB",
"VALUE":30
},
{
"TAGID":"TAGX",
"VALUE":[30.34,245.65,30.34,245.65,245.65,30.34]
}
]
}
]
}
in which the array of values for the "TAGX" is representing a value recorded from a sensor every 10 mins for one hour from the timestamp "TMSDUTC":"2021-10-14T13:00:00.000".
I was wondering how could make a query that would give me a similar output:
output
my main doubts are how to create the sequence of 10 mins from the timestamp and cross apply the values to it.
That's a good one! Note that I highly recommend you use VSCode and the ASA extension when working on these queries. The developer experience is much nicer than in the portal thanks to local testing, and you can also unit test your query via the npm package.
I took the following assumptions:
THING is an array of a single record. Let me know if that's not the case
[edited] TMSDUTC needs to be incremented by 10 minutes according to the position of each item in the array when applicable (TAGX)
With that, here is the query. It's split in multiple code blocks to explain the flow, but I also pasted it whole in the last code block.
First we bring all the required fields to the first level. It makes things easier to read, but not only. GetArrayElements needs an array to CROSS APPLY, but GetArrayElement (singular) doesn't return the type at compile time. Using an intermediary query step solves that.
WITH things AS (
SELECT
ID,
GetArrayElement(THING,0).TMSDUTC AS TMSDUTC,
MSGTYP AS MessageType,
GetArrayElement(THING,0).DATA AS DATA
FROM [input]
),
Then we expand DATA:
dataAll AS (
SELECT
T.ID,
T.TMSDUTC,
T.MessageType,
D.ArrayValue.TAGID AS Tag,
D.ArrayValue.Value AS [Value]
FROM things T
CROSS APPLY GetArrayElements(T.DATA) AS D
),
Then we create a subset for records that have a VALUE of type array (TAGX in your example). Here I avoid hard-coding per tag by detecting the type at runtime. These records will need another round of array processing in the following step.
dataArrays AS (
SELECT
A.ID,
A.TMSDUTC,
A.MessageType,
A.Tag,
A.[Value]
FROM dataAll A
WHERE GetType(A.[Value]) = 'array'
),
Now we can focus on expanding VALUE for those records. Note that we could not do that in a single pass (filter on arrays above and CROSS APPLY below), as GetArrayElements checks types before filtering is done.
[edited] To increment TMSDUTC, we use DATEADD on the index of each item in its array (ArrayIndex/ArrayValue are both returned from the array expansion, see doc below).
dataArraysExpanded AS (
SELECT
A.ID,
DATEADD(minute,10*V.ArrayIndex,A.TMSDUTC) AS TMSDUTC,
A.MessageType,
A.Tag,
V.ArrayValue AS [Value]
FROM dataArrays A
CROSS APPLY GetArrayElements(A.[Value]) AS V
),
We union back everything together:
newSchema AS (
SELECT ID, TMSDUTC, MessageType, Tag, [Value] FROM dataAll WHERE GetType([Value]) != 'array'
UNION
SELECT ID, TMSDUTC, MessageType, Tag, [Value] FROM dataArraysExpanded
)
And finally insert everything into the destination:
SELECT
*
INTO myOutput
FROM newSchema
[edited] Please note that the only order guaranteed on a result set is the one defined by the timestamp. If multiple records occur on the same timestamp, no order is guaranteed by default. Here, at the end of the query, all of the newly created events are still timestamped on the timestamp of the original event. If you now need to apply time logic on the newly generated TMSDUTC, you will need to output these records to Event Hub, and load them in another job using TIMESTAMP BY TMSDUTC. Currently the timestamp can only be changed directly at the very first step of a query.
What is used here :
GetArrayElement (singular) : doc
WITH aka Common Table Expression (CTE) : doc
CROSS APPLY + GetArrayElements : doc and doc, plus very good ref
GetType : doc
The entire thing for easier copy/pasting:
WITH things AS (
SELECT
ID,
GetArrayElement(THING,0).TMSDUTC AS TMSDUTC,
MSGTYP AS MessageType,
GetArrayElement(THING,0).DATA AS DATA
FROM [input]
),
dataAll AS (
SELECT
T.ID,
T.TMSDUTC,
T.MessageType,
D.ArrayValue.TAGID AS Tag,
D.ArrayValue.Value AS [Value]
FROM things T
CROSS APPLY GetArrayElements(T.DATA) AS D
),
dataArrays AS (
SELECT
A.ID,
A.TMSDUTC,
A.MessageType,
A.Tag,
A.[Value]
FROM dataAll A
WHERE GetType(A.[Value]) = 'array'
),
dataArraysExpanded AS (
SELECT
A.ID,
DATEADD(minute,10*V.ArrayIndex,A.TMSDUTC) AS TMSDUTC,
A.MessageType,
A.Tag,
V.ArrayValue AS [Value]
FROM dataArrays A
CROSS APPLY GetArrayElements(A.[Value]) AS V
),
newSchema AS (
SELECT ID, TMSDUTC, MessageType, Tag, [Value] FROM dataAll WHERE GetType([Value]) != 'array'
UNION
SELECT ID, TMSDUTC, MessageType, Tag, [Value] FROM dataArraysExpanded
)
SELECT
*
INTO myOutput
FROM newSchema

RedShift Correlated Sub-query

Need your help. I am trying to convert below SQL query into RedShift, but getting error message "Invalid operation: This type of correlated subquery pattern is not supported yet"
SELECT
Comp_Key,
Comp_Reading_Key,
Row_Num,
Prev_Reading_Date,
( SELECT MAX(X) FROM (
SELECT CAST(dateadd(day, 1, Prev_Reading_Date) AS DATE) AS X
UNION ALL
SELECT dim_date.calendar_date
) a
) as start_dt
FROM stage5
JOIN dim_date ON calendar_date BETWEEN '2020-04-01' and '2020-04-15'
WHERE Comp_Key =50906055
The same query works fine in SQL Server. Could you please help me to run it in RedShift?
Regards,
Kiru
Kiru - you need to convert the correlated query into a join structure. Not knowing the data content of your tables and the exact expected out put I'm just guessing but here's a swag:
SELECT
Comp_Key,
Comp_Reading_Key,
Row_Num,
Prev_Reading_Date,
Max_X
FROM stage5
JOIN dim_date ON calendar_date BETWEEN '2020-04-01' and '2020-04-15'
JOIN ( SELECT MAX(X) as Max_X, MAX(calendar_date) as date FROM (
SELECT CAST(dateadd(day, 1, Prev_Reading_Date) AS DATE) AS X FROM stage5
cross join
SELECT dim_date.calendar_date from dim_date
) a
) as start_dt ON a.date = dim_date.calendar_date
WHERE Comp_Key =50906055
This is just a starting guess but might get you started.
However, you are likely better off rewriting this query to use window functions as they are the fastest way to perform these types of looping queries in Redshift.
Thanks Bill. It won't work in RedShift as it still has correalted sub-query.
However I have modified query in another method and it works fine.
I am closing ticket.

Azure stream analyics - Compiling query failed

When I try to use the last function (https://msdn.microsoft.com/en-us/library/azure/mt421186.aspx). I get the following error:
Compiling query failed.
SELECT
deviceId
,System.TimeStamp as timestamp
,avg(events.externaltemp) as externaltemp
,LAST(System.Timestamp) OVER (PARTITION BY deviceId LIMIT DURATION(second, 1) when [externaltemp] is not null ) as Latest
INTO
[powerBI]
FROM
[EventHub] as events timestamp by [timestamp]
GROUP BY deviceId, TumblingWindow(second,1)
My last function looks very similar to the one in the msdn sample, so I'm not sure why there is a problem.
You are using [externaltemp] in your query, but it is not included in group by. That is the reason. And "last" function does not allow aggregates inside it, so below wouldn't work as well
LAST(System.Timestamp) OVER (PARTITION BY deviceId LIMIT DURATION(second, 1) when avg([externaltemp]) is not null ) as Latest
It can be achieved by splitting the query into two steps, like this
with DeviceAggregates
as
(
SELECT
System.TimeStamp as [Timestamp],
deviceId,
avg(events.externaltemp) as [externaltemp]
FROM
[EventHub] as events timestamp by [timestamp]
GROUP BY
deviceId,
TumblingWindow(second,1)
),
DeviceAggregatesWithLast as
(
select
*,
last([Timestamp]) over (partition by deviceId limit duration(second,1) when [externaltemp] is not null) [LastTimeThereWasANonNullTemperature]
from
DeviceAggregates
)
select *
INTO
[powerBI]
from
DeviceAggregatesWithLast

Adding a date range to existing query

I have just started a new job and am working with existing queries. As I am no expert on SQL I'm wondering if a date range such as 2015-8-1 through 2015-8-31 can be inserted into the query below. Any help offered is greatly appreciated.
SELECT
RANK() OVER (PARTITION BY DoctorFacility.ListName ORDER BY ApptSlot.Start)
as SlotNumber
, DoctorFacility.ListName as ProviderName
, ApptSlot.Start as ApptStartTime
, AppointmentsAlloc.Type as ApptType
INTO #TEMP
FROM CentricityPS.dbo.ApptSlot ApptSlot
INNER JOIN CentricityPS.dbo.AppointmentsAlloc AppointmentsAlloc
ON ApptSlot.ApptSlotId=AppointmentsAlloc.ApptSlotId
INNER JOIN CentricityPS.dbo.Schedule Schedule
ON ApptSlot.ScheduleId=Schedule.ScheduleId
INNER JOIN CentricityPS.dbo.DoctorFacility DoctorFacility
ON Schedule.DoctorResourceId=DoctorFacility.DoctorFacilityId
WHERE AppointmentsAlloc.Type IN
(
'Behavioral Health - 30'
,'Behavioral Health 45'
,'Established Patient - 15'
,'Established Patient - 20'
,'Fin Counsel - 30'
,'Gyn Visit - 15'
,'Pediatric Visit - 15'
)
AND ApptSlot.ListOrder=1
AND ApptSlot.Status IS NULL
AND ApptSlot.Start>= GETDATE()
ORDER BY DoctorFacility.ListName
SELECT
ProviderName
, ApptStartTime
, ApptType
FROM #TEMP
WHERE SlotNumber = 3
ORDER BY ProviderName
DROP TABLE #TEMP
There are two separate select queries here. It would be done the same way for either part of the query regardless. Just add it to your WHERE statement
AND ApptSlot.Start>= 2015-8-1
AND ApptSlot.Start<= 2015-8-31
Another syntax would be
AND ApptSlot.Start between 2015-8-1 and 2015-8-31
Ideally you want to pass those begin and start dates as variables so values would not be hard coded. It might look more like this.
AND ApptSlot.Start between #StartDate and #EndDate

Count null columns as zeros with oracle

I am running a query with Oracle:
SELECT
c.customer_number,
COUNT(DISTINCT o.ORDER_NUMBER),
COUNT(DISTINCT q.QUOTE_NUMBER)
FROM
Customer c
JOIN Orders o on c.customer_number = o.party_number
JOIN Quote q on c.customer_number = q.account_number
GROUP BY
c.customer_number
This works beautifully and I can get the customer and their order and quote counts.
However, not all customers have orders or quotes but I still want their data. When I use LEFT JOIN I get this error from Oracle:
ORA-24347: Warning of a NULL column in an aggregate function
Seemingly this error is caused by the eventual COUNT(NULL) for customers that are missing orders and/or quotes.
How can I get a COUNT of null values to come out to 0 in this query?
I can do COUNT(DISTINCT NVL(o.ORDER_NUMBER, 0)) but then the counts will come out to 1 if orders/quotes are missing which is no good. Using NVL(o.ORDER_NUMBER, NULL) has the same problem.
Try using inline views:
SELECT
c.customer_number,
o.order_count,
q.quote_count
FROM
customer c,
( SELECT
party_number,
COUNT(DISTINCT order_number) AS order_count
FROM
orders
GROUP BY
party_number
) o,
( SELECT
account_number,
COUNT(DISTINCT quote_number) AS quote_count
FROM
quote
GROUP BY
account_number
) q
WHERE 1=1
AND c.customer_number = o.party_number (+)
AND c.customer_number = q.account_number (+)
;
Sorry, but I'm not working with any databases right now to test this, or to test whatever the ANSI SQL version might be. Just going on memory.

Resources