U-SQL Stored Procedure Datetime.Tryparse in where - azure

Question

Is there any way we can use DateTime.TryParse in U-SQL WHERE condition?
I am creating a stored procedure in Azure U-SQL catalog.
Simple U-SQL script supports C# functions.
For stored procedure, the script is not adding any assembly reference to my custom C# code OR if adding it's not being used while calling the stored procedure.

Answer 1

Yes this is possible using in-line functions. This article shows how to do that. Here is a simple example:
#input = SELECT *
FROM (
VALUES
( (int)1, (string)"1/1/2017" ),
( (int)2, (string)"1/2/2017" ),
( (int)3, (string)"bad date" )
) AS x ( rn, someString );
// inline function example
#output =
SELECT *
FROM
(
SELECT *,
(
(Func<string, DateTime?>)
(dateString => // input_paramater
{
DateTime dateValue;
return DateTime.TryParse(dateString, out dateValue) ? (DateTime?)dateValue : (DateTime?)null;
}
)
) (someString) AS someDate
FROM #input
) AS x
WHERE someDate IS NOT NULL;
OUTPUT #output TO "/output/output.csv"
USING Outputters.Csv(quoting:false);
If you want to use the assembly you have created as a U-SQL Class project and registered then you can just reference it in the stored procedure using REFERENCE ASSEMBLY:
DROP PROCEDURE IF EXISTS dbo.usp_testCleanDate;
CREATE PROCEDURE dbo.usp_testCleanDate()
AS
BEGIN
REFERENCE ASSEMBLY USQLCSharpProject1;
#input = SELECT *
FROM (
VALUES
( (int)1, (string)"1/1/2017" ),
( (int)2, (string)"1/2/2017" ),
( (int)3, (string)"bad date" )
) AS x ( rn, someString );
#output =
SELECT *,
USQLCSharpProject1.Class1.tryParseDate(someString) AS x
FROM #input;
OUTPUT #output
TO "/output/output.csv"
USING Outputters.Csv(quoting : false);
END;

Answer 2

Using Function dt_TryParse_USQL
Using above code-behind and calling function. Function consumes a string and attempts to convert the string to a DateTime value using DateTime.TryParse. Using the Code-Behind above.
#employees =
SELECT * FROM
( VALUES
(1, "Noah", "2/16/2008"),
(2, "Sophia", "2/16/2008 12:15:12 PM"),
(3, "Liam", "16/02/2008 12:15:12"),
(4, "Amy", "2017-01-11T16:52:07"),
(5, "Justin", "")
) AS T(EmpID, EmpName, StartDate);
#result =
SELECT
EmpID,
EmpName,
ReferenceGuide_Examples.MyClass.dt_TryParse_USQL(StartDate) AS validated_StartDate
FROM #employees;
OUTPUT #result
TO "/Output/ReferenceGuide/DDL/Functions/dt_TryParse_USQL.csv"
USING Outputters.Csv(outputHeader: true);
https://msdn.microsoft.com/en-us/azure/data-lake-analytics/u-sql/u-sql-functions
Additional information: Click here

Answer 3

To answer this part of the question:
For stored procedure, the script is not adding any assembly reference to my custom C# code OR if adding it's not being used while calling the stored procedure.
U-SQL Functions and Procedures carry their own static context and do not inherit the static name context from the invocation context. That allows to understand the semantics of the function/procedure independently from where it is being called.
Thus in order to refer to assembly code, you will need to register the assembly in the catalog and explicitly reference it inside the procedure with REFERENCE ASSEMBLY. Code-behind is not meant to be used within a procedure (since the procedure is stored for future use).