I am currently working with Presto 0.80. I have to write a user defined function to convert degree celsius to degree fahrenheit during select query. I did the same using Hive QL but was wondering if we can replicate the same in Facebook Presto.
Any help would be highly appreciated.
Thanks!!
Here is a guide for writing a new function in presto.
https://trino.io/docs/current/develop/functions.html
After writing your function, add the plugin to the plugin directory as explained in SPI Overview.
There is another example for writing presto UDF from Qubole blog.
http://www.qubole.com/blog/product/plugging-in-presto-udfs/
You can try it if you can make it work.
Related
We store very complex json in one of our table columns. I would like to write a parser for this. I was reading thru table functions and functions but I never saw a great guide that would tell me how to create a function / deploy it to our cluster. Does anyone have any good pointers.
I was wondering if the LOESS (locally estimated scatterplot smoothing) regression was a function built-in Spark/PySpark (I'm more interested in the PySpark answer but both would be interesting).
I did some research and couldn't find one so decided to try and code it myself using pandas-udf functions but while doing it, when I displayed the scatter_plot of the manufactured data I created to begin testing my algo, Azure Databricks (on which I'm coding) proposed to me to automatically compute/display the LOESS of my dataset :
So maybe there is indeed a built-in LOESS that I just couldn't find ? If not (and Databricks is the only one responsible for this), is there any way to access the result of databricks's LOESS computation/access the function Databricks is using to do that ?
Thank you in advance :)
I just got introduced to Spark SQL higher order functions transform(), filter() etc. I searched online, but couldn't find much advanced use-cases leveraging these functions.
Can anyone please explain transform() with a couple of advanced real-life use-case using sql query. Does it always need to work on nested complex types (arrays, struct etc) ? Or can it be used to process simple data-type records as well ?
Any help is appreciated.
Thanks
The following online resource vividly demonstrates in %sql mode :
https://docs.databricks.com/delta/data-transformation/higher-order-lambda-functions.html
I came across this interesting date conversion table for postgresql and am wondering if this syntax for date patterns can also be used in presto. It might or might not work, anyone familiar with this?
Equivalent functionality is available through the date_format() Trino (formerly Presto) function
The to_char function does not exist in Presto.
Documentation contains the list of builtin Presto functions. When in doubt, you can get a list of all the functions available using the SHOW FUNCTIONS statement.
I'm working with cassandra + spark + spark-sql. I'n not using Hive.
I'd like to create my custom aggregation function, like:
select percentile(column, 0.95) from cassandra_table
spark-sql support avg(), min(), etc.: I want to implement others like percentile, but I cannot find documentation on this.
Can someone point me to to any doc or class to start with?
Thanks!