when concatenating more than one field in iif clause condition showing error in SSRS report builder 2016 - ssrs-2016

When i do concatenation of more than one field in if true clues it showing error. my expression code as as below:
=IIf(Fields!testnm.Value="High Temperature Test","Temperature Tolerance :"&Fields!val1.Value&" "&"Duration :"&Fields!val2.Value&" "&"Remarks :"&Fields!val3.Value,
IIf(Fields!testnm.Value="Burn In Test","Temperature Tolerance :"&Fields!val1.Value&" "&"Duration :"&Fields!val2.Value&" "&"Remarks :"&Fields!val3.Value,
IIf(Fields!testnm.Value="Low Temperature Test","Temperature Tolerance :"&Fields!val1.Value&" "&"Duration :"&Fields!val2.Value&" "&"Remarks :"&Fields!val3.Value,
IIf(Fields!testnm.Value="Dump Heat Test","Temperature Tolerance :"&Fields!val1.Value&" "&"RH Tolerance ""&Fields!val2.Value&" "&"Duration :"&Fields!val3.Value&" "&"Remarks :"&Fields!val4.Value,
IIf(Fields!testnm.Value="Rendom Vibration Test","Frequency Range :"&Fields!val1.Value&" "&"BreakUp Details :"&Fields!val2.Value&" "&"duration :"&Fields!val3.Value&" "&"Vibration Axis :"&Fields!val4.Value&" "&"Remarks :"&Fields!val5.Value&,
IIf(Fields!testnm.Value="Shock Test","Amplitude :"&Fields!val1.Value&" "&"Pulse Duration :"&Fields!val2.Value&" "&"Shocks :"&Fields!val3.Value&" "&"Wave :"&Fields!val4.Value&" "&"Shock Axis :"&Fields!val5.Value&" "&"Remarks :"&Fields!val6.Value,
IIf(Fields!testnm.Value="Sine Vibration Test","Frequency Range :"&Fields!val1.Value&" "&"BreakUp Details :"&Fields!val2.Value&" "&"Duration :"&Fields!val3.Value&" "&"Endurance :"&Fields!val4.Value&" "&"Sweeps :"&Fields!val5.Value&" "&"Endurance Duration :"&Fields!val6.Value&" "&"Vibration Axis :"&Fields!val7.Value&" "&"Remarks :"&Fields!val8.Value,
IIf(Fields!testnm.Value="Thermal Cycling Test","Temperature Tolerance To :"&Fields!val1.Value&" "&"From :"&Fields!val2.Value&" "&"Soak for PTemp :"&Fields!val3.Value&" "&"Soak for NTemp :"&Fields!val4.Value&" "&"Cooling Rate :"&Fields!val5.Value&" "&"Cycles :"&Fields!val6.Value&" "&"Remarks :"&Fields!val7.Value,
IIf(Fields!testnm.Value="Thermal Shock Test","Temperature Tolerance To :"&Fields!val1.Value&" "&"From :"&Fields!val2.Value&" "&"Changeover Time :"&Fields!val3.Value&" "&"Soak Duration :"&Fields!val4.Value&" "&"Cycles :"&Fields!val5.Value&" "&"Remarks :"&Fields!val6.Value,"BreakUp Details :"&Fields!val1.Value&" "&"RH Tolerance :"&Fields!val2.Value&" "&"Cycles :"&Fields!val3.Value&" "&"Remarks :"&Fields!val4.Value)))))))))
can anyone guide .. appreciate your guidance

At the end of line 5 you have an extra & Or am I missing the why of it being there?

Related

Influxdb time range query

I am trying to form a sql query for the below condition in where clause but it is giving an error.
date(\"time\") BETWEEN date_trunc('month', current_date - INTERVAL '1 months') \
AND (date_trunc('MONTH', current_date - INTERVAL '1 months' + INTERVAL '1 MONTH - 1 day'))
Influxdb query which I am trying and it is giving an error is as below.
SELECT * FROM "series" WHERE time >= (now() - 30d) AND time < (now() - (30d + 30d - 1d))
But above query is giving badrequest error. Could you please help me to correct this query.
The time clause in the InfluxQL query is unfulfillable, time lower and upper bounds seems reversed. Or the comparison operators are, depending on your coding preferences.

How to create a Series of datetime values for the nth calendar of the month for this year?

I would like to create a [function that returns] a pandas series of datetime values for the Nth calendar day of each month for the current year. An added wrinkle is I would also need it to be the previous business day if it happens to fall on the weekend. Bonus would be to check against known holidays as well.
For example, I'd like the output to look like this for the [business day prior to or equal to the] 14th day of the month
0 2021-01-14
1 2021-02-12
2 2021-03-12
3 2021-04-14
4 2021-05-14
5 2021-06-14
6 2021-07-14
7 2021-08-13
8 2021-09-14
9 2021-10-14
10 2021-11-12
11 2021-12-14
I've tried using pd.date_range() and pd.bdate_range() and did not get the desired results. Example:
pd.date_range("2021-01-14","2021-12-14", periods=12)
>> DatetimeIndex(['2021-01-14 00:00:00',
'2021-02-13 08:43:38.181818182',
'2021-03-15 17:27:16.363636364',
'2021-04-15 02:10:54.545454546',
'2021-05-15 10:54:32.727272728',
'2021-06-14 19:38:10.909090910',
'2021-07-15 04:21:49.090909092',
'2021-08-14 13:05:27.272727272',
'2021-09-13 21:49:05.454545456',
'2021-10-14 06:32:43.636363640',
'2021-11-13 15:16:21.818181820',
'2021-12-14 00:00:00'],
dtype='datetime64[ns]', freq=None)>>
Additionally this requires knowing the first and last month days that would be the start and end. Analogous tests with pd.bdate_range() resulted mostly in errors.
Similar approach to Pandas Date Range Monthly on Specific Day of Month but subtract a Bday to get the previous buisness day. Also start at 12/31 of the previous year to get all values for the current year:
def get_date_range(day_of_month, year=pd.Timestamp.now().year):
return (
pd.date_range(start=pd.Timestamp(year=year - 1, month=12, day=31),
periods=12, freq='MS') +
pd.Timedelta(days=day_of_month) -
pd.tseries.offsets.BDay()
)
Usage for year:
get_date_range(14)
DatetimeIndex(['2021-01-14', '2021-02-12', '2021-03-12', '2021-04-14',
'2021-05-14', '2021-06-14', '2021-07-14', '2021-08-13',
'2021-09-14', '2021-10-14', '2021-11-12', '2021-12-14'],
dtype='datetime64[ns]', freq=None)
Or for another year:
get_date_range(14, 2020)
DatetimeIndex(['2020-01-14', '2020-02-14', '2020-03-13', '2020-04-14',
'2020-05-14', '2020-06-12', '2020-07-14', '2020-08-14',
'2020-09-14', '2020-10-14', '2020-11-13', '2020-12-14'],
dtype='datetime64[ns]', freq=None)
With Holidays (this is non-vectorized so it will raise a PerformanceWarning):
import pandas as pd
from pandas.tseries.holiday import USFederalHolidayCalendar
from pandas.tseries.offsets import CustomBusinessDay
bday_us = CustomBusinessDay(calendar=USFederalHolidayCalendar())
def get_date_range(day_of_month, year=pd.Timestamp.now().year):
return (
pd.date_range(start=pd.Timestamp(year=year - 1, month=12, day=31),
periods=12, freq='MS') +
pd.Timedelta(days=day_of_month) -
bday_us
)
get_date_range(25)
DatetimeIndex(['2021-01-25', '2021-02-25', '2021-03-25', '2021-04-23',
'2021-05-25', '2021-06-25', '2021-07-23', '2021-08-25',
'2021-09-24', '2021-10-25', '2021-11-24', '2021-12-23'],
dtype='datetime64[ns]', freq=None)
You can use the months start and then add a timedelta to get it to the day you want. So for your example it would be:
pd.date_range(start=pd.Timestamp("2020-12-14"), periods=12, freq='MS') + pd.Timedelta(days=13)
Output:
DatetimeIndex(['2021-01-14', '2021-02-14', '2021-03-14', '2021-04-14',
'2021-05-14', '2021-06-14', '2021-07-14', '2021-08-14',
'2021-09-14', '2021-10-14', '2021-11-14', '2021-12-14'],
dtype='datetime64[ns]', freq=None)
to move to the previous business day use (see: Pandas offset DatetimeIndex to next business if date is not a business day and Most recent previous business day in Python) :
(pd.date_range(start=pd.Timestamp("2021-06-04"), periods=12, freq='MS') + pd.Timedelta(days=4)).map(lambda x: x - pd.tseries.offsets.BDay())
output:
DatetimeIndex(['2021-07-02', '2021-08-05', '2021-09-03', '2021-10-04',
'2021-11-04', '2021-12-03', '2022-01-06', '2022-02-04',
'2022-03-04', '2022-04-04', '2022-05-05', '2022-06-03'],
dtype='datetime64[ns]', freq=None)

Calculation returning wrong resutls

I am created a calculation using two calculationsi n Tableau The results returned are incorrect. But each piece of the calculation returns the correct number.
Cal 1:
If YEAR([TRANSACTION_DATE ) = YEAR(TODAY()) AND [TRANSACTION_DATE (Custom SQL Query)]< TODAY() THEN [Handled] END
Results: 57,619
Cal 2:
`If YEAR([TRANSACTION_DATE (Custom SQL Query)]) = YEAR(TODAY())
AND [TRANSACTION_DATE (Custom SQL Query)]< TODAY()
and [Contact Type His]= 'CHAT Agt' THEN [AS ACTIVE TIME (S)]/3600
ELSEIF YEAR([TRANSACTION_DATE (Custom SQL Query)]) = YEAR(TODAY())
AND [TRANSACTION_DATE (Custom SQL Query)]< TODAY()
and[Contact Type His] = 'BLENDED Agt' Then [Busy Time (S)]/3600 end`
Results: 6,012
Final Calculation: CAL 1 /CAL 2
Results 20997.79 expecting 9.58
Can someone tell me what I am doing wrong.
If you want to calculate these aggregations without any further level of detail, use this calculation
COL1
{SUM(
If YEAR([TRANSACTION_DATE ) = YEAR(TODAY()) AND [TRANSACTION_DATE (Custom SQL Query)]< TODAY() THEN [Handled] END
)}
COL2
{SUM(
If YEAR([TRANSACTION_DATE (Custom SQL Query)]) = YEAR(TODAY())
AND [TRANSACTION_DATE (Custom SQL Query)]< TODAY()
and [Contact Type His]= 'CHAT Agt' THEN [AS ACTIVE TIME (S)]/3600
ELSEIF YEAR([TRANSACTION_DATE (Custom SQL Query)]) = YEAR(TODAY())
AND [TRANSACTION_DATE (Custom SQL Query)]< TODAY()
and[Contact Type His] = 'BLENDED Agt' Then [Busy Time (S)]/3600 end
)}
and then Col1/Col2

Spark 3.0 UTC to AKST conversion fails with ZoneRulesException: Unknown time-zone ID

I am not able to convert timestamps in UTC to AKST timezone in Spark 3.0. The same works in Spark 2.4. All other conversions work (to EST, PST, MST etc).
Appreciate any inputs on how to fix this error.
Below command:
spark.sql("select from_utc_timestamp('2020-10-01 11:12:30', 'AKST')").show
returns the error:
java.time.zone.ZoneRulesException: Unknown time-zone ID: AKST
Detailed log:
java.time.zone.ZoneRulesException: Unknown time-zone ID: AKST
at java.time.zone.ZoneRulesProvider.getProvider(ZoneRulesProvider.java:272)
at java.time.zone.ZoneRulesProvider.getRules(ZoneRulesProvider.java:227)
at java.time.ZoneRegion.ofId(ZoneRegion.java:120)
at java.time.ZoneId.of(ZoneId.java:411)
at java.time.ZoneId.of(ZoneId.java:359)
at java.time.ZoneId.of(ZoneId.java:315)
at org.apache.spark.sql.catalyst.util.DateTimeUtils$.getZoneId(DateTimeUtils.scala:62)
at org.apache.spark.sql.catalyst.util.DateTimeUtils$.fromUTCTime(DateTimeUtils.scala:833)
at org.apache.spark.sql.catalyst.expressions.FromUTCTimestamp.nullSafeEval(datetimeExpressions.scala:1299)
at org.apache.spark.sql.catalyst.expressions.BinaryExpression.eval(Expression.scala:552)
at org.apache.spark.sql.catalyst.expressions.UnaryExpression.eval(Expression.scala:457)
at org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$1$$anonfun$applyOrElse$1.applyOrElse(expressions.scala:52)
at org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$1$$anonfun$applyOrElse$1.applyOrElse(expressions.scala:45)
at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$1(TreeNode.scala:321)
at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:72)
at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:321)
at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$3(TreeNode.scala:326)
at org.apache.spark.sql.catalyst.trees.TreeNode.applyFunctionIfChanged$1(TreeNode.scala:380)
at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$mapChildren$1(TreeNode.scala:416)
at org.apache.spark.sql.catalyst.trees.TreeNode.mapProductIterator(TreeNode.scala:248)
at org.apache.spark.sql.catalyst.trees.TreeNode.mapChildren(TreeNode.scala:414)
at org.apache.spark.sql.catalyst.trees.TreeNode.mapChildren(TreeNode.scala:362)
at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:326)
at org.apache.spark.sql.catalyst.plans.QueryPlan.$anonfun$transformExpressionsDown$1(QueryPlan.scala:96)
at org.apache.spark.sql.catalyst.plans.QueryPlan.$anonfun$mapExpressions$1(QueryPlan.scala:118)
at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:72)
at org.apache.spark.sql.catalyst.plans.QueryPlan.transformExpression$1(QueryPlan.scala:118)
at org.apache.spark.sql.catalyst.plans.QueryPlan.recursiveTransform$1(QueryPlan.scala:129)
at org.apache.spark.sql.catalyst.plans.QueryPlan.$anonfun$mapExpressions$3(QueryPlan.scala:134)
at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
at scala.collection.immutable.List.foreach(List.scala:392)
at scala.collection.TraversableLike.map(TraversableLike.scala:238)
at scala.collection.TraversableLike.map$(TraversableLike.scala:231)
at scala.collection.immutable.List.map(List.scala:298)
at org.apache.spark.sql.catalyst.plans.QueryPlan.recursiveTransform$1(QueryPlan.scala:134)
at org.apache.spark.sql.catalyst.plans.QueryPlan.$anonfun$mapExpressions$4(QueryPlan.scala:139)
at org.apache.spark.sql.catalyst.trees.TreeNode.mapProductIterator(TreeNode.scala:248)
at org.apache.spark.sql.catalyst.plans.QueryPlan.mapExpressions(QueryPlan.scala:139)
at org.apache.spark.sql.catalyst.plans.QueryPlan.transformExpressionsDown(QueryPlan.scala:96)
at org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$1.applyOrElse(expressions.scala:45)
at org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$1.applyOrElse(expressions.scala:44)
at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$1(TreeNode.scala:321)
at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:72)
at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:321)
at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDown(LogicalPlan.scala:29)
at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDown(AnalysisHelper.scala:149)
at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDown$(AnalysisHelper.scala:147)
at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDown(LogicalPlan.scala:29)
at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDown(LogicalPlan.scala:29)
at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$3(TreeNode.scala:326)
at org.apache.spark.sql.catalyst.trees.TreeNode.applyFunctionIfChanged$1(TreeNode.scala:380)
at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$mapChildren$1(TreeNode.scala:416)
at org.apache.spark.sql.catalyst.trees.TreeNode.mapProductIterator(TreeNode.scala:248)
at org.apache.spark.sql.catalyst.trees.TreeNode.mapChildren(TreeNode.scala:414)
at org.apache.spark.sql.catalyst.trees.TreeNode.mapChildren(TreeNode.scala:362)
at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:326)
at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDown(LogicalPlan.scala:29)
at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDown(AnalysisHelper.scala:149)
at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDown$(AnalysisHelper.scala:147)
at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDown(LogicalPlan.scala:29)
at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDown(LogicalPlan.scala:29)
at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$3(TreeNode.scala:326)
at org.apache.spark.sql.catalyst.trees.TreeNode.applyFunctionIfChanged$1(TreeNode.scala:380)
at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$mapChildren$1(TreeNode.scala:416)
at org.apache.spark.sql.catalyst.trees.TreeNode.mapProductIterator(TreeNode.scala:248)
at org.apache.spark.sql.catalyst.trees.TreeNode.mapChildren(TreeNode.scala:414)
at org.apache.spark.sql.catalyst.trees.TreeNode.mapChildren(TreeNode.scala:362)
at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:326)
at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDown(LogicalPlan.scala:29)
at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDown(AnalysisHelper.scala:149)
at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDown$(AnalysisHelper.scala:147)
at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDown(LogicalPlan.scala:29)
at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDown(LogicalPlan.scala:29)
at org.apache.spark.sql.catalyst.trees.TreeNode.transform(TreeNode.scala:310)
at org.apache.spark.sql.catalyst.optimizer.ConstantFolding$.apply(expressions.scala:44)
at org.apache.spark.sql.catalyst.optimizer.ConstantFolding$.apply(expressions.scala:43)
at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$2(RuleExecutor.scala:149)
at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
at scala.collection.immutable.List.foldLeft(List.scala:89)
at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$1(RuleExecutor.scala:146)
at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$1$adapted(RuleExecutor.scala:138)
at scala.collection.immutable.List.foreach(List.scala:392)
at org.apache.spark.sql.catalyst.rules.RuleExecutor.execute(RuleExecutor.scala:138)
at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$executeAndTrack$1(RuleExecutor.scala:116)
at org.apache.spark.sql.catalyst.QueryPlanningTracker$.withTracker(QueryPlanningTracker.scala:98)
at org.apache.spark.sql.catalyst.rules.RuleExecutor.executeAndTrack(RuleExecutor.scala:116)
at org.apache.spark.sql.execution.QueryExecution.$anonfun$optimizedPlan$1(QueryExecution.scala:82)
at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:121)
at org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$1(QueryExecution.scala:153)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:763)
at org.apache.spark.sql.execution.QueryExecution.executePhase(QueryExecution.scala:153)
at org.apache.spark.sql.execution.QueryExecution.optimizedPlan$lzycompute(QueryExecution.scala:82)
at org.apache.spark.sql.execution.QueryExecution.optimizedPlan(QueryExecution.scala:79)
at org.apache.spark.sql.execution.QueryExecution.$anonfun$writePlans$4(QueryExecution.scala:217)
at org.apache.spark.sql.catalyst.plans.QueryPlan$.append(QueryPlan.scala:381)
at org.apache.spark.sql.execution.QueryExecution.org$apache$spark$sql$execution$QueryExecution$$writePlans(QueryExecution.scala:217)
at org.apache.spark.sql.execution.QueryExecution.toString(QueryExecution.scala:227)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:96)
at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:207)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:88)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:763)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:65)
at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3653)
at org.apache.spark.sql.Dataset.head(Dataset.scala:2737)
at org.apache.spark.sql.Dataset.take(Dataset.scala:2944)
at org.apache.spark.sql.Dataset.getRows(Dataset.scala:301)
at org.apache.spark.sql.Dataset.showString(Dataset.scala:338)
at org.apache.spark.sql.Dataset.show(Dataset.scala:864)
at org.apache.spark.sql.Dataset.show(Dataset.scala:823)
at org.apache.spark.sql.Dataset.show(Dataset.scala:832)
... 47 elided
Adding further to mck's answer. You are using the old Java data-time API zone's short IDs. According to this Databricks blog post A Comprehensive Look at Dates and Timestamps in Apache Spark™ 3.0, Spark migrated to the new API since version 3.0:
Since Java 8, the JDK has exposed a new API for date-time manipulation
and time zone offset resolution, and Spark migrated to this new API in
version 3.0. Although the mapping of time zone names to offsets has
the same source, IANA TZDB, it is implemented differently in Java 8
and higher versus Java 7.
You can verify it by opening spark-shell and list available zones like this:
import java.time.ZoneId
import scala.collection.JavaConverters._
ZoneId.SHORT_IDS.asScala.keys
//res0: Iterable[String] = Set(CTT, ART, CNT, PRT, PNT, PLT, AST, BST, CST, EST, HST, JST, IST, AGT, NST, MST, AET, BET, PST, ACT, SST, VST, CAT, ECT, EAT, IET, MIT, NET)
That said, you should not use abbreviations when you specify timezones, instead use area/city format. See Which three-letter time zone IDs are not deprecated?
Seems it can't understand AKST, but Spark 3 seems to understand America/Anchorage, which I suppose to have the timezone AKST:
spark.sql("select from_utc_timestamp('2020-10-01 11:12:30', 'America/Anchorage')").show

Max and min values in pandas

I have the following data:
High Low Open Close Volume Adj Close
Date
1999-12-31 1472.420044 1458.189941 1464.469971 1469.250000 374050000 1469.250000
2000-01-03 1478.000000 1438.359985 1469.250000 1455.219971 931800000 1455.219971
2000-01-04 1455.219971 1397.430054 1455.219971 1399.420044 1009000000 1399.420044
2000-01-05 1413.270020 1377.680054 1399.420044 1402.109985 1085500000 1402.109985
2000-01-06 1411.900024 1392.099976 1402.109985 1403.449951 1092300000 1403.449951
... ... ... ... ... ... ...
2020-01-06 3246.840088 3214.639893 3217.550049 3246.280029 3674070000 3246.280029
2020-01-07 3244.909912 3232.429932 3241.860107 3237.179932 3420380000 3237.179932
2020-01-08 3267.070068 3236.669922 3238.590088 3253.050049 3720890000 3253.050049
2020-01-09 3275.580078 3263.669922 3266.030029 3274.699951 3638390000 3274.699951
2020-01-10 3282.989990 3268.010010 3281.810059 3273.739990 920449258 3273.739990
5039 rows × 6 columns
Since this is the daily data this was resampled to weekly to find the 52 week high and low.
weekly_high = data.High.groupby(pd.Grouper(freq='M')).tail(52)
weekly_low = data.Low.groupby(pd.Grouper(freq='M')).tail(52)
Here is the problem:
weekly_high.max()
yields: 3282.989990234375
weekly_low.min()
yeilds: 666.7899780273438
These value are are issue because 3283.0 is the high so why am i getting in deimals? Secondly weekly low is is 666 which i know for a fact is incorrect. How can i fix this?
hi you can try the following code:
data['52weekhigh'] = data.High.rolling(252).max()
data['52weeklow'] = data.Low.rolling(252).min()
This allows you to prevent having to resample on a monthly basis and gives you the rolling 52 week high (52 weeks == 252 trading days) Let me know if you need any further clarification.

Resources