.net core 2.0 memory leak - memory-leaks
I am running .net core 2.0 app on Ubuntu 16.04.
I have a small memory leak. The leak growing ~1 percent every one hour.
I was created a dump but I cant find the source of the leak. I see huge amount of Byte[],SQLClient and threads but still i can't find the source even I run the GCRoot on byte[] objects (after DumpHeap on byte[]).
The mostly used components are: EF Core,Quartz,AWS..
When I run the app on windows machine the memory looks good.
Please find below the main part on the dump (three hours up time):
Any help would be greatly appreciated.
0007f3f77d0f730 303 24240 System.Threading.Thread
00007f3f79b86d60 448 25088 Microsoft.Extensions.DependencyInjection.ServiceDescriptor
00007f3f78c2c1e0 33 25200 System.Collections.Generic.Dictionary`2+Entry[[System.String, System.Private.CoreLib],[Newtonsoft.Json.Linq.JToken, Newtonsoft.Json]][]
00007f3f77d13700 450 25200 System.RuntimeType+RuntimeTypeCache+MemberInfoCache`1[[System.Reflection.RuntimePropertyInfo, System.Private.CoreLib]]
00007f3f7d550d10 32 25568 System.Data.ProviderBase.DbConnectionInternal[]
00007f3f7b96bfb0 272 26400 System.Collections.Generic.Dictionary`2+Entry[[System.String, System.Private.CoreLib],[StructureMap.Pipeline.Instance, StructureMap]][]
00007f3f7bd2a780 413 26432 System.Action`1[[StructureMap.Graph.PluginFamily, StructureMap]]
00007f3f77cc2948 26 26808 System.UInt32[]
00007f3f77d24500 849 27168 System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunnerWithContext
00007f3f7d7d6760 173 27680 System.Security.Cryptography.X509Certificates.X509Certificate2
00007f3f77cc69e8 522 28416 System.Reflection.Emit.__ExceptionInfo[]
00007f3f77d138b0 525 29400 System.RuntimeType+RuntimeTypeCache+MemberInfoCache`1[[System.Reflection.RuntimeMethodInfo, System.Private.CoreLib]]
00007f3f7d20d0c0 764 30560 Microsoft.EntityFrameworkCore.Metadata.Internal.PropertyIndexes
00007f3f78da4b88 81 30912 System.Collections.Hashtable+bucket[]
00007f3f77d13940 570 31920 System.RuntimeType+RuntimeTypeCache+MemberInfoCache`1[[System.Reflection.RuntimeConstructorInfo, System.Private.CoreLib]]
00007f3f7d3fc4d8 836 33440 Microsoft.EntityFrameworkCore.Query.Expressions.ColumnExpression
00007f3f7d19ac20 158 35392 Microsoft.EntityFrameworkCore.Metadata.Internal.EntityType
00007f3f7dccbb90 249 35424 System.Collections.Generic.Dictionary`2+Entry[[System.String, System.Private.CoreLib],[System.Net.Http.Headers.HttpHeaders+HeaderStoreItemInfo, System.Net.Http]][]
00007f3f7dccb1a0 768 36864 System.Net.Http.Headers.HttpHeaders+HeaderStoreItemInfo
00007f3f77cc2a88 10 37306 System.UInt16[]
00007f3f77d44360 1594 38256 System.Reflection.Emit.DynamicScope
00007f3f7d1a7040 1598 38352 System.Collections.Generic.SortedDictionary`2+KeyValuePairComparer[[System.String, System.Private.CoreLib],[Microsoft.EntityFrameworkCore.Infrastructure.Annotation, Microsoft.EntityFrameworkCore]]
00007f3f7b96c618 616 39424 System.Action`1[[StructureMap.Graph.PluginGraph, StructureMap]]
00007f3f77d1e0a8 518 41440 System.Threading.Tasks.Task+ContingentProperties
00007f3f77d0b3b8 867 41616 System.Text.StringBuilder
00007f3f77cc30f8 710 42192 System.Type[]
00007f3f77358f30 229 43712 System.Reflection.RuntimePropertyInfo[][]
00007f3f7e126f08 616 44352 System.Threading.Tasks.Task`1[[Amazon.SQS.Model.ReceiveMessageResponse, AWSSDK.SQS]]
00007f3f77cc6a48 238 45168 System.Reflection.Emit.__FixupData[]
00007f3f7d2065d0 1438 46016 Microsoft.EntityFrameworkCore.Metadata.Internal.InternalPropertyBuilder
00007f3f78c29dd8 1998 47952 Newtonsoft.Json.Linq.JToken+LineInfoAnnotation
00007f3f7bec1040 1508 48256 System.Runtime.CompilerServices.Closure
00007f3f77d43ec0 1594 51008 System.Reflection.Emit.LineNumberInfo
00007f3f7db731e0 1345 53800 System.Collections.Generic.List`1[[System.Collections.Generic.KeyValuePair`2[[System.String, System.Private.CoreLib],[ThirdParty.Json.LitJson.JsonData, AWSSDK.Core]], System.Private.CoreLib]]
00007f3f7db73610 461 55416 System.Collections.Generic.KeyValuePair`2[[System.String, System.Private.CoreLib],[ThirdParty.Json.LitJson.JsonData, AWSSDK.Core]][]
00007f3f77d43fd0 630 60480 System.Reflection.Emit.__ExceptionInfo
00007f3f77c9f510 956 61184 System.Action
00007f3f78c2ad00 978 62592 Newtonsoft.Json.Linq.JValue
00007f3f7d1a1710 1598 63920 System.Collections.Generic.SortedDictionary`2[[System.String, System.Private.CoreLib],[Microsoft.EntityFrameworkCore.Infrastructure.Annotation, Microsoft.EntityFrameworkCore]]
00007f3f7d494e60 2 65584 Microsoft.AspNetCore.Server.Kestrel.Transport.Libuv.Internal.LibuvThread+Work[]
00007f3f7e7f7de0 2743 65832 System.Collections.Generic.Dictionary`2+ValueCollection[[System.Int32, System.Private.CoreLib],[System.Data.SqlClient.SNI.SNIMarsHandle, System.Data.SqlClient]]
00007f3f7d557368 2763 66312 System.Data.SqlClient.LastIOTimer
00007f3f7d550ff8 2763 66312 System.Data.SqlClient.SqlInternalConnectionTds+SyncAsyncLock
00007f3f77d0d850 1698 67920 System.Collections.Generic.List`1[[System.Object, System.Private.CoreLib]]
00007f3f7d1a24d0 2136 68352 Microsoft.EntityFrameworkCore.Internal.LazyRef`1[[System.Collections.Generic.SortedDictionary`2[[System.String, System.Private.CoreLib],[Microsoft.EntityFrameworkCore.Infrastructure.Annotation, Microsoft.EntityFrameworkCore]], System.Collections]]
00007f3f77d4ff58 2180 69760 System.Threading.AsyncLocalValueMap+OneElementAsyncLocalValueMap
00007f3f7d1a6e78 1782 71280 Microsoft.EntityFrameworkCore.Metadata.Internal.ConventionalAnnotation
00007f3f7d848968 1 71680 System.Collections.Concurrent.ConcurrentDictionary`2+Node[[System.Data.SqlClient.SNI.SNIHandle, System.Data.SqlClient],[System.Data.SqlClient.SNI.SNIMarsConnection, System.Data.SqlClient]][]
00007f3f7db73bd0 460 72960 System.Collections.Generic.Dictionary`2+Entry[[System.String, System.Private.CoreLib],[ThirdParty.Json.LitJson.JsonData, AWSSDK.Core]][]
00007f3f77d446e8 1594 76512 System.Reflection.Emit.DynamicMethod+RTDynamicMethod
00007f3f77d43e60 1594 76512 System.Reflection.Emit.ScopeTree
00007f3f77d43ab8 1594 76512 System.Reflection.Emit.SignatureHelper
00007f3f7dcf2d30 332 77024 Newtonsoft.Json.Serialization.JsonProperty
00007f3f7d534508 2473 79136 System.Data.SqlClient.SqlConnectionPoolKey
00007f3f7d2099e8 1438 80528 System.Collections.Generic.SortedSet`1+Node[[System.Collections.Generic.KeyValuePair`2[[System.String, System.Private.CoreLib],[Microsoft.EntityFrameworkCore.Metadata.Internal.Property, Microsoft.EntityFrameworkCore]], System.Private.CoreLib]]
00007f3f7b96b948 299 81328 System.Collections.Concurrent.ConcurrentDictionary`2+Node[[System.Type, System.Private.CoreLib],[System.Int32, System.Private.CoreLib]][]
00007f3f77cc46e8 2022 81704 System.Reflection.RuntimePropertyInfo[]
00007f3f77d1b720 832 86528 System.Reflection.RuntimeConstructorInfo
00007f3f78c2b168 984 86592 Newtonsoft.Json.Linq.JProperty
00007f3f77d15fa0 2181 87240 System.Threading.ExecutionContext
00007f3f7d678118 2763 88416 System.Net.CallbackClosure
00007f3f77d3eb08 2763 88416 System.Threading.Tasks.SingleProducerSingleConsumerQueue`1[[System.Threading.Tasks.Task, System.Private.CoreLib]]
00007f3f7d485468 2764 88448 System.Net.IPEndPoint
00007f3f7d1a7740 1598 89488 System.Collections.Generic.TreeSet`1[[System.Collections.Generic.KeyValuePair`2[[System.String, System.Private.CoreLib],[Microsoft.EntityFrameworkCore.Infrastructure.Annotation, Microsoft.EntityFrameworkCore]], System.Private.CoreLib]]
00007f3f7d847320 3815 91560 System.Data.SqlClient.SqlCollation
00007f3f77d29668 2983 95456 Microsoft.Win32.SafeHandles.SafeWaitHandle
00007f3f7d537ff8 2473 98920 System.Data.ProviderBase.DbConnectionPoolGroupOptions
00007f3f7d1a8070 1782 99792 System.Collections.Generic.SortedSet`1+Node[[System.Collections.Generic.KeyValuePair`2[[System.String, System.Private.CoreLib],[Microsoft.EntityFrameworkCore.Infrastructure.Annotation, Microsoft.EntityFrameworkCore]], System.Private.CoreLib]]
00007f3f7d538c50 2526 101040 System.Collections.Concurrent.ConcurrentDictionary`2+Tables[[System.Data.ProviderBase.DbConnectionPoolIdentity, System.Data.SqlClient],[System.Data.ProviderBase.DbConnectionPool, System.Data.SqlClient]]
00007f3f7d8af0d8 2805 103136 System.Data.SqlClient._SqlMetaData[]
00007f3f7db72af0 1345 107600 System.Collections.Generic.Dictionary`2[[System.String, System.Private.CoreLib],[ThirdParty.Json.LitJson.JsonData, AWSSDK.Core]]
00007f3f7d551380 2735 109400 System.Data.SqlClient.SessionStateRecord
00007f3f7d8499a8 2763 110520 System.Net.Sockets.Socket+CachedTaskEventArgs
00007f3f7d8402f0 2763 110520 System.Collections.Generic.List`1[[System.Data.SqlClient.TdsParserStateObject, System.Data.SqlClient]]
00007f3f77d3d7f8 2764 110560 System.Threading.AutoResetEvent
00007f3f77d1ea98 2764 110560 System.Threading.Tasks.TaskFactory
00007f3f798fbd48 2770 110800 System.Net.IPAddress
00007f3f77d443c0 1594 114768 System.Reflection.Emit.DynamicResolver
00007f3f7d539258 2473 118704 System.Data.SqlClient.SqlConnectionPoolGroupProviderInfo
00007f3f77d44950 3757 120224 System.Reflection.Emit.GenericFieldInfo
00007f3f77d13670 839 127528 System.RuntimeType+RuntimeTypeCache
00007f3f77d10968 5401 129624 System.RuntimeTypeHandle
00007f3f7dac9e38 1633 130640 ThirdParty.Json.LitJson.JsonData
00007f3f7d8ac368 2763 132624 System.Data.SqlClient.SqlReferenceCollection
00007f3f7d8488c0 2763 132624 System.Collections.Concurrent.ConcurrentDictionary`2+Node[[System.Data.SqlClient.SNI.SNIHandle, System.Data.SqlClient],[System.Data.SqlClient.SNI.SNIMarsConnection, System.Data.SqlClient]]
00007f3f7d840260 2763 132624 System.Data.SqlClient.TdsParserSessionPool
00007f3f7d678008 2763 132624 System.Net.Sockets.Socket+CacheSet
00007f3f7d5519e0 2763 132624 System.Data.SqlClient.SqlConnectionTimeoutErrorInternal
00007f3f77d67e10 2763 132624 System.Collections.Concurrent.ConcurrentDictionary`2+Tables[[System.Int32, System.Private.CoreLib],[System.Threading.Tasks.ConcurrentExclusiveSchedulerPair+ProcessingMode, System.Private.CoreLib]]
00007f3f77d3efb0 2763 132624 System.Threading.Tasks.MultiProducerMultiConsumerQueue`1[[System.Threading.Tasks.Task, System.Private.CoreLib]]
00007f3f77d3e840 2763 132624 System.Collections.Concurrent.ConcurrentDictionary`2[[System.Int32, System.Private.CoreLib],[System.Threading.Tasks.ConcurrentExclusiveSchedulerPair+ProcessingMode, System.Private.CoreLib]]
00007f3f77cc0168 1707 136640 System.String[]
00007f3f77d350e8 1594 140272 System.RuntimeMethodInfoStub
00007f3f7d5382a8 2526 141456 System.Collections.Concurrent.ConcurrentDictionary`2[[System.Data.ProviderBase.DbConnectionPoolIdentity, System.Data.SqlClient],[System.Data.ProviderBase.DbConnectionPool, System.Data.SqlClient]]
00007f3f77cc4b80 657 144808 System.Reflection.RuntimeMethodInfo[]
00007f3f77d1b910 2033 146376 System.Reflection.RtFieldInfo
00007f3f77d135e0 6204 148896 System.RuntimeMethodHandle
00007f3f77d444f0 1594 153024 System.Reflection.Emit.DynamicMethod
00007f3f7d553a78 2741 153496 System.Data.SqlClient.SNI.SNIError
00007f3f7d8aeac8 2805 157080 System.Data.SqlClient._SqlMetaDataSet
00007f3f7d840898 2795 157976 System.Data.SqlClient.TdsParserStateObject[]
00007f3f7d555f48 2929 164024 System.Data.SqlClient.SNI.SNIPacket
00007f3f7d84a148 2650 169600 System.Action`5[[System.Int32, System.Private.CoreLib],[System.Byte[], System.Private.CoreLib],[System.Int32, System.Private.CoreLib],[System.Net.Sockets.SocketFlags, System.Net.Sockets],[System.Net.Sockets.SocketError, System.Net.Primitives]]
00007f3f7d673a48 2763 176832 System.Net.Sockets.SafeCloseSocket
00007f3f7d534cb0 2473 178056 System.Data.ProviderBase.DbConnectionPoolGroup
00007f3f77d202a8 2787 178368 System.Threading.SemaphoreSlim
00007f3f77d3e7e0 2763 198936 System.Threading.Tasks.ConcurrentExclusiveSchedulerPair
00007f3f77d12000 2666 213280 System.Collections.Generic.Dictionary`2[[System.String, System.Private.CoreLib],[System.String, System.Private.CoreLib]]
00007f3f7d849190 2763 221040 System.Collections.Generic.Dictionary`2[[System.Int32, System.Private.CoreLib],[System.Data.SqlClient.SNI.SNIMarsHandle, System.Data.SqlClient]]
00007f3f7d307708 2763 221040 System.Collections.Generic.Dictionary`2[[System.String, System.Private.CoreLib],[System.Tuple`2[[System.String, System.Private.CoreLib],[System.String, System.Private.CoreLib]], System.Private.CoreLib]]
00007f3f77d44180 1594 242288 System.Reflection.Emit.DynamicILGenerator
00007f3f7d8480a8 2763 243144 System.Data.SqlClient.SNI.SNIMarsConnection
00007f3f77cc4d10 8408 252208 System.Reflection.ParameterInfo[]
00007f3f77d143c8 2457 255528 System.Reflection.RuntimePropertyInfo
00007f3f7d55b050 2763 265248 System.Net.Sockets.Socket
00007f3f7d551b68 2763 265248 System.Data.SqlClient.SqlConnectionTimeoutPhaseDuration[]
00007f3f7d551288 2763 265248 System.Data.SqlClient.SessionData
00007f3f7d308170 2763 265248 System.Collections.Generic.Dictionary`2+Entry[[System.String, System.Private.CoreLib],[System.Tuple`2[[System.String, System.Private.CoreLib],[System.String, System.Private.CoreLib]], System.Private.CoreLib]][]
00007f3f77d50730 5526 265248 System.Threading.Tasks.ConcurrentExclusiveSchedulerPair+ConcurrentExclusiveTaskScheduler
00007f3f7d84a5d8 5538 265824 System.Collections.Generic.Queue`1[[System.Data.SqlClient.SNI.SNIMarsQueuedPacket, System.Data.SqlClient]]
00007f3f7d84a380 5538 265824 System.Collections.Generic.Queue`1[[System.Data.SqlClient.SNI.SNIPacket, System.Data.SqlClient]]
00007f3f7d556b20 8313 266016 System.Data.SqlClient.SNI.TdsParserStateObjectManaged+WritePacketCache
00007f3f7d84ab38 8325 266400 System.Data.SqlClient.SNI.SNISMUXHeader
00007f3f7d84ace8 2763 266592 System.Collections.Generic.Dictionary`2+Entry[[System.Int32, System.Private.CoreLib],[System.Data.SqlClient.SNI.SNIMarsHandle, System.Data.SqlClient]][]
00007f3f77d0e400 6853 274120 System.RuntimeType
00007f3f7d676570 2763 287352 System.Net.Sockets.SocketAsyncContext
00007f3f7d556f38 5522 309200 System.Data.SqlClient.SNI.SNIPacket[]
00007f3f7d5572b8 8313 332520 System.Data.SqlClient.SNI.SspiClientContextStatus
00007f3f7d556bd0 8313 332520 System.Collections.Generic.Stack`1[[System.Data.SqlClient.SNI.SNIPacket, System.Data.SqlClient]]
00007f3f77cc20f8 10619 341864 System.RuntimeType[]
00007f3f7d200b78 1438 345120 Microsoft.EntityFrameworkCore.Metadata.Internal.Property
00007f3f7d67bd48 5533 354112 System.Data.SqlClient.SNI.SNIAsyncCallback
00007f3f7d558118 2763 375768 System.Data.SqlClient.SNI.SNITCPHandle
00007f3f7d552f80 2763 397872 System.Data.SqlClient.TdsParser
00007f3f7d5379a8 2473 415464 System.Data.SqlClient.SqlConnectionString
00007f3f77d1a1d0 2728 436480 System.IO.IOException
00007f3f77cc3440 983 439804 System.Char[]
00007f3f77d1df58 11076 443040 System.Threading.ManualResetEventSlim
00007f3f79a0e038 2728 458304 System.Net.Sockets.SocketException
00007f3f7d551ad8 19341 464184 System.Data.SqlClient.SqlConnectionTimeoutPhaseDuration
00007f3f77d25470 22299 535176 System.WeakReference
00007f3f77359fd0 2614 562896 System.Collections.Generic.Dictionary`2+Entry[[System.String, System.Private.CoreLib],[System.String, System.Private.CoreLib]][]
00007f3f77d1ad60 6231 598176 System.Reflection.RuntimeParameterInfo
00007f3f7d308800 2763 641016 System.Collections.Concurrent.ConcurrentQueue`1+Segment[[System.Threading.Tasks.Task, System.Private.CoreLib]]
00007f3f7d556078 8313 665040 System.Collections.Generic.Dictionary`2[[System.Data.SqlClient.SNI.SNIPacket, System.Data.SqlClient],[System.Data.SqlClient.SNI.SNIPacket, System.Data.SqlClient]]
00007f3f7d538ad8 2526 687072 System.Collections.Concurrent.ConcurrentDictionary`2+Node[[System.Data.ProviderBase.DbConnectionPoolIdentity, System.Data.SqlClient],[System.Data.ProviderBase.DbConnectionPool, System.Data.SqlClient]][]
00007f3f7d53f5c0 2763 707328 System.Data.SqlClient.SqlInternalConnectionTds
00007f3f7d849098 5538 708864 System.Data.SqlClient.SNI.SNIMarsHandle
00007f3f77d1ace0 8936 714880 System.Signature
00007f3f7d671d78 2651 721072 System.Net.Sockets.Socket+Int32TaskSocketAsyncEventArgs
00007f3f77cf7e98 2763 751536 System.Collections.Concurrent.ConcurrentDictionary`2+Node[[System.Int32, System.Private.CoreLib],[System.Threading.Tasks.ConcurrentExclusiveSchedulerPair+ProcessingMode, System.Private.CoreLib]][]
00007f3f77ccb428 2765 773720 System.Threading.Tasks.Task[]
00007f3f7d539ab0 19454 778160 System.Diagnostics.Stopwatch
00007f3f7d8af068 4477 823768 System.Data.SqlClient._SqlMetaData
00007f3f7d537c88 17363 833424 System.Data.Common.NameValuePair
00007f3f7d8ae630 2763 950472 System.Data.ProviderBase.DbReferenceCollection+CollectionEntry[]
00007f3f77cc08a8 22211 1036504 System.Int32[]
00007f3f77cbefd8 9102 1043760 System.Object[]
00007f3f7d308750 2763 1171512 System.Threading.Tasks.SingleProducerSingleConsumerQueue`1+Segment[[System.Threading.Tasks.Task, System.Private.CoreLib]]
00007f3f77d13b80 11725 1219400 System.Reflection.RuntimeMethodInfo
00007f3f7d3089c8 2763 1480968 System.Collections.Concurrent.ConcurrentQueue`1+Segment+Slot[[System.Threading.Tasks.Task, System.Private.CoreLib]][]
00007f3f77d0b890 68168 1636032 System.Object
00007f3f7d555bb0 8313 3391704 System.Data.SqlClient.SNI.TdsParserStateObjectManaged
00007f3f77d0b460 68673 5385818 System.String
00007f3f77cd3890 2784 5727816 System.Byte[][]
00007f3f7d551410 2777 5753944 System.Data.SqlClient.SessionStateRecord[]
000000000239fd20 69262 41368920 Free
00007f3f77cc1210 76267 161547850 System.Byte[]
apparently, MARS in SqlClient caused the memory leak.
In my connection string MultipleActiveResultSets was set to true.I still don't know why this is affected just in Ububntu and not Windows too but when I set it to false, memory back to be stabled.
Related
TypeError: float() argument must be a string or a number, not 'Timestamp' to REGRESSION LINEAR
This is my code and it has an error that I don't know how to fix
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import datetime
from time import time
from sklearn.svm import SVR
from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score
Data
proof_df = pd.read_excel("WORK_NUM_3.xlsx")
Show information (only 10 of 1550)
proof_df.head(10)
results
ORDEN DATA N1 N2 N3 N4 N5
0 1 1994-03-13 25 45 60 76 79
1 2 1994-03-17 13 30 58 63 64
2 3 1994-03-20 5 15 32 33 48
3 4 1994-03-24 27 57 60 61 77
4 5 1994-03-27 19 44 53 54 71
5 6 1994-04-03 4 45 54 65 67
6 7 1994-04-07 9 21 37 42 68
7 8 1994-04-10 5 16 26 28 62
8 9 1994-04-14 4 15 44 64 73
9 10 1994-04-17 20 32 49 54 62
declare variables
y = proof_df.iloc[:, 2:len(quina_df.columns)]
X = proof_df[['ORDEN','DATA']]
regression algorithm
regresor = SVR(kernel='linear')
hora_inicio = time()
train of algorithm
regresor.fit(X_train.values, y_train.values.ravel())
print('train finish in {time() - hora_inicio} segundos')
result:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Input In [37], in <cell line: 2>()
1 # Entrenamiento del algoritmo
----> 2 regresor.fit(X_train.values, y_train.values.ravel())
3 print('Entrenamiento finalizado en {time() - hora_inicio} segundos')
File ~\anaconda3\lib\site-packages\sklearn\svm\_base.py:190, in BaseLibSVM.fit(self, X, y, sample_weight)
188 check_consistent_length(X, y)
189 else:
--> 190 X, y = self._validate_data(
191 X,
192 y,
193 dtype=np.float64,
194 order="C",
195 accept_sparse="csr",
196 accept_large_sparse=False,
197 )
199 y = self._validate_targets(y)
201 sample_weight = np.asarray(
202 [] if sample_weight is None else sample_weight, dtype=np.float64
203 )
File ~\anaconda3\lib\site-packages\sklearn\base.py:581, in BaseEstimator._validate_data(self, X, y, reset, validate_separately, **check_params)
579 y = check_array(y, **check_y_params)
580 else:
--> 581 X, y = check_X_y(X, y, **check_params)
582 out = X, y
584 if not no_val_X and check_params.get("ensure_2d", True):
File ~\anaconda3\lib\site-packages\sklearn\utils\validation.py:964, in check_X_y(X, y, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, multi_output, ensure_min_samples, ensure_min_features, y_numeric, estimator)
961 if y is None:
962 raise ValueError("y cannot be None")
--> 964 X = check_array(
965 X,
966 accept_sparse=accept_sparse,
967 accept_large_sparse=accept_large_sparse,
968 dtype=dtype,
969 order=order,
970 copy=copy,
971 force_all_finite=force_all_finite,
972 ensure_2d=ensure_2d,
973 allow_nd=allow_nd,
974 ensure_min_samples=ensure_min_samples,
975 ensure_min_features=ensure_min_features,
976 estimator=estimator,
977 )
979 y = _check_y(y, multi_output=multi_output, y_numeric=y_numeric)
981 check_consistent_length(X, y)
File ~\anaconda3\lib\site-packages\sklearn\utils\validation.py:746, in check_array(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, estimator)
744 array = array.astype(dtype, casting="unsafe", copy=False)
745 else:
--> 746 array = np.asarray(array, order=order, dtype=dtype)
747 except ComplexWarning as complex_warning:
748 raise ValueError(
749 "Complex data not supported\n{}\n".format(array)
750 ) from complex_warning
TypeError: float() argument must be a string or a number, not 'Timestamp'
I I am trying to work with the dates as the variable ( X ) that the model should receive since the variable ( y ) are the other results
I really appreciate any help to help understand what is going on
How to fix SVG animate which does not work in Safari
Not later than yesterday, I managed to make an animated SVG which works flawlessly in Chrome. I tried to open it in Safari today and the animation was not working. Is there a solution to this? I have tried to do some research but most of them show CSS3 and the d:path CSS method does not work either and is actually worse because nothing shows up. Here's my code <svg width="1279" height="650" viewBox="0 0 1279 650" xmlns="http://www.w3.org/2000/svg"> <path d="M-138 -167.889C-138 -167.889 2002 -494.389 1020.73 -167.889C39.4558 158.611 1111.22 305.526 942.746 516.955C681.205 845.177 -138 458.649 -138 458.649V -167.889Z" fill="#000000"> <!-- First shape and last shape in values are the same --> <animate repeatCount="indefinite" fill="#454599" attributeName="d" dur="12s" values=" M-138 -167.889C-138 -167.889 1986.46 -591.778 1020.73 -167.889C55 256 1148.99 306.909 942.746 516.954C736.5 727 -138 458.649 -138 458.649V-167.889Z; M-138 -167.889C-138 -167.889 1923.96 -760.278 1020.73 -167.889C117.5 424.5 1248.99 269.909 942.746 516.954C636.5 764 -138 458.649 -138 458.649V-167.889Z; M-138 -167.889C-138 -167.889 1989.46 -765.777 1020.73 -167.889C52 430 1302.99 226.348 942.746 516.955C582.5 807.561 -138 458.649 -138 458.649V-167.889Z; M-138 -167.889C-138 -167.889 1986.46 -591.778 1020.73 -167.889C55 256 1148.99 306.909 942.746 516.954C736.5 727 -138 458.649 -138 458.649V-167.889Z; "> </animate> </path> </svg>
I have tried to do some research but most of them show CSS3 and the
d:path CSS method does not work either and is actually worse because
nothing shows up.
Please see browser support d: path animations.
Unfortunately, this does not work in the specified browsers, including Safari.
Let's hope to see better support in the near future.
This answer will not solve your problem with CSS3 path animations as it is not possible, but it will make it easier with SVG animations that work in all modern browsers, including Safari.
Specifying path data: the d property:
For animation, two d property values can only be interpolated smoothly
when the path data strings contain have the same structure, (i.e.
exactly the same number and types of path data commands which are in
the same order). If an animation is specified and the lists of path
data commands do not have the same structure, then the values must be
interpolated using the discrete animation type.
To fulfill these requirements, you can use this technique
Step #1
First you need to get the shape of the curve at the starting position:
Step #2
Then, by moving the anchor points of the curve in the vector editor, we get the final shape of the curve:
Step #3
Next, we write a command to animate the curve:
<animate attributeName="d" values="path1;path2;path1" />, where
path1 - Curve shape at the start
path2 - Finish curve shape
as #Kaiido said in the comments try not to allow extra spaces before the semicolon
svg {
width:40%;
height:40%;
}
path {
fill:#2F3136;
stroke:black;
stroke-width:4;
}
<svg xmlns:svg="http://www.w3.org/2000/svg" xmlns="http://www.w3.org/2000/svg" version="1.1" width="770" height="770" viewBox="0 0 770 770" preserveAspectRatio="xMinYMin meet" >
<path d="M8.1 1.2V721.6c0 0 73.2 20.7 105 16.1 39.9-5.6 63.5-61.7 105-66.9 35-4.4 74.9 1 105-13.8 41.9-20.7 60.2-90.1 92.3-108.4 30.9-17.7 74.9-13.9 101.5-33.5 63.3-46.3 5.3-81.5 76.1-145.3 0 0 47.9-24.3 61.1-46.1 19.6-32.3 18.5-61.6 18.5-111.9 0-21.7 15.2-38.8 32.3-56.5C730.2 121.6 752.7 68.8 719.8 2.9 516.8-0.6 218-0.6 8.1 1.2Z" >
<animate attributeName="d"
dur="4s"
values="
M8.1 1.2V721.6c0 0 73.2 20.7 105 16.1 39.9-5.6 63.5-61.7 105-66.9 35-4.4 74.9 1 105-13.8 41.9-20.7 60.2-90.1 92.3-108.4 30.9-17.7 74.9-13.9 101.5-33.5 63.3-46.3 5.3-81.5 76.1-145.3 0 0 47.9-24.3 61.1-46.1 19.6-32.3 18.5-61.6 18.5-111.9 0-21.7 15.2-38.8 32.3-56.5C730.2 121.6 752.7 68.8 719.8 2.9 516.8-0.6 218-0.6 8.1 1.2Z;
M8.1 1.2V721.6c0 0 76.8 2.3 110.7-24.2 67.7-52.9 94.4-1.3 137.3-18.5 40.1-16.1 39.1-57.3 105-76.1 93.8-26.8 81.5-74.9 124.6-109.6 22.3-17.9 50.6-23.4 69.2-50.8 41.5-61.1 14.5-80.4 66.9-115.4 0 0 28.1-21.9 36.9-36.9 16.5-28.3 27.7-30.3 26.5-94.6-0.4-21.7 2.5-76.9 19.6-94.6C730.2 67.3 752.7 68.8 719.8 2.9 516.8-0.6 218-0.6 8.1 1.2Z;
M8.1 1.2V721.6c0 0 73.2 20.7 105 16.1 39.9-5.6 63.5-61.7 105-66.9 35-4.4 74.9 1 105-13.8 41.9-20.7 60.2-90.1 92.3-108.4 30.9-17.7 74.9-13.9 101.5-33.5 63.3-46.3 5.3-81.5 76.1-145.3 0 0 47.9-24.3 61.1-46.1 19.6-32.3 18.5-61.6 18.5-111.9 0-21.7 15.2-38.8 32.3-56.5C730.2 121.6 752.7 68.8 719.8 2.9 516.8-0.6 218-0.6 8.1 1.2Z"
repeatcount="indefinite"
/>
</path>
</svg>
Option with background image and shadow
.container {
width:75%;
height:75%;
}
path {
fill:#2F3136;
stroke:black;
stroke-width:0;
fill-opacity:0.5;
-webkit-filter: drop-shadow(4px 4px 1px black);
filter: drop-shadow(4px 4px 1px black);
}
<div class="container">
<svg xmlns:svg="http://www.w3.org/2000/svg" xmlns="http://www.w3.org/2000/svg" version="1.1" viewBox="0 0 770 770" preserveAspectRatio="none" >
<image xlink:href="https://i.stack.imgur.com/ABxSm.jpg" x="10" width="1125px" height="750px" >
</image>
<path d="M8.1 1.2V721.6c0 0 73.2 20.7 105 16.1 39.9-5.6 63.5-61.7 105-66.9 35-4.4 74.9 1 105-13.8 41.9-20.7 60.2-90.1 92.3-108.4 30.9-17.7 74.9-13.9 101.5-33.5 63.3-46.3 5.3-81.5 76.1-145.3 0 0 47.9-24.3 61.1-46.1 19.6-32.3 18.5-61.6 18.5-111.9 0-21.7 15.2-38.8 32.3-56.5C730.2 121.6 752.7 68.8 719.8 2.9 516.8-0.6 218-0.6 8.1 1.2Z" >
<animate id="an_d" attributeName="d"
dur="6s"
begin="0s"
values="
M8.1 1.2V721.6c0 0 73.2 20.7 105 16.1 39.9-5.6 63.5-61.7 105-66.9 35-4.4 74.9 1 105-13.8 41.9-20.7 60.2-90.1 92.3-108.4 30.9-17.7 74.9-13.9 101.5-33.5 63.3-46.3 5.3-81.5 76.1-145.3 0 0 47.9-24.3 61.1-46.1 19.6-32.3 18.5-61.6 18.5-111.9 0-21.7 15.2-38.8 32.3-56.5C730.2 121.6 752.7 68.8 719.8 2.9 516.8-0.6 218-0.6 8.1 1.2Z;
M8.1 1.2V721.6c0 0 76.8 2.3 110.7-24.2 67.7-52.9 94.4-1.3 137.3-18.5 40.1-16.1 39.1-57.3 105-76.1 93.8-26.8 81.5-74.9 124.6-109.6 22.3-17.9 50.6-23.4 69.2-50.8 41.5-61.1 14.5-80.4 66.9-115.4 0 0 28.1-21.9 36.9-36.9 16.5-28.3 27.7-30.3 26.5-94.6-0.4-21.7 2.5-76.9 19.6-94.6C730.2 67.3 752.7 68.8 719.8 2.9 516.8-0.6 218-0.6 8.1 1.2Z;
M8.1 1.2V721.6c0 0 73.2 20.7 105 16.1 39.9-5.6 63.5-61.7 105-66.9 35-4.4 74.9 1 105-13.8 41.9-20.7 60.2-90.1 92.3-108.4 30.9-17.7 74.9-13.9 101.5-33.5 63.3-46.3 5.3-81.5 76.1-145.3 0 0 47.9-24.3 61.1-46.1 19.6-32.3 18.5-61.6 18.5-111.9 0-21.7 15.2-38.8 32.3-56.5C730.2 121.6 752.7 68.8 719.8 2.9 516.8-0.6 218-0.6 8.1 1.2Z"
repeatCount="indefinite"
/>
</path>
</svg>
</div>
Border animation plus whole block size animation
.container {
width:75%;
height:75%;
}
path {
fill:#2F3136;
stroke:black;
stroke-width:0;
fill-opacity:0.5;
}
<div class="container">
<svg xmlns:svg="http://www.w3.org/2000/svg" xmlns="http://www.w3.org/2000/svg" version="1.1" viewBox="0 0 770 770" preserveAspectRatio="none" >
<defs>
<filter id='shadow'>
<feGaussianBlur in='SourceAlpha' stdDeviation='2' />
<feOffset dx='3' dy='3' result='blur' />
<feMerge>
<feMergeNode in='blur' />
<feMergeNode in='SourceGraphic' />
</feMerge>
</filter>
</defs>
<image xlink:href="https://i.stack.imgur.com/ABxSm.jpg" x="10" width="1125px" height="750px" >
</image>
<path filter="url(#shadow)" d="M8.1 1.2V721.6c0 0 73.2 20.7 105 16.1 39.9-5.6 63.5-61.7 105-66.9 35-4.4 74.9 1 105-13.8 41.9-20.7 60.2-90.1 92.3-108.4 30.9-17.7 74.9-13.9 101.5-33.5 63.3-46.3 5.3-81.5 76.1-145.3 0 0 47.9-24.3 61.1-46.1 19.6-32.3 18.5-61.6 18.5-111.9 0-21.7 15.2-38.8 32.3-56.5C730.2 121.6 752.7 68.8 719.8 2.9 516.8-0.6 218-0.6 8.1 1.2Z" >
<animate id="an_d" attributeName="d"
dur="6s"
begin="0s;an_t.end"
values="
M8.1 1.2V721.6c0 0 73.2 20.7 105 16.1 39.9-5.6 63.5-61.7 105-66.9 35-4.4 74.9 1 105-13.8 41.9-20.7 60.2-90.1 92.3-108.4 30.9-17.7 74.9-13.9 101.5-33.5 63.3-46.3 5.3-81.5 76.1-145.3 0 0 47.9-24.3 61.1-46.1 19.6-32.3 18.5-61.6 18.5-111.9 0-21.7 15.2-38.8 32.3-56.5C730.2 121.6 752.7 68.8 719.8 2.9 516.8-0.6 218-0.6 8.1 1.2Z;
M8.1 1.2V721.6c0 0 76.8 2.3 110.7-24.2 67.7-52.9 94.4-1.3 137.3-18.5 40.1-16.1 39.1-57.3 105-76.1 93.8-26.8 81.5-74.9 124.6-109.6 22.3-17.9 50.6-23.4 69.2-50.8 41.5-61.1 14.5-80.4 66.9-115.4 0 0 28.1-21.9 36.9-36.9 16.5-28.3 27.7-30.3 26.5-94.6-0.4-21.7 2.5-76.9 19.6-94.6C730.2 67.3 752.7 68.8 719.8 2.9 516.8-0.6 218-0.6 8.1 1.2Z;
M8.1 1.2V721.6c0 0 73.2 20.7 105 16.1 39.9-5.6 63.5-61.7 105-66.9 35-4.4 74.9 1 105-13.8 41.9-20.7 60.2-90.1 92.3-108.4 30.9-17.7 74.9-13.9 101.5-33.5 63.3-46.3 5.3-81.5 76.1-145.3 0 0 47.9-24.3 61.1-46.1 19.6-32.3 18.5-61.6 18.5-111.9 0-21.7 15.2-38.8 32.3-56.5C730.2 121.6 752.7 68.8 719.8 2.9 516.8-0.6 218-0.6 8.1 1.2Z"
repeatcount="1"
/>
<animateTransform id="an_t" attributeName="transform" type="scale" dur="12s" begin="an_d.end" values="1;0.5;0.5;1" />
</path>
</svg>
</div>
Transition animation does not work "path"
I cannot understand why a smooth transition does not work - like here <path class="st4" d="M1278.4000244,0c0,0-302.7999878,70.4000244-412.5999756,267.9000244s-47.5,285.5999756-172.7000122,423.5 C569.2999878,827.9000244,477.6000061,894.5999756-0.3,1022.2000122v55.8999634c0,0.1999512,0,0.4000244,0,0.5999756 c0.1,0.4000244,0.1,0.8000488,0.1,1.1999512c0,0.1999512,0,0.4000244,0,0.5999756H716.5 c0,0,342.5-392.2000122,434.8000488-485.2000122s244.9000244-240.7000122,393.4000244-339.5 s128.0999756-96.7000122,278.5999756-159.2000122c56.5-23.5,84.3000488-42.9000244,97.0999756-58.0999756V0H1278.4000244z"> <animate dur="2s" repeatCount="indefinite" attributeName="d" values=" M1278.4000244,0c0,0-302.7999878,70.4000244-412.5999756,267.9000244s-47.5,285.5999756-172.7000122,423.5 C569.2999878,827.9000244,477.6000061,894.5999756-0.3,1022.2000122v55.8999634c0,0.1999512,0,0.4000244,0,0.5999756 c0.1,0.4000244,0.1,0.8000488,0.1,1.1999512c0,0.1999512,0,0.4000244,0,0.5999756H716.5 c0,0,342.5-392.2000122,434.8000488-485.2000122s244.9000244-240.7000122,393.4000244-339.5 s128.0999756-96.7000122,278.5999756-159.2000122c56.5-23.5,84.3000488-42.9000244,97.0999756-58.0999756V0H1278.4000244z; M1278.4000244,0c0,0-302.7999878,70.4000244-412.5999756,267.9000244s-47.5,285.5999756-172.7000122,423.5 C569.2999878,827.9000244,477.6000061,894.5999756-0.3,1022.2000122v55.8999634c0,0.1999512,0,0.4000244,0,0.5999756 c0.1,0.4000244,0.1,0.8000488,0.1,1.1999512c0,0.1999512,0,0.4000244,0,0.5999756H716.5 c248.9938354-689.5708618,423.4897461-842.3468628,541.8529053-835.0881348 c68.5238037,4.2022705,177.447876,65.6728516,286.347168,10.3881226 c51.019043-25.9007568,64.4385986-51.3814697,151.5378418-100.8539429 c28.6922607-16.2972412,61.8179932-31.2513428,127.0621338-58.3460693c56.5-23.5,84.3000488-42.9000244,97.0999756-58.0999756V0 H1278.4000244z;"/> </path> It turns out that the animation jumps from one position to another.
As I've commented the two paths have to have the same number of points and the same commands. The values attribute has now 3 values: first path; second path; first path. When you draw your paths in Illustrator you can count the points. This may be useful although sometimes Illustrator can change commands. <svg viewBox="-50 -50 2000 2000"> <path class="st4" d="M1278.4000244,0 c0,0-302.7999878,70.4000244-412.5999756,267.9000244s-47.5,285.5999756-172.7000122,423.5 C569.2999878,827.9000244,477.6000061,894.5999756-0.3,1022.2000122v55.8999634 c0,0.1999512,0,0.4000244,0,0.5999756 c0.1,0.4000244,0.1,0.8000488,0.1,1.1999512 c0,0.1999512,0,0.4000244,0,0.5999756 H716.5 c0,0, 342.5-392.2000122, 434.8000488-485.2000122 s244.9000244 -240.7000122, 393.4000244 -339.5 s128.0999756 -96.7000122,278.5999756-159.2000122 c56.5-23.5,84.3000488-42.9000244,97.0999756-58.0999756 V0 H1278.4000244z"> <animate dur="2s" repeatCount="indefinite" attributeName="d" values=" M1278.400, 0.000 C1278.400, 0.000 975.600, 70.400 865.800, 267.900 C756.000, 465.400 818.300, 553.500 693.100, 691.400 C569.300, 827.900 477.600, 894.600 -0.300, 1022.200 C-0.300, 1040.833 -0.300, 1059.467 -0.300, 1078.100 C-0.300, 1078.300 -0.300, 1078.500 -0.300, 1078.700 C-0.200, 1079.100 -0.200, 1079.500 -0.200, 1079.900 C-0.200, 1080.100 -0.200, 1080.300 -0.200, 1080.500 C238.700, 1080.500 477.600, 1080.500 716.500, 1080.500 C716.500, 1080.500 1059.000, 688.300 1151.300, 595.300 C1243.600, 502.300 1396.200, 354.600 1544.700, 255.800 C1693.200, 157.000 1672.800, 159.100 1823.300, 96.600 C1879.800, 73.100 1907.600, 53.700 1920.400, 38.500 C1920.400, 25.667 1920.400, 12.833 1920.400, 0.000 C1706.400, 0.000 1492.400, 0.000 1278.400, 0.000 z; M1278.400, 0.000 C1278.400, 0.000 975.600, 70.400 865.800, 267.900 C756.000, 465.400 818.300, 553.500 693.100, 691.400 C569.300, 827.900 477.600, 894.600 -0.300, 1022.200 C-0.300, 1040.833 -0.300, 1059.467 -0.300, 1078.100 C-0.300, 1078.300 -0.300, 1078.500 -0.300, 1078.700 C-0.200, 1079.100 -0.200, 1079.500 -0.200, 1079.900 C-0.200, 1080.100 -0.200, 1080.300 -0.200, 1080.500 C238.700, 1080.500 477.600, 1080.500 716.500, 1080.500 C965.494, 390.929 1139.990, 238.153 1258.353, 245.412 C1326.877, 249.614 1435.801, 311.085 1544.700, 255.800 C1724.930, 138.649 1758.056, 123.695 1823.300, 96.600 C1879.800, 73.100 1907.600, 53.700 1920.400, 38.500 C1920.400, 25.667 1920.400, 12.833 1920.400, 0.000 C1706.400, 0.000 1492.400, 0.000 1278.400, 0.000 z; M1278.400, 0.000 C1278.400, 0.000 975.600, 70.400 865.800, 267.900 C756.000, 465.400 818.300, 553.500 693.100, 691.400 C569.300, 827.900 477.600, 894.600 -0.300, 1022.200 C-0.300, 1040.833 -0.300, 1059.467 -0.300, 1078.100 C-0.300, 1078.300 -0.300, 1078.500 -0.300, 1078.700 C-0.200, 1079.100 -0.200, 1079.500 -0.200, 1079.900 C-0.200, 1080.100 -0.200, 1080.300 -0.200, 1080.500 C238.700, 1080.500 477.600, 1080.500 716.500, 1080.500 C716.500, 1080.500 1059.000, 688.300 1151.300, 595.300 C1243.600, 502.300 1396.200, 354.600 1544.700, 255.800 C1693.200, 157.000 1672.800, 159.100 1823.300, 96.600 C1879.800, 73.100 1907.600, 53.700 1920.400, 38.500 C1920.400, 25.667 1920.400, 12.833 1920.400, 0.000 C1706.400, 0.000 1492.400, 0.000 1278.400, 0.000 z;"/> </path> </svg>
How to divide the dataframe into bins of specific length with unequal number of points?
I have a dataframe and I want to divide that dataframe into bins of equal width (Number of data points in each bins may not be same). I have tried the following approach df = pc13.sort_values(by = ['A'], ascending=True) df_temp = np.array_split(df, 20) But this approach is dividing the dataframe into bins with equal number of data points. Instead of that I want to divide the dataframe into bins of particular width, also number of data points in each bin may not be same. The minimum value in the dataframe column A is -0.04843731030699292 and maximum value is 0.05417013917000033. I tried uploading the entire dataframe but it is very big file.
you can do something like:
# create a random df
df = pd.DataFrame(np.random.randn(10, 10), columns=list('ABCDEFGHIJ'))
# sort valeus
df = df.sort_values(by = ['A'], ascending=True)
# use your code but on a transposed dataframe
new = np.array_split(df.T, 5) # split columns into 5 bins
# list comprehension to transposed dataframes
dfs = [new[i].T for i in range(len(new))]
update
# random df
df = pd.DataFrame(np.random.randn(1000, 5), columns=list('ABCDE'))
# sort on A
df.sort_values('A', inplace=True)
# create bins
df['bin'] = pd.cut(df['A'], 20, include_lowest = True)
# group on bin
group = df.groupby('bin')
# list comprehension to split groups into list of dataframes
dfs = [group.get_group(x) for x in group.groups]
[ A B C D E bin
218 -2.716093 0.833726 -0.771400 0.691251 0.162448 (-2.723, -2.413]
207 -2.581388 -2.318333 -0.001467 0.035277 1.219666 (-2.723, -2.413]
380 -2.499710 1.946709 -0.519070 1.653383 0.309689 (-2.723, -2.413]
866 -2.492050 0.246500 -0.596392 0.872888 2.371652 (-2.723, -2.413]
876 -2.469238 -0.156470 -0.841065 -1.248793 -0.489665 (-2.723, -2.413]
314 -2.456308 0.630691 -0.072146 1.139697 0.663674 (-2.723, -2.413]
310 -2.455353 0.075842 0.589515 -0.427233 1.207979 (-2.723, -2.413]
660 -2.427255 0.890125 -0.042716 -1.038401 0.651324 (-2.723, -2.413],
A B C D E bin
571 -2.355430 0.383794 -1.266575 -1.214833 -0.862611 (-2.413, -2.11]
977 -2.354416 -1.964189 0.440376 0.028032 -0.181360 (-2.413, -2.11]
83 -2.276908 0.288462 0.370555 -0.546359 -2.033892 (-2.413, -2.11]
196 -2.213729 -1.087783 -0.592884 1.233886 1.051164 (-2.413, -2.11]
227 -2.146631 0.365183 -0.095293 -0.882414 0.385117 (-2.413, -2.11]
39 -2.136800 -1.150065 0.180182 -0.424071 0.040370 (-2.413, -2.11],
A B C D E bin
104 -2.108961 -0.396602 -1.014224 -1.277124 0.001030 (-2.11, -1.806]
360 -2.098928 1.093483 1.438421 -0.980215 0.010359 (-2.11, -1.806]
530 -2.088592 1.043201 -0.522468 0.482176 -0.680166 (-2.11, -1.806]
158 -2.062759 2.070387 2.124621 -2.751532 0.674055 (-2.11, -1.806]
971 -2.053039 0.347577 -0.498513 1.917305 -1.746493 (-2.11, -1.806]
658 -2.002482 -1.222292 -0.398816 0.279228 -1.485782 (-2.11, -1.806]
90 -1.985261 3.499251 -2.089028 1.238524 -1.781089 (-2.11, -1.806]
466 -1.973640 -1.609920 -1.029454 0.809143 -0.228893 (-2.11, -1.806]
40 -1.966016 -1.479240 -1.564966 -0.310133 1.338023 (-2.11, -1.806]
279 -1.943666 0.762493 0.060038 0.449159 0.244411 (-2.11, -1.806]
204 -1.940045 0.844901 -0.343691 -1.144836 1.385915 (-2.11, -1.806]
780 -1.918548 0.212452 0.225789 0.216110 1.710532 (-2.11, -1.806]
289 -1.897438 0.847664 0.689778 -0.454152 -0.747836 (-2.11, -1.806]
159 -1.848425 0.477726 0.391384 -0.477804 0.168160 (-2.11, -1.806],
. . .
Areas between lines not filled correctly with "fill_between" in Matplotlib
Set of time-series data are plotted with Matplotlib as:
import matplotlib.pyplot as plt
%matplotlib inline
def WillR(ohlc, window):
high, low, close = ohlc['High'], ohlc['Low'], ohlc['Close']
R = (np.nanmax(high, axis=0) - close) / (np.nanmax(high, axis=0) - np.nanmin(low, axis=0)) * -100
return R
ohlc = df[::-1]
R = WillR(ohlc, 14)
xAxis = ohlc.index
fig = plt.figure(figsize=(18,9), dpi=100)
ax = fig.add_axes([0.0, 0.6, 1.0, 0.6])
plt.ylabel('Williams '+'%R', fontsize = 16)
ax.xaxis.grid(color='grey', linestyle=':')
ax.yaxis.grid(color='grey', linestyle=':')
ax.plot(R, color='k', alpha=0.8, linewidth=1.2)
ax.axhline(-20, color='r', linewidth=0.8)
ax.axhline(-50, color='k', linestyle='dashed', linewidth=1.2)
ax.axhline(-80, color='g', linewidth=0.8)
ax.fill_between(xAxis, R, -20, where=(R >= -20), facecolor='r', edgecolor='r', alpha=0.5)
ax.fill_between(xAxis, R, -80, where=(R <= -80), facecolor='g', edgecolor='g', alpha=0.5)
In the "Williams %R" graph, the area above "-20" line is to be filled in red and below "-80" green. I could not get the bound areas filled completely - see Figure 1:
I have also tried adding the argument "interpolate=True" and the result is still unsatisfactory - see Figure 2:
How do I make it work correctly? Thanks
Sample data for df as follows:
Date Open High Low Close
Date
2017-06-27 2017-06-27 10.75 10.850 10.665 10.70
2017-06-26 2017-06-26 10.57 10.740 10.570 10.69
2017-06-23 2017-06-23 10.59 10.710 10.500 10.63
2017-06-22 2017-06-22 10.56 10.720 10.530 10.65
2017-06-21 2017-06-21 10.75 10.750 10.500 10.53
2017-06-20 2017-06-20 10.88 10.925 10.780 10.81
2017-06-19 2017-06-19 10.77 10.920 10.760 10.82
2017-06-16 2017-06-16 10.74 10.910 10.740 10.78
2017-06-15 2017-06-15 10.86 10.930 10.690 10.76
2017-06-14 2017-06-14 10.96 11.040 10.910 11.02
2017-06-13 2017-06-13 10.57 10.930 10.570 10.93
2017-06-09 2017-06-09 10.59 10.650 10.500 10.57
2017-06-08 2017-06-08 10.28 10.590 10.280 10.57
2017-06-07 2017-06-07 10.35 10.420 10.290 10.35
2017-06-06 2017-06-06 10.66 10.700 10.280 10.38
2017-06-05 2017-06-05 10.80 10.850 10.560 10.70
2017-06-02 2017-06-02 11.32 11.390 11.190 11.19
2017-06-01 2017-06-01 11.26 11.330 11.160 11.20
2017-05-31 2017-05-31 11.22 11.400 11.190 11.27
2017-05-30 2017-05-30 11.10 11.260 11.070 11.23
2017-05-29 2017-05-29 11.36 11.370 11.045 11.08
2017-05-26 2017-05-26 11.55 11.590 11.320 11.36
2017-05-25 2017-05-25 11.62 11.670 11.455 11.66
2017-05-24 2017-05-24 11.67 11.755 11.590 11.62
2017-05-23 2017-05-23 11.77 11.835 11.590 11.64
2017-05-22 2017-05-22 12.06 12.110 11.800 11.83
2017-05-19 2017-05-19 12.15 12.235 11.990 12.03
2017-05-18 2017-05-18 12.00 12.130 11.910 12.13
2017-05-17 2017-05-17 12.28 12.300 12.120 12.14
2017-05-16 2017-05-16 12.23 12.385 12.220 12.30
2017-05-15 2017-05-15 12.10 12.255 12.030 12.22
2017-05-12 2017-05-12 12.30 12.300 12.050 12.12
2017-05-11 2017-05-11 12.30 12.445 12.270 12.34
2017-05-10 2017-05-10 11.99 12.415 11.950 12.25
2017-05-09 2017-05-09 12.07 12.100 11.730 11.79
2017-05-08 2017-05-08 12.14 12.230 12.060 12.10
2017-05-05 2017-05-05 12.24 12.250 12.020 12.04
2017-05-04 2017-05-04 12.23 12.270 12.050 12.14
2017-05-03 2017-05-03 12.43 12.460 12.200 12.23
2017-05-02 2017-05-02 12.44 12.470 12.310 12.46
2017-05-01 2017-05-01 12.25 12.440 12.230 12.44
2017-04-28 2017-04-28 12.47 12.480 12.230 12.32
2017-04-27 2017-04-27 12.39 12.515 12.380 12.50
2017-04-26 2017-04-26 12.03 12.510 12.030 12.37
2017-04-24 2017-04-24 12.00 12.055 11.920 12.03
2017-04-21 2017-04-21 11.88 11.990 11.840 11.88
2017-04-20 2017-04-20 11.78 11.840 11.710 11.80
2017-04-19 2017-04-19 11.70 11.745 11.610 11.74
2017-04-18 2017-04-18 11.95 11.950 11.605 11.74
2017-04-13 2017-04-13 11.95 12.010 11.920 11.95
2017-04-12 2017-04-12 12.05 12.050 11.945 12.01
2017-04-11 2017-04-11 11.95 12.140 11.945 12.08
2017-04-10 2017-04-10 11.79 11.930 11.780 11.91
2017-04-07 2017-04-07 11.83 11.830 11.690 11.78
2017-04-06 2017-04-06 11.85 11.870 11.740 11.78
2017-04-05 2017-04-05 12.06 12.140 11.800 11.96
2017-04-04 2017-04-04 12.03 12.160 11.930 12.07
2017-04-03 2017-04-03 12.14 12.180 11.860 11.98
2017-03-30 2017-03-30 12.01 12.230 11.970 12.14
2017-03-29 2017-03-29 11.99 12.050 11.900 12.03
The data needs to be sorted. Then the option interpolate=True will provide the required plot.
u = u""" Date Date1 Open High Low Close
2017-06-27 2017-06-27 10.75 10.850 10.665 10.70
2017-06-26 2017-06-26 10.57 10.740 10.570 10.69
2017-06-23 2017-06-23 10.59 10.710 10.500 10.63
2017-06-22 2017-06-22 10.56 10.720 10.530 10.65
2017-06-21 2017-06-21 10.75 10.750 10.500 10.53
2017-06-20 2017-06-20 10.88 10.925 10.780 10.81
2017-06-19 2017-06-19 10.77 10.920 10.760 10.82
2017-06-16 2017-06-16 10.74 10.910 10.740 10.78
2017-06-15 2017-06-15 10.86 10.930 10.690 10.76
2017-06-14 2017-06-14 10.96 11.040 10.910 11.02
2017-06-13 2017-06-13 10.57 10.930 10.570 10.93
2017-06-09 2017-06-09 10.59 10.650 10.500 10.57
2017-06-08 2017-06-08 10.28 10.590 10.280 10.57
2017-06-07 2017-06-07 10.35 10.420 10.290 10.35
2017-06-06 2017-06-06 10.66 10.700 10.280 10.38
2017-06-05 2017-06-05 10.80 10.850 10.560 10.70
2017-06-02 2017-06-02 11.32 11.390 11.190 11.19
2017-06-01 2017-06-01 11.26 11.330 11.160 11.20
2017-05-31 2017-05-31 11.22 11.400 11.190 11.27
2017-05-30 2017-05-30 11.10 11.260 11.070 11.23
2017-05-29 2017-05-29 11.36 11.370 11.045 11.08
2017-05-26 2017-05-26 11.55 11.590 11.320 11.36
2017-05-25 2017-05-25 11.62 11.670 11.455 11.66
2017-05-24 2017-05-24 11.67 11.755 11.590 11.62
2017-05-23 2017-05-23 11.77 11.835 11.590 11.64
2017-05-22 2017-05-22 12.06 12.110 11.800 11.83
2017-05-19 2017-05-19 12.15 12.235 11.990 12.03
2017-05-18 2017-05-18 12.00 12.130 11.910 12.13
2017-05-17 2017-05-17 12.28 12.300 12.120 12.14
2017-05-16 2017-05-16 12.23 12.385 12.220 12.30
2017-05-15 2017-05-15 12.10 12.255 12.030 12.22
2017-05-12 2017-05-12 12.30 12.300 12.050 12.12
2017-05-11 2017-05-11 12.30 12.445 12.270 12.34
2017-05-10 2017-05-10 11.99 12.415 11.950 12.25
2017-05-09 2017-05-09 12.07 12.100 11.730 11.79
2017-05-08 2017-05-08 12.14 12.230 12.060 12.10
2017-05-05 2017-05-05 12.24 12.250 12.020 12.04
2017-05-04 2017-05-04 12.23 12.270 12.050 12.14
2017-05-03 2017-05-03 12.43 12.460 12.200 12.23
2017-05-02 2017-05-02 12.44 12.470 12.310 12.46
2017-05-01 2017-05-01 12.25 12.440 12.230 12.44
2017-04-28 2017-04-28 12.47 12.480 12.230 12.32
2017-04-27 2017-04-27 12.39 12.515 12.380 12.50
2017-04-26 2017-04-26 12.03 12.510 12.030 12.37
2017-04-24 2017-04-24 12.00 12.055 11.920 12.03
2017-04-21 2017-04-21 11.88 11.990 11.840 11.88
2017-04-20 2017-04-20 11.78 11.840 11.710 11.80
2017-04-19 2017-04-19 11.70 11.745 11.610 11.74
2017-04-18 2017-04-18 11.95 11.950 11.605 11.74
2017-04-13 2017-04-13 11.95 12.010 11.920 11.95
2017-04-12 2017-04-12 12.05 12.050 11.945 12.01
2017-04-11 2017-04-11 11.95 12.140 11.945 12.08
2017-04-10 2017-04-10 11.79 11.930 11.780 11.91
2017-04-07 2017-04-07 11.83 11.830 11.690 11.78
2017-04-06 2017-04-06 11.85 11.870 11.740 11.78
2017-04-05 2017-04-05 12.06 12.140 11.800 11.96
2017-04-04 2017-04-04 12.03 12.160 11.930 12.07
2017-04-03 2017-04-03 12.14 12.180 11.860 11.98
2017-03-30 2017-03-30 12.01 12.230 11.970 12.14
2017-03-29 2017-03-29 11.99 12.050 11.900 12.03"""
import io
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
#%matplotlib inline
def WillR(ohlc, window):
high, low, close = ohlc['High'], ohlc['Low'], ohlc['Close']
R = (np.nanmax(high, axis=0) - close) / (np.nanmax(high, axis=0) - np.nanmin(low, axis=0)) * -100
return R
df = pd.read_csv(io.StringIO(u), delim_whitespace=True, index_col=0, parse_dates=True)
ohlc = df[::-1]
R = WillR(ohlc, 14)
xAxis = ohlc.index
fig = plt.figure(figsize=(18,9), dpi=100)
ax = plt.subplot()
plt.ylabel('Williams '+'%R', fontsize = 16)
ax.xaxis.grid(color='grey', linestyle=':')
ax.yaxis.grid(color='grey', linestyle=':')
ax.plot(R, color='k', alpha=0.8, linewidth=1.2)
ax.axhline(-20, color='r', linewidth=0.8)
ax.axhline(-50, color='k', linestyle='dashed', linewidth=1.2)
ax.axhline(-80, color='g', linewidth=0.8)
ax.fill_between(xAxis, R, -20, where=(R >= -20), facecolor='r', edgecolor='r', alpha=0.5, interpolate=True)
ax.fill_between(xAxis, R, -80, where=(R <= -80), facecolor='g', edgecolor='g', alpha=0.5, interpolate=True)
plt.show()