After porting our app with Gluon to iOS we noticed that is runs very slowly in the simulator as well as on real devices (in my case iPad 3 running iOS7).
We used profiling tools provided by XCode to examine possible reasons and found some memory leaks.
Then we tried a basic Gluon project (Single View, empty) and found the same memory leaks.
I am neither an expert with RoboVM nor with JavaFXPorts/Gluon so I don't know where to look. But I could provide more information when you tell me what you need.
I appreciate any help and any other suggestions how to make the app more responsive and faster because the memory leaks seem to be just a part of the problem.
Here is the memory leak stacktrace provided by XCode for a basic HelloWorld Application:
Bytes Used # Leaks Symbol Name\
8.72 KB 86.5% 62 start\
8.72 KB 86.5% 62 main\
8.72 KB 86.5% 62 rvmRun\
8.72 KB 86.5% 62 rvmCallVoidClassMethod\
8.72 KB 86.5% 62 rvmCallVoidClassMethodA\
8.72 KB 86.5% 62 callVoidMethod\
8.72 KB 86.5% 62 _call0\
8.72 KB 86.5% 62 [J]org.javafxports.jfxmobile.ios.BasicLauncher.main([Ljava/lang/String;)V\
8.72 KB 86.5% 62 [j]org.robovm.apple.uikit.UIApplication.main([Ljava/lang/String;Ljava/lang/Class;Ljava/lang/Class;)V[clinit]\
8.72 KB 86.5% 62 [J]org.robovm.apple.uikit.UIApplication.main([Ljava/lang/String;Ljava/lang/Class;Ljava/lang/Class;)V\
8.72 KB 86.5% 62 [J]org.robovm.apple.uikit.UIApplication.main(ILorg/robovm/rt/bro/ptr/BytePtr$BytePtrPtr;Ljava/lang/String;Ljava/lang/String;)I\
8.72 KB 86.5% 62 UIApplicationMain\
8.72 KB 86.5% 62 GSEventRunModal\
8.72 KB 86.5% 62 CFRunLoopRunInMode\
8.72 KB 86.5% 62 CFRunLoopRunSpecific\
8.72 KB 86.5% 62 __CFRunLoopRun\
8.72 KB 86.5% 62 __CFRunLoopDoSources0\
8.72 KB 86.5% 62 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__\
8.72 KB 86.5% 62 __NSThreadPerformPerform\
8.72 KB 86.5% 62 -[GlassRunnable run]\
8.72 KB 86.5% 62 CallVoidMethod\
8.72 KB 86.5% 62 rvmCallVoidInstanceMethodA\
8.72 KB 86.5% 62 callVoidMethod\
8.72 KB 86.5% 62 _call0\
8.72 KB 86.5% 62 [J]com.sun.javafx.application.PlatformImpl$$Lambda$6.run()V\
8.72 KB 86.5% 62 [j]com.sun.javafx.application.PlatformImpl.lambda$runLater$174(Ljava/lang/Runnable;Ljava/security/AccessControlContext;)V[clinit]\
8.72 KB 86.5% 62 [J]com.sun.javafx.application.PlatformImpl.lambda$runLater$174(Ljava/lang/Runnable;Ljava/security/AccessControlContext;)V\
8.72 KB 86.5% 62 [j]java.security.AccessController.doPrivileged(Ljava/security/PrivilegedAction;Ljava/security/AccessControlContext;)Ljava/lang/Object;[clinit]\
8.72 KB 86.5% 62 [J]java.security.AccessController.doPrivileged(Ljava/security/PrivilegedAction;Ljava/security/AccessControlContext;)Ljava/lang/Object;\
8.72 KB 86.5% 62 [J]com.sun.javafx.application.PlatformImpl$$Lambda$19.run()Ljava/lang/Object;\
8.72 KB 86.5% 62 [j]com.sun.javafx.application.PlatformImpl.lambda$null$173(Ljava/lang/Runnable;)Ljava/lang/Void;[clinit]\
8.72 KB 86.5% 62 [J]com.sun.javafx.application.PlatformImpl.lambda$null$173(Ljava/lang/Runnable;)Ljava/lang/Void;\
8.72 KB 86.5% 62 [J]com.sun.javafx.application.PlatformImpl$$Lambda$7.run()V\
8.72 KB 86.5% 62 [j]com.sun.javafx.application.PlatformImpl.lambda$runAndWait$175(Ljava/lang/Runnable;Ljava/util/concurrent/CountDownLatch;)V[clinit]\
8.72 KB 86.5% 62 [J]com.sun.javafx.application.PlatformImpl.lambda$runAndWait$175(Ljava/lang/Runnable;Ljava/util/concurrent/CountDownLatch;)V\
8.72 KB 86.5% 62 [J]com.sun.javafx.application.LauncherImpl$$Lambda$9.run()V\
8.72 KB 86.5% 62 [j]com.sun.javafx.application.LauncherImpl.lambda$launchApplication1$162(Ljava/util/concurrent/atomic/AtomicBoolean;Ljavafx/application/Application;)V[clinit]\
8.72 KB 86.5% 62 [J]com.sun.javafx.application.LauncherImpl.lambda$launchApplication1$162(Ljava/util/concurrent/atomic/AtomicBoolean;Ljavafx/application/Application;)V\
8.72 KB 86.5% 62 [J]com.gluonhq.charm.glisten.application.MobileApplication.start(Ljavafx/stage/Stage;)V\
8.72 KB 86.5% 62 [j]java.lang.String.format(Ljava/lang/String;[Ljava/lang/Object;)Ljava/lang/String;[clinit]\
8.72 KB 86.5% 62 [J]java.lang.String.format(Ljava/lang/String;[Ljava/lang/Object;)Ljava/lang/String;\
8.72 KB 86.5% 62 [J]java.lang.String.format(Ljava/util/Locale;Ljava/lang/String;[Ljava/lang/Object;)Ljava/lang/String;\
8.72 KB 86.5% 62 [J]java.util.Formatter.format(Ljava/lang/String;[Ljava/lang/Object;)Ljava/util/Formatter;\
8.72 KB 86.5% 62 [J]java.util.Formatter.format(Ljava/util/Locale;Ljava/lang/String;[Ljava/lang/Object;)Ljava/util/Formatter;\
8.72 KB 86.5% 62 [j]libcore.icu.LocaleData.get(Ljava/util/Locale;)Llibcore/icu/LocaleData;[clinit]\
8.72 KB 86.5% 62 _bcInitializeClass\
8.72 KB 86.5% 62 rvmInitialize\
8.72 KB 86.5% 62 _call0\
8.72 KB 86.5% 62 [J]libcore.icu.LocaleData.<clinit>()V\
8.72 KB 86.5% 62 [J]libcore.icu.LocaleData.get(Ljava/util/Locale;)Llibcore/icu/LocaleData;\
8.72 KB 86.5% 62 [J]libcore.icu.LocaleData.initLocaleData(Ljava/util/Locale;)Llibcore/icu/LocaleData;\
7.31 KB 72.5% 52 [j]libcore.icu.ICU.getBestDateTimePattern(Ljava/lang/String;Ljava/lang/String;)Ljava/lang/String;[clinit]\
7.31 KB 72.5% 52 [J]libcore.icu.ICU.getBestDateTimePattern(Ljava/lang/String;Ljava/lang/String;)Ljava/lang/String;\
7.31 KB 72.5% 52 [J]libcore.icu.ICU.getBestDateTimePatternNative(Ljava/lang/String;Ljava/lang/String;)Ljava/lang/String;\
7.31 KB 72.5% 52 Java_libcore_icu_ICU_getBestDateTimePatternNative\
7.31 KB 72.5% 52 icu_51::DateTimePatternGenerator::createInstance(icu_51::Locale const&, UErrorCode&)\
7.31 KB 72.5% 52 icu_51::DateTimePatternGenerator::DateTimePatternGenerator(icu_51::Locale const&, UErrorCode&)\
6.47 KB 64.1% 46 icu_51::DateTimePatternGenerator::initData(icu_51::Locale const&, UErrorCode&)\
6.47 KB 64.1% 46 icu_51::DateTimePatternGenerator::addICUPatterns(icu_51::Locale const&, UErrorCode&)\
6.47 KB 64.1% 46 icu_51::DateFormat::create(icu_51::DateFormat::EStyle, icu_51::DateFormat::EStyle, icu_51::Locale const&)\
6.47 KB 64.1% 46 icu_51::SimpleDateFormat::SimpleDateFormat(icu_51::DateFormat::EStyle, icu_51::DateFormat::EStyle, icu_51::Locale const&, UErrorCode&)\
6.47 KB 64.1% 46 icu_51::SimpleDateFormat::construct(icu_51::DateFormat::EStyle, icu_51::DateFormat::EStyle, icu_51::Locale const&, UErrorCode&)\
6.47 KB 64.1% 46 icu_51::SimpleDateFormat::initialize(icu_51::Locale const&, UErrorCode&)\
6.47 KB 64.1% 46 icu_51::NumberFormat::makeInstance(icu_51::Locale const&, UNumberFormatStyle, signed char, UErrorCode&)\
6.47 KB 64.1% 46 icu_51::DecimalFormatSymbols::DecimalFormatSymbols(icu_51::Locale const&, UErrorCode&)\
6.47 KB 64.1% 46 icu_51::DecimalFormatSymbols::initialize(icu_51::Locale const&, UErrorCode&, signed char)\
6.47 KB 64.1% 46 ures_open_51\
6.47 KB 64.1% 46 malloc\
6.47 KB 64.1% 46 malloc_zone_malloc\
864 Bytes 8.3% 6 icu_51::DateTimePatternGenerator::setDecimalSymbols(icu_51::Locale const&, UErrorCode&)\
864 Bytes 8.3% 6 icu_51::DecimalFormatSymbols::DecimalFormatSymbols(icu_51::Locale const&, UErrorCode&)\
864 Bytes 8.3% 6 icu_51::DecimalFormatSymbols::initialize(icu_51::Locale const&, UErrorCode&, signed char)\
864 Bytes 8.3% 6 ures_open_51\
864 Bytes 8.3% 6 malloc\
864 Bytes 8.3% 6 malloc_zone_malloc\
1.41 KB 13.9% 10 [j]libcore.icu.ICU.initLocaleDataNative(Ljava/lang/String;Llibcore/icu/LocaleData;)Z[clinit]\
1.41 KB 13.9% 10 [J]libcore.icu.ICU.initLocaleDataNative(Ljava/lang/String;Llibcore/icu/LocaleData;)Z\
1.41 KB 13.9% 10 Java_libcore_icu_ICU_initLocaleDataNative\
1.12 KB 11.1% 8 icu_51::NumberFormat::makeInstance(icu_51::Locale const&, UNumberFormatStyle, signed char, UErrorCode&)\
1.12 KB 11.1% 8 icu_51::DecimalFormatSymbols::DecimalFormatSymbols(icu_51::Locale const&, UErrorCode&)\
1.12 KB 11.1% 8 icu_51::DecimalFormatSymbols::initialize(icu_51::Locale const&, UErrorCode&, signed char)\
1.12 KB 11.1% 8 ures_open_51\
1.12 KB 11.1% 8 malloc\
1.12 KB 11.1% 8 malloc_zone_malloc\
288 Bytes 2.7% 2 icu_51::DecimalFormatSymbols::DecimalFormatSymbols(icu_51::Locale const&, UErrorCode&)\
288 Bytes 2.7% 2 icu_51::DecimalFormatSymbols::initialize(icu_51::Locale const&, UErrorCode&, signed char)\
288 Bytes 2.7% 2 ures_open_51\
288 Bytes 2.7% 2 malloc\
1.36 KB 13.4% 5 thread_start\
1.36 KB 13.4% 5 _pthread_start\
1.36 KB 13.4% 5 _pthread_body\
1.36 KB 13.4% 5 GC_start_routine\
1.36 KB 13.4% 5 GC_call_with_stack_base\
1.36 KB 13.4% 5 GC_inner_start_routine\
1.36 KB 13.4% 5 startThreadEntryPoint\
1.36 KB 13.4% 5 rvmCallVoidInstanceMethodA\
1.36 KB 13.4% 5 callVoidMethod\
1.36 KB 13.4% 5 _call0\
1.36 KB 13.4% 5 [J]java.lang.Thread.run()V\
1.36 KB 13.4% 5 [J]com.sun.javafx.tk.quantum.QuantumRenderer$PipelineRunnable.run()V\
1.36 KB 13.4% 5 [J]java.util.concurrent.ThreadPoolExecutor$Worker.run()V\
1.36 KB 13.4% 5 [J]java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V\
1.36 KB 13.4% 5 [J]com.sun.javafx.tk.RenderJob.run()V\
1.36 KB 13.4% 5 [J]java.util.concurrent.FutureTask.runAndReset()Z\
1.36 KB 13.4% 5 [J]java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;\
1.36 KB 13.4% 5 [J]com.sun.javafx.tk.quantum.QuantumRenderer$$Lambda$2.run()V\
1.36 KB 13.4% 5 [j]com.sun.javafx.tk.quantum.QuantumRenderer.lambda$createResourceFactory$414()V[clinit]\
1.36 KB 13.4% 5 [J]com.sun.javafx.tk.quantum.QuantumRenderer.lambda$createResourceFactory$414()V\
1.36 KB 13.4% 5 [j]com.sun.prism.GraphicsPipeline.getDefaultResourceFactory()Lcom/sun/prism/ResourceFactory;[clinit]\
1.36 KB 13.4% 5 [J]com.sun.prism.GraphicsPipeline.getDefaultResourceFactory()Lcom/sun/prism/ResourceFactory;\
1.36 KB 13.4% 5 [J]com.sun.prism.es2.ES2Pipeline.getDefaultResourceFactory(Ljava/util/List;)Lcom/sun/prism/ResourceFactory;\
1.36 KB 13.4% 5 [J]com.sun.prism.es2.ES2Pipeline.findDefaultResourceFactory(Ljava/util/List;)Lcom/sun/prism/es2/ES2ResourceFactory;\
1.36 KB 13.4% 5 [J]com.sun.prism.es2.ES2Pipeline.getES2ResourceFactory(ILcom/sun/glass/ui/Screen;)Lcom/sun/prism/es2/ES2ResourceFactory;\
1.36 KB 13.4% 5 [J]com.sun.prism.es2.ES2ResourceFactory.<init>(Lcom/sun/glass/ui/Screen;)V\
1.36 KB 13.4% 5 [J]com.sun.prism.es2.ES2Context.<init>(Lcom/sun/glass/ui/Screen;Lcom/sun/prism/ps/ShaderFactory;)V\
1.36 KB 13.4% 5 [J]com.sun.prism.es2.IOSGLFactory.createGLContext(Lcom/sun/prism/es2/GLDrawable;Lcom/sun/prism/es2/GLPixelFormat;Lcom/sun/prism/es2/GLContext;Z)Lcom/sun/prism/es2/GLContext;\
1.36 KB 13.4% 5 [J]com.sun.prism.es2.IOSGLContext.<init>(Lcom/sun/prism/es2/GLDrawable;Lcom/sun/prism/es2/GLPixelFormat;Lcom/sun/prism/es2/GLContext;Z)V\
1.36 KB 13.4% 5 [J]com.sun.prism.es2.IOSGLContext.nInitialize(JJJZ)J\
1.36 KB 13.4% 5 Java_com_sun_prism_es2_IOSGLContext_nInitialize\
1.08 KB 10.6% 4 strdup\
1.08 KB 10.6% 4 malloc\
288 Bytes 2.7% 1 malloc\
Related
Hi I do not know what it's happening but I run the command:
npm run watch
and it displays
/public/js/app.js 1.94 MiB /public/js/app [emitted] /public/js/app
0.js 40.4 KiB 0 [emitted]
1.js 690 KiB 1 [emitted]
10.js 47.6 KiB 10 [emitted]
100.js 31.2 KiB 100 [emitted]
101.js 26.6 KiB 101 [emitted]
102.js 15.4 KiB 102 [emitted]
103.js 15.8 KiB 103 [emitted]
104.js 30 KiB 104 [emitted]
105.js 23.6 KiB 105 [emitted]
106.js 17.8 KiB 106 [emitted]
107.js 19.3 KiB 107 [emitted]
108.js 64.6 KiB 108 [emitted]
109.js 21.4 KiB 109 [emitted]
11.js 44.7 KiB 11 [emitted]
110.js 7.95 KiB 110 [emitted]
111.js 21 KiB 111 [emitted]
12.js 44.5 KiB 12 [emitted]
13.js 32.7 KiB 13 [emitted]
14.js 44.7 KiB 14 [emitted]
15.js 47 KiB 15 [emitted]
16.js 29.9 KiB 16 [emitted]
17.js 57.1 KiB 17 [emitted]
18.js 37.8 KiB 18 [emitted]
19.js 40.6 KiB 19 [emitted]
2.js 29.1 KiB 2 [emitted]
20.js 53.6 KiB 20 [emitted]
21.js 30.6 KiB 21 [emitted]
22.js 46.9 KiB 22 [emitted]
23.js 44.5 KiB 23 [emitted]
24.js 30.5 KiB 24 [emitted]
25.js 42.7 KiB 25 [emitted]
26.js 28.6 KiB 26 [emitted]
27.js 37.7 KiB 27 [emitted]
28.js 41.1 KiB 28 [emitted]
29.js 74.1 KiB 29 [emitted]
3.js 45.6 KiB 3 [emitted]
30.js 39.5 KiB 30 [emitted]
31.js 57.2 KiB 31 [emitted]
32.js 28.6 KiB 32 [emitted]
33.js 31.7 KiB 33 [emitted]
34.js 30.7 KiB 34 [emitted]
35.js 39 KiB 35 [emitted]
36.js 28.2 KiB 36 [emitted]
37.js 62 KiB 37 [emitted]
38.js 30.1 KiB 38 [emitted]
39.js 47.8 KiB 39 [emitted]
4.js 72.7 KiB 4 [emitted]
40.js 36.3 KiB 40 [emitted]
41.js 41.7 KiB 41 [emitted]
42.js 35.7 KiB 42 [emitted]
43.js 40 KiB 43 [emitted]
44.js 62.2 KiB 44 [emitted]
45.js 29.2 KiB 45 [emitted]
46.js 45.5 KiB 46 [emitted]
47.js 35.5 KiB 47 [emitted]
48.js 30.9 KiB 48 [emitted]
49.js 45 KiB 49 [emitted]
5.js 61 KiB 5 [emitted]
50.js 35.2 KiB 50 [emitted]
51.js 37.9 KiB 51 [emitted]
52.js 30.3 KiB 52 [emitted]
53.js 57.1 KiB 53 [emitted]
54.js 39.1 KiB 54 [emitted]
55.js 41.9 KiB 55 [emitted]
56.js 23.2 KiB 56 [emitted]
57.js 29.9 KiB 57 [emitted]
58.js 24.9 KiB 58 [emitted]
59.js 25.2 KiB 59 [emitted]
6.js 65 KiB 6 [emitted]
60.js 39 KiB 60 [emitted]
61.js 41 KiB 61 [emitted]
62.js 35.4 KiB 62 [emitted]
63.js 49.5 KiB 63 [emitted]
64.js 72.9 KiB 64 [emitted]
65.js 57.1 KiB 65 [emitted]
66.js 60.7 KiB 66 [emitted]
67.js 39.4 KiB 67 [emitted]
68.js 29.9 KiB 68 [emitted]
69.js 72.7 KiB 69 [emitted]
7.js 62.4 KiB 7 [emitted]
70.js 53.4 KiB 70 [emitted]
71.js 50.9 KiB 71 [emitted]
72.js 79.2 KiB 72 [emitted]
73.js 38.8 KiB 73 [emitted]
74.js 56.6 KiB 74 [emitted]
75.js 50.6 KiB 75 [emitted]
76.js 68 KiB 76 [emitted]
77.js 34 KiB 77 [emitted]
78.js 45.9 KiB 78 [emitted]
79.js 37.9 KiB 79 [emitted]
8.js 77.1 KiB 8 [emitted]
80.js 32 KiB 80 [emitted]
81.js 40.3 KiB 81 [emitted]
82.js 56.6 KiB 82 [emitted]
83.js 50.3 KiB 83 [emitted]
84.js 45 KiB 84 [emitted]
85.js 48 KiB 85 [emitted]
86.js 42.6 KiB 86 [emitted]
87.js 62.4 KiB 87 [emitted]
88.js 29.4 KiB 88 [emitted]
89.js 29.3 KiB 89 [emitted]
9.js 73.3 KiB 9 [emitted]
90.js 49.5 KiB 90 [emitted]
91.js 20 KiB 91 [emitted]
92.js 8.3 KiB 92 [emitted]
93.js 8.09 KiB 93 [emitted]
94.js 35.9 KiB 94 [emitted]
95.js 30.6 KiB 95 [emitted]
96.js 28.7 KiB 96 [emitted]
97.js 40.2 KiB 97 [emitted]
98.js 17.6 KiB 98 [emitted]
99.js 14.3 KiB 99 [emitted]
public/css/app.css 193 KiB /public/js/app [emitted] /public/js/app
and it is OK but the thing is that I do not why when I compile it and it says Ok it does not compile in public/ because if I go to public_html and I check it displays this:
console
So it is out of the folder public/ and I wonder why is it happening?
My webpack.mix is like this one:
const mix = require('laravel-mix');
mix.js('resources/js/app.js', 'public/js')
.sass('resources/sass/app.scss', 'public/css');
so I wonder why is it out of public folder if it says that it will compile in public folder?
Thanks
When running hot or watch, no actual files are generated in the /public directory. Instead, a mini server is created to serve these files, normally on port 8080.
Until you run npm run production you will see the actual files on the folder.
The mix() helper on the view automatically handles this difference and serves either the files or the ones generated on port 8080
Read in the same hsv_2020_climo_data.csv file into Pandas DataFrame with the Date column as the index (similar to examples before).
Answer the following questions with formatted print statements.
What are the data types of each column? (i.e. 'MaxTemperature' dtype is int64)
What is the highest maximum temperature for the entire year?
What is the lowest minimum temperature for the entire year?
How much total rain did we get for 2020? (Hint you will need to handle the "Trace" values first)
Make a plot with Maximum Temperature, Minimum Temperature, and Average Temperature with labels and title.
Data below
Date MaxTemperature MinTemperature AvgTemperature Precipitation Snowfall SnowDepth
2020-01-01 53 31 42.0 0.00 0.0 0
2020-01-02 54 45 49.5 3.42 0.0 0
2020-01-03 59 53 56.0 0.32 0.0 0
2020-01-04 56 31 43.5 0.08 0.0 0
2020-01-05 55 29 42.0 0.00 0.0 0
2020-01-06 60 35 47.5 0.03 0.0 0
2020-01-07 55 35 45.0 T 0.0 0
2020-01-08 61 30 45.5 0.00 0.0 0
2020-01-09 60 36 48.0 0.00 0.0 0
2020-01-10 65 58 61.5 T 0.0 0
2020-01-11 72 50 61.0 1.08 0.0 0
2020-01-12 59 46 52.5 0.00 0.0 0
2020-01-13 62 43 52.5 0.08 0.0 0
2020-01-14 64 60 62.0 0.96 0.0 0
2020-01-15 67 60 63.5 0.61 0.0 0
2020-01-16 60 42 51.0 0.03 0.0 0
2020-01-17 59 40 49.5 0.00 0.0 0
2020-01-18 57 45 51.0 0.17 0.0 0
2020-01-19 45 27 36.0 0.00 0.0 0
2020-01-20 28 22 25.0 T T 0
2020-01-21 37 23 30.0 0.00 0.0 0
2020-01-22 46 20 33.0 0.00 0.0 0
2020-01-23 47 39 43.0 0.72 0.0 0
2020-01-24 54 40 47.0 0.19 0.0 0
2020-01-25 41 33 37.0 T 0.0 0
2020-01-26 49 29 39.0 0.03 0.0 0
2020-01-27 58 37 47.5 0.01 0.0 0
2020-01-28 60 31 45.5 0.00 0.0 0
2020-01-29 50 44 47.0 0.03 0.0 0
2020-01-30 60 37 48.5 0.00 0.0 0
2020-01-31 52 45 48.5 T 0.0 0
2020-02-01 50 37 43.5 0.06 0.0 0
2020-02-02 68 31 49.5 0.00 0.0 0
2020-02-03 71 40 55.5 T 0.0 0
2020-02-04 67 55 61.0 0.18 0.0 0
2020-02-05 68 62 65.0 1.16 0.0 0
2020-02-06 64 36 50.0 1.49 0.0 0
2020-02-07 41 33 37.0 T T 0
2020-02-08 53 32 42.5 0.10 T 0
2020-02-09 61 33 47.0 T 0.0 0
2020-02-10 57 49 53.0 1.78 0.0 0
2020-02-11 66 45 55.5 1.12 0.0 0
2020-02-12 70 44 57.0 1.31 0.0 0
2020-02-13 60 36 48.0 0.38 0.0 0
2020-02-14 41 26 33.5 0.00 0.0 0
2020-02-15 54 22 38.0 0.00 0.0 0
2020-02-16 58 42 50.0 0.00 0.0 0
2020-02-17 56 38 47.0 0.00 0.0 0
2020-02-18 62 46 54.0 1.32 0.0 0
2020-02-19 52 43 47.5 T 0.0 0
2020-02-20 47 33 40.0 0.85 T 0
2020-02-21 44 26 35.0 0.00 0.0 0
2020-02-22 56 24 40.0 0.00 0.0 0
2020-02-23 53 34 43.5 0.01 0.0 0
2020-02-24 55 44 49.5 0.62 0.0 0
2020-02-25 62 45 53.5 0.00 0.0 0
2020-02-26 48 36 42.0 0.04 T 0
2020-02-27 46 31 38.5 T T 0
2020-02-28 51 34 42.5 T 0.0 0
2020-02-29 55 36 45.5 0.00 0.0 0
2020-03-01 66 34 50.0 0.00 0.0 0
2020-03-02 60 52 56.0 0.44 0.0 0
2020-03-03 69 55 62.0 0.33 0.0 0
2020-03-04 60 51 55.5 0.04 0.0 0
2020-03-05 59 42 50.5 0.15 0.0 0
2020-03-06 56 37 46.5 0.00 0.0 0
2020-03-07 58 30 44.0 0.00 0.0 0
2020-03-08 65 35 50.0 0.00 0.0 0
2020-03-09 68 45 56.5 T 0.0 0
2020-03-10 70 56 63.0 0.27 0.0 0
2020-03-11 66 51 58.5 0.12 0.0 0
2020-03-12 76 57 66.5 0.26 0.0 0
2020-03-13 67 54 60.5 0.14 0.0 0
2020-03-14 74 50 62.0 0.56 0.0 0
2020-03-15 61 44 52.5 1.07 0.0 0
2020-03-16 59 44 51.5 0.02 0.0 0
2020-03-17 74 53 63.5 0.24 0.0 0
2020-03-18 77 52 64.5 0.00 0.0 0
2020-03-19 78 64 71.0 T 0.0 0
2020-03-20 72 60 66.0 1.11 0.0 0
2020-03-21 59 43 51.0 0.01 0.0 0
2020-03-22 66 43 54.5 0.07 0.0 0
2020-03-23 64 57 60.5 1.80 0.0 0
2020-03-24 76 57 66.5 2.96 0.0 0
2020-03-25 66 51 58.5 0.00 0.0 0
2020-03-26 81 47 64.0 0.00 0.0 0
2020-03-27 85 63 74.0 0.00 0.0 0
2020-03-28 82 66 74.0 0.00 0.0 0
2020-03-29 75 53 64.0 0.41 0.0 0
2020-03-30 70 52 61.0 0.02 0.0 0
2020-03-31 56 43 49.5 0.65 0.0 0
2020-04-01 62 39 50.5 0.00 0.0 0
2020-04-02 70 38 54.0 0.00 0.0 0
2020-04-03 75 42 58.5 0.00 0.0 0
2020-04-04 78 54 66.0 0.00 0.0 0
2020-04-05 81 54 67.5 0.00 0.0 0
2020-04-06 83 52 67.5 0.00 0.0 0
2020-04-07 74 62 68.0 0.00 0.0 0
2020-04-08 80 63 71.5 0.24 0.0 0
2020-04-09 71 57 64.0 0.32 0.0 0
2020-04-10 60 40 50.0 0.00 0.0 0
2020-04-11 71 37 54.0 0.00 0.0 0
2020-04-12 66 54 60.0 3.02 0.0 0
2020-04-13 66 42 54.0 T 0.0 0
2020-04-14 59 39 49.0 0.00 0.0 0
2020-04-15 61 34 47.5 0.00 0.0 0
2020-04-16 69 36 52.5 0.00 0.0 0
2020-04-17 76 45 60.5 0.07 0.0 0
2020-04-18 62 45 53.5 0.21 0.0 0
2020-04-19 63 46 54.5 1.41 0.0 0
2020-04-20 72 51 61.5 0.11 0.0 0
2020-04-21 76 50 63.0 0.00 0.0 0
2020-04-22 68 42 55.0 0.29 0.0 0
2020-04-23 70 54 62.0 0.92 0.0 0
2020-04-24 73 56 64.5 0.01 0.0 0
2020-04-25 74 53 63.5 0.21 0.0 0
2020-04-26 61 41 51.0 0.00 0.0 0
2020-04-27 72 38 55.0 0.00 0.0 0
2020-04-28 77 53 65.0 T 0.0 0
2020-04-29 72 53 62.5 0.13 0.0 0
2020-04-30 71 48 59.5 T 0.0 0
2020-05-01 76 43 59.5 0.00 0.0 0
2020-05-02 83 51 67.0 0.00 0.0 0
2020-05-03 85 58 71.5 0.00 0.0 0
2020-05-04 84 60 72.0 T 0.0 0
2020-05-05 76 56 66.0 T 0.0 0
2020-05-06 65 44 54.5 0.00 0.0 0
2020-05-07 71 39 55.0 0.00 0.0 0
2020-05-08 61 48 54.5 0.86 0.0 0
2020-05-09 64 40 52.0 0.00 0.0 0
2020-05-10 73 39 56.0 0.00 0.0 0
2020-05-11 68 43 55.5 0.00 0.0 0
2020-05-12 66 48 57.0 T 0.0 0
2020-05-13 79 55 67.0 0.05 0.0 0
2020-05-14 85 61 73.0 0.00 0.0 0
2020-05-15 84 67 75.5 0.00 0.0 0
2020-05-16 86 62 74.0 0.00 0.0 0
2020-05-17 79 65 72.0 0.23 0.0 0
2020-05-18 82 60 71.0 T 0.0 0
2020-05-19 71 54 62.5 T 0.0 0
2020-05-20 77 54 65.5 0.25 0.0 0
2020-05-21 79 57 68.0 0.00 0.0 0
2020-05-22 82 63 72.5 1.52 0.0 0
2020-05-23 86 64 75.0 0.28 0.0 0
2020-05-24 88 65 76.5 T 0.0 0
2020-05-25 88 67 77.5 0.00 0.0 0
2020-05-26 76 67 71.5 0.31 0.0 0
2020-05-27 79 61 70.0 0.74 0.0 0
2020-05-28 83 62 72.5 0.21 0.0 0
2020-05-29 83 64 73.5 0.18 0.0 0
2020-05-30 84 63 73.5 0.00 0.0 0
2020-05-31 83 59 71.0 0.00 0.0 0
2020-06-01 85 53 69.0 0.00 0.0 0
2020-06-02 89 67 78.0 0.00 0.0 0
2020-06-03 88 71 79.5 0.06 0.0 0
2020-06-04 87 68 77.5 T 0.0 0
2020-06-05 90 69 79.5 0.41 0.0 0
2020-06-06 91 68 79.5 0.00 0.0 0
2020-06-07 91 71 81.0 0.00 0.0 0
2020-06-08 84 75 79.5 0.43 0.0 0
2020-06-09 87 75 81.0 0.11 0.0 0
2020-06-10 92 65 78.5 0.00 0.0 0
2020-06-11 85 61 73.0 0.00 0.0 0
2020-06-12 88 61 74.5 0.00 0.0 0
2020-06-13 90 58 74.0 0.00 0.0 0
2020-06-14 92 62 77.0 0.00 0.0 0
2020-06-15 83 61 72.0 0.00 0.0 0
2020-06-16 81 60 70.5 0.00 0.0 0
2020-06-17 80 63 71.5 0.00 0.0 0
2020-06-18 85 61 73.0 0.00 0.0 0
2020-06-19 91 64 77.5 0.00 0.0 0
2020-06-20 94 66 80.0 0.00 0.0 0
2020-06-21 88 69 78.5 0.14 0.0 0
2020-06-22 88 68 78.0 0.14 0.0 0
2020-06-23 87 70 78.5 0.25 0.0 0
2020-06-24 75 68 71.5 0.70 0.0 0
2020-06-25 85 70 77.5 0.00 0.0 0
2020-06-26 75 68 71.5 0.15 0.0 0
2020-06-27 82 68 75.0 0.30 0.0 0
2020-06-28 90 73 81.5 0.25 0.0 0
2020-06-29 90 71 80.5 0.01 0.0 0
2020-06-30 88 70 79.0 0.84 0.0 0
2020-07-01 82 68 75.0 0.73 0.0 0
2020-07-02 89 68 78.5 0.00 0.0 0
2020-07-03 94 71 82.5 0.00 0.0 0
2020-07-04 92 71 81.5 0.00 0.0 0
2020-07-05 93 71 82.0 0.47 0.0 0
2020-07-06 88 71 79.5 0.00 0.0 0
2020-07-07 90 73 81.5 0.07 0.0 0
2020-07-08 87 73 80.0 T 0.0 0
2020-07-09 90 71 80.5 0.00 0.0 0
2020-07-10 92 73 82.5 0.00 0.0 0
2020-07-11 92 67 79.5 0.00 0.0 0
2020-07-12 82 68 75.0 1.38 0.0 0
2020-07-13 89 69 79.0 0.00 0.0 0
2020-07-14 91 70 80.5 0.00 0.0 0
2020-07-15 93 70 81.5 0.00 0.0 0
2020-07-16 91 70 80.5 0.00 0.0 0
2020-07-17 94 73 83.5 0.00 0.0 0
2020-07-18 95 73 84.0 0.00 0.0 0
2020-07-19 95 73 84.0 0.00 0.0 0
2020-07-20 95 73 84.0 0.00 0.0 0
2020-07-21 94 74 84.0 T 0.0 0
2020-07-22 92 73 82.5 0.19 0.0 0
2020-07-23 92 71 81.5 0.00 0.0 0
2020-07-24 90 73 81.5 0.00 0.0 0
2020-07-25 94 72 83.0 0.07 0.0 0
2020-07-26 94 71 82.5 0.00 0.0 0
2020-07-27 91 73 82.0 T 0.0 0
2020-07-28 90 72 81.0 T 0.0 0
2020-07-29 92 73 82.5 0.02 0.0 0
2020-07-30 90 74 82.0 0.14 0.0 0
2020-07-31 92 74 83.0 0.25 0.0 0
2020-08-01 87 70 78.5 T 0.0 0
2020-08-02 86 66 76.0 0.00 0.0 0
2020-08-03 91 67 79.0 0.00 0.0 0
2020-08-04 90 70 80.0 0.01 0.0 0
2020-08-05 92 68 80.0 0.00 0.0 0
2020-08-06 92 71 81.5 0.00 0.0 0
2020-08-07 94 69 81.5 0.00 0.0 0
2020-08-08 97 68 82.5 0.00 0.0 0
2020-08-09 96 71 83.5 0.00 0.0 0
2020-08-10 98 74 86.0 0.00 0.0 0
2020-08-11 95 73 84.0 0.49 0.0 0
2020-08-12 93 74 83.5 0.01 0.0 0
2020-08-13 94 71 82.5 T 0.0 0
2020-08-14 90 74 82.0 T 0.0 0
2020-08-15 92 71 81.5 0.00 0.0 0
2020-08-16 93 67 80.0 T 0.0 0
2020-08-17 91 67 79.0 0.00 0.0 0
2020-08-18 93 64 78.5 0.24 0.0 0
2020-08-19 91 68 79.5 1.24 0.0 0
2020-08-20 87 67 77.0 T 0.0 0
2020-08-21 82 68 75.0 0.10 0.0 0
2020-08-22 85 64 74.5 0.00 0.0 0
2020-08-23 88 68 78.0 0.00 0.0 0
2020-08-24 88 72 80.0 T 0.0 0
2020-08-25 82 72 77.0 0.15 0.0 0
2020-08-26 85 70 77.5 1.83 0.0 0
2020-08-27 91 75 83.0 0.22 0.0 0
2020-08-28 86 72 79.0 0.92 0.0 0
2020-08-29 90 74 82.0 0.02 0.0 0
2020-08-30 91 71 81.0 0.23 0.0 0
2020-08-31 87 71 79.0 0.94 0.0 0
2020-09-01 89 71 80.0 0.05 0.0 0
2020-09-02 89 74 81.5 0.00 0.0 0
2020-09-03 89 73 81.0 0.00 0.0 0
2020-09-04 90 67 78.5 T 0.0 0
2020-09-05 88 59 73.5 0.00 0.0 0
2020-09-06 86 57 71.5 0.00 0.0 0
2020-09-07 86 60 73.0 0.00 0.0 0
2020-09-08 87 64 75.5 0.00 0.0 0
2020-09-09 88 65 76.5 0.00 0.0 0
2020-09-10 90 66 78.0 0.00 0.0 0
2020-09-11 93 68 80.5 0.00 0.0 0
2020-09-12 90 73 81.5 0.01 0.0 0
2020-09-13 91 71 81.0 T 0.0 0
2020-09-14 90 69 79.5 0.00 0.0 0
2020-09-15 83 69 76.0 0.00 0.0 0
2020-09-16 74 68 71.0 0.12 0.0 0
2020-09-17 87 70 78.5 0.00 0.0 0
2020-09-18 79 61 70.0 0.00 0.0 0
2020-09-19 76 59 67.5 0.00 0.0 0
2020-09-20 81 58 69.5 0.00 0.0 0
2020-09-21 76 53 64.5 0.00 0.0 0
2020-09-22 75 51 63.0 T 0.0 0
2020-09-23 71 53 62.0 0.79 0.0 0
2020-09-24 66 55 60.5 2.65 0.0 0
2020-09-25 73 64 68.5 T 0.0 0
2020-09-26 76 62 69.0 0.00 0.0 0
2020-09-27 83 61 72.0 0.00 0.0 0
2020-09-28 82 56 69.0 0.42 0.0 0
2020-09-29 70 49 59.5 0.00 0.0 0
2020-09-30 77 47 62.0 0.00 0.0 0
2020-10-01 76 51 63.5 0.00 0.0 0
2020-10-02 69 44 56.5 0.00 0.0 0
2020-10-03 71 40 55.5 0.00 0.0 0
2020-10-04 76 50 63.0 0.00 0.0 0
2020-10-05 76 48 62.0 0.00 0.0 0
2020-10-06 80 48 64.0 0.00 0.0 0
2020-10-07 82 52 67.0 0.00 0.0 0
2020-10-08 82 49 65.5 0.00 0.0 0
2020-10-09 73 63 68.0 0.47 0.0 0
2020-10-10 74 64 69.0 1.35 0.0 0
2020-10-11 75 68 71.5 0.23 0.0 0
2020-10-12 80 64 72.0 0.00 0.0 0
2020-10-13 76 52 64.0 0.00 0.0 0
2020-10-14 82 45 63.5 0.00 0.0 0
2020-10-15 80 54 67.0 0.00 0.0 0
2020-10-16 66 39 52.5 0.01 0.0 0
2020-10-17 68 37 52.5 0.00 0.0 0
2020-10-18 76 50 63.0 0.00 0.0 0
2020-10-19 80 56 68.0 0.00 0.0 0
2020-10-20 81 59 70.0 0.00 0.0 0
2020-10-21 81 58 69.5 0.00 0.0 0
2020-10-22 83 62 72.5 0.00 0.0 0
2020-10-23 83 63 73.0 0.03 0.0 0
2020-10-24 66 55 60.5 0.44 0.0 0
2020-10-25 69 55 62.0 0.00 0.0 0
2020-10-26 75 58 66.5 0.00 0.0 0
2020-10-27 75 58 66.5 T 0.0 0
2020-10-28 74 69 71.5 2.87 0.0 0
2020-10-29 72 48 60.0 0.58 0.0 0
2020-10-30 57 42 49.5 0.00 0.0 0
2020-10-31 68 40 54.0 0.00 0.0 0
2020-11-01 68 43 55.5 0.00 0.0 0
2020-11-02 57 33 45.0 0.00 0.0 0
2020-11-03 66 34 50.0 0.00 0.0 0
2020-11-04 71 39 55.0 0.00 0.0 0
2020-11-05 70 44 57.0 0.00 0.0 0
2020-11-06 76 46 61.0 0.00 0.0 0
2020-11-07 75 48 61.5 T 0.0 0
2020-11-08 79 59 69.0 0.00 0.0 0
2020-11-09 78 62 70.0 0.00 0.0 0
2020-11-10 74 65 69.5 T 0.0 0
2020-11-11 77 58 67.5 0.04 0.0 0
2020-11-12 68 44 56.0 0.00 0.0 0
2020-11-13 71 42 56.5 0.00 0.0 0
2020-11-14 73 41 57.0 0.00 0.0 0
2020-11-15 69 40 54.5 0.02 0.0 0
2020-11-16 63 32 47.5 0.00 0.0 0
2020-11-17 62 34 48.0 0.00 0.0 0
2020-11-18 64 31 47.5 0.00 0.0 0
2020-11-19 66 38 52.0 0.00 0.0 0
2020-11-20 72 40 56.0 0.00 0.0 0
2020-11-21 73 42 57.5 0.00 0.0 0
2020-11-22 69 46 57.5 0.02 0.0 0
2020-11-23 57 35 46.0 0.00 0.0 0
2020-11-24 65 32 48.5 0.00 0.0 0
2020-11-25 65 53 59.0 0.48 0.0 0
2020-11-26 62 40 51.0 0.00 0.0 0
2020-11-27 66 38 52.0 0.58 0.0 0
2020-11-28 57 41 49.0 T 0.0 0
2020-11-29 55 39 47.0 0.73 0.0 0
2020-11-30 44 30 37.0 0.08 T 0
2020-12-01 41 25 33.0 0.00 0.0 0
2020-12-02 52 20 36.0 0.00 0.0 0
2020-12-03 58 25 41.5 0.16 0.0 0
2020-12-04 48 35 41.5 0.82 0.0 0
2020-12-05 56 28 42.0 0.00 0.0 0
2020-12-06 59 30 44.5 T 0.0 0
2020-12-07 47 28 37.5 0.00 0.0 0
2020-12-08 49 25 37.0 0.00 0.0 0
2020-12-09 64 28 46.0 0.00 0.0 0
2020-12-10 71 35 53.0 0.00 0.0 0
2020-12-11 66 37 51.5 0.00 0.0 0
2020-12-12 63 46 54.5 0.30 0.0 0
2020-12-13 60 34 47.0 0.80 0.0 0
2020-12-14 44 35 39.5 0.81 0.0 0
2020-12-15 48 30 39.0 T 0.0 0
2020-12-16 50 35 42.5 0.26 0.0 0
2020-12-17 43 26 34.5 0.00 0.0 0
2020-12-18 50 23 36.5 0.00 0.0 0
2020-12-19 53 27 40.0 0.03 0.0 0
2020-12-20 51 40 45.5 0.16 0.0 0
2020-12-21 61 38 49.5 0.00 0.0 0
2020-12-22 60 31 45.5 0.00 0.0 0
2020-12-23 61 35 48.0 0.12 0.0 0
2020-12-24 52 28 40.0 1.13 T 0
2020-12-25 32 20 26.0 0.00 0.0 0
2020-12-26 50 18 34.0 0.00 0.0 0
2020-12-27 60 26 43.0 0.00 0.0 0
2020-12-28 57 37 47.0 0.00 0.0 0
2020-12-29 61 33 47.0 0.00 0.0 0
2020-12-30 69 41 55.0 0.00 0.0 0
2020-12-31 59 50 54.5 0.03 0.0 0
First of all, notice that some columns contain the value "T". To solve some of the questions, you'll have to replace those:
df = pd.read_csv(r"C:\users\....\DATA.csv", sep=";")
df.replace('T',0, inplace = True)
To get the datatypes:
df.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 366 entries, 0 to 365
Data columns (total 7 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 Date 366 non-null object
1 MaxTemperature 366 non-null int64
2 MinTemperature 366 non-null int64
3 AvgTemperature 366 non-null float64
4 Precipitation 366 non-null object
5 Snowfall 366 non-null object
6 SnowDepth 366 non-null int64
dtypes: float64(1), int64(3), object(3)
memory usage: 20.1+ KB
To get all the information you ask, you need to transform object values to float. They are string because you hade "T" values instead of numeric:
df['Precipitation'] = df.Precipitation.astype(float)
df['Snowfall'] = df.Snowfall.astype(float)
df['SnowDepth'] = df.SnowDepth.astype(float)
Note now that
df.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 366 entries, 0 to 365
Data columns (total 7 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 Date 366 non-null object
1 MaxTemperature 366 non-null int64
2 MinTemperature 366 non-null int64
3 AvgTemperature 366 non-null float64
4 Precipitation 366 non-null float64
5 Snowfall 366 non-null float64
6 SnowDepth 366 non-null float64
dtypes: float64(4), int64(2), object(1)
memory usage: 20.1+ KB
Now to answer all the questions:
df.describe()
returns:
MaxTemperature MinTemperature AvgTemperature Precipitation \
count 366.000000 366.000000 366.000000 366.000000
mean 73.021858 52.229508 62.625683 0.192678
std 14.496086 14.966696 14.317651 0.471406
min 28.000000 18.000000 25.000000 0.000000
25% 61.250000 40.000000 51.000000 0.000000
50% 74.000000 53.000000 63.250000 0.000000
75% 86.000000 67.000000 75.500000 0.140000
max 98.000000 75.000000 86.000000 3.420000
Snowfall SnowDepth
count 366.0 366.0
mean 0.0 0.0
std 0.0 0.0
min 0.0 0.0
25% 0.0 0.0
50% 0.0 0.0
75% 0.0 0.0
max 0.0 0.0
you have the max, min, .... for all variables.
As for the plot
lines = df.plot.line()
I have a huge dataset with ID, MEAN, and DATE (in day of the year). I'm sub-setting it below just for example:
OBJECTID MEAN DATE
1 0.960337524 27
2 1.024530873 27
3 1.07565201 27
4 1.32608937 27
5 1.115863256 27
6 0.738648832 27
7 1.209547088 27
8 1.190287749 27
1 1.311272704 43
2 1.421150386 43
3 1.341622942 43
4 1.343600738 43
5 1.322288454 43
6 1.057037145 43
7 1.262514248 43
8 1.148541133 43
1 1.141311572 75
2 1.12654984 75
3 1.125632558 75
4 1.128487158 75
5 1.181200445 75
6 0.820567181 75
7 0.973662794 75
8 0.903646102 75
In this example, the first date is DAY 27 (Jan 27th). I want to reformat this in Excel so DATE is the horizontal header and ID is the first vertical column like this:
OBJECTID 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365
1
2
3
4
5
6
7
8
How do I use INDEX and MATCH to populate the cells in the blank table above with the values in the dataset? Not all dates in the table will have a value so I need it to populate it with zero like this:
OBJECTID 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.960337524 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1.311272704 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1.141311572 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
2
3
4
5
6
7
8
This is what I came up with so far:
=IFERROR(INDEX($B$2:$B$2617, MATCH(0, COUNTIF($E2:E2,$B$2:$B$2617)+IF($A$2:$A$2617<>$E2, 1, 0), 0)), 0)
But it doesn't account for the dates with no values. It put the MEAN value for Day 27 into the cell for Day 1.
OBJECTID MEAN DATE OBJECTID 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365
1 0.960337524 27 1 0.960337524
2 1.024530873 27 2
3 1.07565201 27 3
4 1.32608937 27 4
5 1.115863256 27 5
6 0.738648832 27 6
7 1.209547088 27 7
8 1.190287749 27 8
1 1.311272704 43
2 1.421150386 43
3 1.341622942 43
4 1.343600738 43
5 1.322288454 43
6 1.057037145 43
7 1.262514248 43
8 1.148541133 43
1 1.141311572 75
2 1.12654984 75
3 1.125632558 75
4 1.128487158 75
5 1.181200445 75
6 0.820567181 75
7 0.973662794 75
8 0.903646102 75
Any advice or push in the right direction would be appreciated!
On condition that there is only one Mean in your data for each combination of ID and Date the formula below will do the job. (Input is the sheet with your data in it.)
=SUMIFS(Input!$B$2:$B$2617,Input!$A$2:$A$2617,$A2,Input!$C$2:$C$2617,B$1)
However, this formula makes the range management too error prone. Luckily most errors in the range setting will result in a formula error but the process can be much simplified by separating range management from data extraction.
I assigned the name Data to the range Input!$A$2:$C$2617. In fact, you would probably construct this range to be dynamic so that it adjusts to the number of data rows you have automatically. But that is an extra benefit of properly managing ranges which is outside the scope of your present question.
Use the range Data in the above formula you arrive at this:-
=SUMIFS(INDEX(Data,,2),INDEX(Data,,1),$A2,INDEX(Data,,3),B$1)
Either this formula or the one first above introduced can be copied to the right and down to cover your entire output table. If they bother you (as they did me) you can suppress the display of zeros in the sheet or in each cell, using the sheet settings or the cell format to do so.
What you can do is create another unique ID using simple "&" (concat) function. Basically add a 4th column which has the formula
=A2&"_"&C2
The above takes the assumption that your first entry is in 2nd row and this formula is for 2nd row as well which you can drag. Once done then you can apply index match on this new ID.
Hope it helps in clearing your query.
I have an express API and I want to show the api memory usage, to test the solution, I create a global variable in the module and for each get request I PUSH a data to the array to see if memory increase or no
let array = [];
const randomData = `150 216 86 129 25 116 72 20 155 148 91 30 187 152 136 204 125 182 250
217 97 45 161 48 223 1 55 163 236 240 179 118 234 175 71 56 44 221 245 59 145 66 173 1
12 197 88 146 140 111 223 59 169 53 68 123 0 252 68 96 215 132 236 245 128 43 98 16 110
22 5 179 12 177 87 162 5 134 64 226 250 70 238 114 215 135 147 85 218 140 194 244 107 62
87 19 169 227 2 97 184 215 164 30 34 229 35 46 71 127 120 75 243 68 143 40 203 132 211 212
253 136 159 24 149 241 58 167 199 247 140 229 151 85 65 25 44 106 199 216 110 61 72 135
103 251 122 103 128 134 9 25 38 61 136 159 158 225 137 44 246 176 217 202 149 73 111 73 22
41 151 217 59 69 10 143 85 181 10 194 64 23 244 243 179 240 150 25 111 162 60 221 197 36 191
138 217 185 1 127 226 152 75 7 250 72 147 242 184 70 158 211 154 225 165 130 57 24 50 97 192
125 88 131 183 171 67 209 251 151 64 11 59 206 75 143 17 34 203 95 97 57 209 212 112 99 180
136 142 230 163 82 172 232 134 135 50 101 144 75 94 145 236 206 182 124 120 95 225 144 31 79
75 27 214 115 37 25 122 6 106 26 66 145 135 73 22 53 13 57 202 129 61 42 207 138 143 170 241
`;
exports.getAPIHealth = function (req, res) {
console.log(process.memoryUsage())
const memoryUsedMB = process.memoryUsage().heapUsed / (1024 * 1024);
console.log(memoryUsedMB);
array.push(randomData.toString());
res.status(200).json({
live: true,
memoryUsage: memoryUsedMB.toFixed(2) + ' MB'
})
};
In fact when I run this example I got strange values:
{ rss: 57921536,
heapTotal: 57929728,
heapUsed: 18556128,
external: 655899 }
17.73406219482422
{ rss: 58200064,
heapTotal: 57929728,
heapUsed: 18717496,
external: 655899 }
17.854393005371094
{ rss: 58200064,
heapTotal: 57929728,
heapUsed: 18760720,
external: 655899 }
17.894927978515625
{ rss: 58200064,
heapTotal: 57929728,
heapUsed: 18797824,
external: 655899 }
17.930313110351562
{ rss: 58220544,
heapTotal: 57929728,
heapUsed: 18834928,
external: 655899 }
17.965728759765625
{ rss: 41418752,
heapTotal: 25423872,
heapUsed: 18512640,
external: 153782 }
17.659194946289062
memory increase for each request (and this is normal) but at certain point it decrease,
can any one tell me why I have this behavior ?
NB: I notice that memory usage never reach 18MB (even after many requests)
I need to average three columns of data that are in a row in a text file.
Data:
VALID_TIME STN CDF05 CDF10 CDF20 CDF30 CDF40 CDF50 CDF60 CDF70 CDF80 CDF90 CDF95 MEAN SD
2015031018 KMGM 50.3 51.5 52.9 54.0 54.9 55.8 56.7 57.6 58.6 60.1 61.3 55.8 3.3
2015031106 KMGM 75.7 76.8 78.2 79.2 80.0 80.8 81.6 82.4 83.4 84.8 85.9 81.0 4.0
2015031118 KMGM 54.0 55.1 56.5 57.5 58.4 59.3 60.1 61.0 62.1 63.6 64.8 59.3 3.9
2015031206 KMGM 71.1 72.3 73.9 75.1 76.1 77.0 77.9 78.9 80.1 81.6 82.9 77.0 4.4
2015031218 KMGM 55.5 56.8 58.4 59.5 60.5 61.5 62.4 63.5 64.7 66.3 67.7 61.5 4.4
The columns that I'm interested in averaging together are CDF80, CDF90, and CDF95. The end format should be, for example:
VALID_TIME STN CDF05 CDF10 CDF20 CDF30 CDF40 CDF50 CDF60 CDF70 CDF80 CDF90 CDF95 MEAN NEWAVG
2015031018 KMGM 50.3 51.5 52.9 54.0 54.9 55.8 56.7 57.6 58.6 60.1 61.3 55.8 xx.x
2015031106 KMGM 75.7 76.8 78.2 79.2 80.0 80.8 81.6 82.4 83.4 84.8 85.9 81.0 xx.x
$ cat test.txt | awk -v OFS='\t' 'NR==1{$16="NEWAVG"}; NR!=1{$16=($11+$12+$13)/3};{print $0}'
VALID_TIME STN CDF05 CDF10 CDF20 CDF30 CDF40 CDF50 CDF60 CDF70 CDF80 CDF90 CDF95 MEAN SD NEWAVG
2015031018 KMGM 50.3 51.5 52.9 54.0 54.9 55.8 56.7 57.6 58.6 60.1 61.3 55.8 3.3 60
2015031106 KMGM 75.7 76.8 78.2 79.2 80.0 80.8 81.6 82.4 83.4 84.8 85.9 81.0 4.0 84.7
2015031118 KMGM 54.0 55.1 56.5 57.5 58.4 59.3 60.1 61.0 62.1 63.6 64.8 59.3 3.9 63.5
2015031206 KMGM 71.1 72.3 73.9 75.1 76.1 77.0 77.9 78.9 80.1 81.6 82.9 77.0 4.4 81.5333
2015031218 KMGM 55.5 56.8 58.4 59.5 60.5 61.5 62.4 63.5 64.7 66.3 67.7 61.5 4.4 66.2333
Averaging the columns is trivial, and you can retain most of the original formatting using printf:
$ awk 'NR==1 { printf "%s %8s\n", $0,"NEWAVG"} NR>1 { printf "%s %7.2f\n", $0,($11+$12+$13)/3 }' /tmp/data
VALID_TIME STN CDF05 CDF10 CDF20 CDF30 CDF40 CDF50 CDF60 CDF70 CDF80 CDF90 CDF95 MEAN SD NEWAVG
2015031018 KMGM 50.3 51.5 52.9 54.0 54.9 55.8 56.7 57.6 58.6 60.1 61.3 55.8 3.3 60.00
2015031106 KMGM 75.7 76.8 78.2 79.2 80.0 80.8 81.6 82.4 83.4 84.8 85.9 81.0 4.0 84.70
2015031118 KMGM 54.0 55.1 56.5 57.5 58.4 59.3 60.1 61.0 62.1 63.6 64.8 59.3 3.9 63.50
2015031206 KMGM 71.1 72.3 73.9 75.1 76.1 77.0 77.9 78.9 80.1 81.6 82.9 77.0 4.4 81.53
2015031218 KMGM 55.5 56.8 58.4 59.5 60.5 61.5 62.4 63.5 64.7 66.3 67.7 61.5 4.4 66.23