EDIT1: download file with 2 days of real data
My home automation controller collects data from several 4-in-1 motion sensors in different rooms of my house. The sensor prioritizes motion, sending motion reports every few seconds, but also independently reports temperature, humidity, and illuminance. I am trying to determine if the temp and humidity reports are sent frequently enough to automate control of heaters and exhaust fans.
Sensors independently report each category to the controller, which sends data to excel. Sample data below, but without motion reports that clutter up the real data.
A pivot table generated from the raw data:
Answering the question of frequency takes me several manual steps. Sorting/filtering the dataset for temp/humidity by room, then manually adding a time diff column
where time diff = (<current Date-Time cell> - <prev Date-Time cell>)*24*60. I then calculate the average and stdev of minutes between reports by manually selecting, in turn, each room/category subset in the time diff column; once for the average and once for the stdev.
After a few more manual steps, I end up with this desired result:
BUT I have to do it all over every time new data is added to the table. I'm certain excel can do this automatically, but I didn't find a solution through pivots, power pivots, slicing, or queries. I'm hoping one of you excel gurus can help. Thanks!
Related
I am looking to create an excel table so i can review my colleagues performance on a certain area in work.
The two measurement areas are Resolved Problems and Total time spent. I can pull this info from a dashboard at the end of the week and manually input it to excel.
We have 6 members of the team and i want to be able to input this data as weekly totals per user.
I will then use this data to track trends like who is resolving most problems, who is doing the least. Who spends the most time on problems and who spends the least.
I am not sure the best way to go about this data capture in excel.
Any help is appreciated.
Isn't this something that you would right click, select pivot table, and pivot my person, value column of time, sort by time?
I want to do some data analysis using Excel and this tasks I need to answer.
*data clustering
*margine
*seasonality/forecast
Can somebody give me an input maybe?
I am not coming from financial/statistical/analytical field, and these things are more less new to time, but I would like to learn how to do this in Excel.
Here is my data set, where time axis is non consecutive (for forecast prediction for instance).
I was trying to plot some reports for Covid-19 cases around the Globe, using Excel and Power BI. With Power BI is easier and fancier to do definitely, but I need an Excel file or calculation that makes sense - similar to the PBI. What I actually wanted is to calculate the daily increase in new cases (with %) and also death rate but per day, or total death by day and so on..
I did some calculations (% of column total and I calculated one field to get death rate%) here using Pivot tables but not sure how to do daily increase/decrease? Did anyone get an idea for additional calculations?
This is copied from PBI (calculations) which I wanna have similar in Excel - but I am not sure If I can calculate it properly (last 2 pictures).
The data source from the input data is here:
https://www.ecdc.europa.eu/sites/default/files/documents/COVID-19-geographic-disbtribution-worldwide.xlsx
You need an extra column for the result you want (e.g. daily increase/decrease), then you can plot either the waterfall chart, or using techniques similar to
https://www.extendoffice.com/documents/excel/5945-excel-chart-display-percentage-change.html
I have been downloading and organizing hourly water quality data into Excel for many different states, and have organized them by year. I have done data prep for them to make sure there are no zeros/every day of the year (DOY) has 24 values, but the time series plots were too noisy so want me to get the daily average values instead.
All of the sites annual data is different in terms of how many days are available, and sometimes they are missing whole months due to no recordings.
So my question is, how can I develop a code to give me the average daily value linked to a specific DOY that I can apply to many different Excel sheets. The data appears like this:
And the files are saved like like CA1_2012 (California Site 1 hourly data from 2012)
I know there is a lot on this topic but I have been trying everything and I can't get a code that works!
You can get the summation of the second column based on values in the first column in matlab using accumarray;
[m,~,n] = unique(data(:,1));
sumdata = [m, accumarray(n,data(:,2))];
for mean I would suggest grpstats:
avgdata = grpstats(data, DOY, {'mean'});
or as #gnovice suggested:
avgdata = accumarray(DOY, data, [], #mean);
You can also get what you want by using Pivot Table in excel and group data by DOY and get the mean value for them in the table. (No coding required).
I have a database with time series data of different solar power plants: how strong was the sun and how much power that plant created / harvested. This data is in 15 min increments.
I would like to use data mining to get new insights and to then visualize the findings to the users.
I know this falls into the domain of data mining, but my problem is maybe more specific (dealing with time series data). So what can I extract from this kind of data or where can I read about this?
Time Series Analysis is a whole field in itself. That said, you can always start with a few basics and keep adding more to your analysis.
Here are a few things to try for starters from your solar power data:
First, profile your solar power data. That is, calculate Min, Max, daily averages, hourly peaks and lows etc. to get a feel for the data. Plotting with x-axis as time will give you visual information.
Time Series data can be decomposed into "Trend" & "Seasonality" (can be for any repeating time interval)
Look for outliers, abnormalities in your data stream. Missing values, repeats etc.
If you want to learn more about time-series, (and if know R) then the forecast package is a good way to get started. (Especially this free e-book)
Any search on Time Series will take you to Prof. Hyndman's pages, and I have found the free chapters of his forecasting book very useful.
Hope that helps you get started.