I'm new to python and matplotlib. I need to plot a live graph from a CSV file which is being updated in real time. This is what I'm trying to do:
Just keep plotting as soon as a new value is updated into the file. It's procedural, as soon as I write the new data into the file, I read it again and plot. The number of readings may go higher than 1000. Maybe even 10000. (1000 or 10000 lines in CSV file, each line contains a unique x value and a corresponding y value). I'm plotting in a tkinter canvas. I need to plot the latest 50 values in the file, but also keep the previous values so that I can stop the graph, drag and see previous values. Plotting is fine, I understand how to get it done. But how much Ram/time and other resources does this process take. How does it affect the performance of the application and is there a better way to accomplish this? Note that after a while, I'll have an array with maybe 10000 values in it. Then I'll have to plot it.
Related
My problem is as follows:
The user inputs two numbers between 2 and 25, these numbers are used to create a grid. Every point on the grid has (x,y) coordinates. Based on the amount of points the user chose, my excel sheet is filled up with up to 25x25 (x,y) coordinates.
Example: A 6x7 grid is chosen by the user, the table is filled with 42 (x,y) coordinates and all other values in the table are set to "".
Now I want to use a scatterplot with lines connecting each array to plot the data.
Problem 1: If I only select the 6x7 part of the table that has values in it and create the scatterplot the result is correct. Until the user specifies a different grid, for example 8x9, then the graph is obviously missing two rows and two columns of input data.
Problem 2: If I select the entire 25x25 part of the table, including all the "" values, the graph axes get messed up. The y-axis works properly, but the x-axis shows sequential values (0-7) instead of the x-coordinates.
Problem 3: If I replace all the "" values in the table to 0 or NaN and plot the entire table the axes are correct, but the lines between the scatter data get messed up.
Question:
Is there a way to automatically change the input data for the plot, or is there a way to correctly display the values on the x-axis if I select all the data?
Not sure this will work in your case, but it's worth a try, especially since no one's addressed your post in 3+ hours. I've had success with this approach: 1) charting the largest data set, 2) copying the resulting chart, and 3) trimming the data it draws from to produce all smaller data sets.
To get this to work takes a lot of thought in laying out that largest data set so that all the other plots follow as needed. To illustrate, I've somewhat mimicked your data and in the animated gif I show largest data set, plus 2 others produced by copying it. Then I demonstrate how to make the second one, including the rescaling required to make all plots scaled equally. Notice that I've arranged things so that only one set of x-values feeds all the series. If you can do this, it makes working with the Excel's interface much easier.
After wrestling with it all night I came to the following solution:
Instead of setting all the empty cells to "" or zero the cells should be be set to #N/A (not available). The graph properly ignores the #N/A cells exactly like I want it to and updates when values are entered into them.
I have managed to plot two different data sets on the same axis however, I'm also looking to plotting another line showing their average.
The main problem is that both data sets have different X (time) values so it's not possible to add an average column at the end and plot that. (See the highlighted row 22 for example, corresponding Time values are different)
Is there any way I can plot an average of two plots on the same axis?
One idea that might work is to place the values of both series, one above the other in two new columns, sort this new data according to time, smooth it, then plot the smoothed combined data. Alternatively, you could do the smoothing by simply plotting the new sorted series, adding a moving average trendline to it, then change the formatting of the new series so that it is no longer visible (but the trendline is). Something like this:
In the above picture, series 3 is the plot of the sorted aggregate data of series 1 and 2. If you change the formatting of series 3 so that there is no line, you get something like this:
For my relatively small mock data sets, the results are admittedly poor (it was based on just 25 data points in each series), but if you have a large amount of closely spaced data, and you play around with the moving average window size, you might get something acceptable. If not, you should probably just interpolate both datasets to obtain two consistent time series.
I wish to automate the visualisation process of my activity monitoring process.
To do so, I import a starting time, ending time and activity name of an activity, afterwards I distribute these over separate sheets.
And now I'd like to first manually build a graph that visualizes the data in the desired way, and that's where I run into trouble.
Data and graph nearly visualized:
Obtaining the desired way entails 2 more small steps, the first and most important one, which I have troubles with is:
Visualising the difference between two plotted bars, on the actual location/height of the difference, in stead of two bars going up to the starting, and ending point of an activity. (practically inversing the area that is plotted with the area that is not plotted*)
Clustering the activities into 1 vertical bar consisting of several smaller bars floating in the right time, visualising the time of the activity. But that's something I'll look into after solving 1.
Ps. if you are interested in the actual .xlsm for your own use feel free to send me a pm.
*As seen I already tried to use just the actual duration of the activities with their respective starting time, but it only yielded 1 line of identical bars.
I need to make a graph from a list of values that don't line up with each other. There are samples being taken in a process at certain times, but they aren't always the same. For Sample A, the times are 1pm, 3pm, 5pm, etc. For Sample B the times are 2pm, 4pm, 6pm, etc. For Sample C the times are 1:30pm, 3:30pm, 5:30pm etc.
If I graph each sample individually they are fine, but when you graph them together you can only get the xy scatter points, but no lines since it thinks there are missing values. I just need a rough comparison of increase/decrease over time. If I could connect the dots with lines ignoring the missing values that would be great! I just don't know how to do that... Any suggestions?
This is for Access, otherwise this would work.
Excel - Connect Data Points with Line
You have to use old excel knowledge. In Access, set up the graph with xy scatter like you want. Then Tools > Options > Chart and select Interpolated. That should do it! Just took a while to find what it might be called and where it might be located.
I have 3 columns of data, eg:
http://i.stack.imgur.com/XjGmu.jpg
When Excel creates a line graph of this, the blue line is what i get.
This is not correct because the time stamp shows the time when something is switched on (255) or off (0) (could also be the current state eg 16:08). So I'd like a graph like this - see the red line (with a time-based X axis off course):
http://i.stack.imgur.com/vNvPk.jpg
Anyone can help? Thanks
As #Jon49 indicated, you need to plot additional data points--two y values for each x value: one to plot the point at y=255 and one to plot y=0.
If the time-span of the data is at least a few days, you can use a line chart. But in your case since the values are within a day, the scatter chart with straight lines is the only option due to the limitations of the scale units for line charts.
The key is the values need to be in the correct order. Each y=255 value needs to be followed by the next time-stamp's 255 value followed by it's 0 value, followed by the next time-stamp's 0 value:
Excel doesn't support this type of discrete value graph (at least not excel 2k3 that I am using); your best bet is to use a bar graph and then go into the settings and set the gap width down to zero.
Not sure what the best way is but I would automate what I describe below on how to do (unless this is a one time deal, then just brute force it):
Separate the 255s from the 0s. Make sure for every 0 time there is a corresponding 255 time. Take the zero times and put in a scatter plot then add a y-error bar and make the fixed value equal to 255. Format to how you like it.
Now for the 255s. Add those to the chart by pairs. Make the chart type for these pairs scatter plot with a line. Format how you would like them to look.
Let me know if that doesn't make sense to you.