I'm trying to do something in Grafana/Graphite which should be really easy, but has me baffled. I'd like to get the total by which a count has increased during the period I select in the top right corner of the Grafana UI.
Users of our site can import content from various sources, and every time they do an import, we increment a StatsD count. We include the source type in the StatsD path, so we can break out the numbers for each type of import in Grafana, as in this example.
Unfortunately, in this view of the graph Grafana is giving us crazy numbers. When we check them against another stats service we also use we see very different data. For example, the import Grafana shows as 96.9 in this screenshot, in our other system has a value of 4,466 over the last week. We're totally sure our other system is right and Grafana's number is wrong.
Can anyone suggest what we're doing wrong? We want a chart that'll give us the breakdown of number of imports of each type during the selected date range. Over 7 days, we'd expect the big green item to be 4,466, not 96.6. On a date range like 1 day, we'd expect it to be something like
629, because the count gets incremented by about that much per day.
Related
Do forgive me if this is not the correct Stack Exchange site for this question (and tell me which is).
We log machine breakdowns at a production plant. Of the data logged some is Machine Downtime, Location and Date. Machine Downtime being the duration of the breakdown, Location being which set of machines and well Date, the date of the breakdown.
The graph I'm trying to make will have a column graph and a line graph. There will be a column for each breakdown and the duration dictates the height (y-axis). The line graph will start at zero and increase by one every day there isn't a breakdown. If there is a breakdown, the count will reset to zero.
The current problems encountered are: I can't get the dates on the x-axis to be spaced evenly apart by time, they are only spaced by each data point and some months have twice as many breakdowns leading to uneven spread. Another is how will I filter the Locations? I know you can filter the way I want in PivotCharts but I don't think I'll be able to use a PivotChart for this. The last is the line graph itself, what function or formula would I write to achieve this?
I've done some Googling for each of my issues but haven't found exactly the results as I wanted. Things I need to be done doesn’t probably include any macros/VBA skills, just basic knowledge of Excel.
Now to my spreadsheet. I'm a Dota 2 player and I like statistics. I like it that much that I'd like to keep track of my achievements and results. Only problem is that the game tracker sucks and to get great information in web you have to pay for it, so I decided it's time for me to create my own spreadsheet to track my skills.
I don't know which place is the best to share my spreadsheet but I uploaded it to Estonian uploading host, link is here. I will also provide with pictures so you don't have to download anything.
This is what it looks like in general:
Problem number 1: The left table, or column has 1000 rows. In web design it's possible to make elements fixed depending on the scroll, I'd like to use similar feature here. If the table gets scrolled down, the right table (area with games, bonus and graph) will get scrolled down with it.
Problem number 2: Average MMR. I'd like to show average MMR after each entry depending on the first entries. Right now there's avg MMR for J4:J8. The calculation for J8 looks like this: =AVERAGE(C4:C8). For J7 it looks like this: =AVERAGE(C4:C7). I'd like to do this for all my 1000 rows, but I don't want to type it out. If I try to drag down from the corner, it will continue with C5:C8, C6:C9 etc (so it changes the starting point)
Problem number 3: Under longestGame there's currently Date and Hero. This should show the Date and Hero of which the longest game occurred. I tried to do this with LOOKUP function but it required table to be in ascending order, which I don't want. For current, 44,22, there should be Storm Spirit and 14.06.2015.
Problem number 4: Graph. I'd like to display three series on graph - MMR, average MMR and game length (time). The problem is, that MMR and average MMR will be in the numbers on 3000-7000 but the game length will only be probably in timeframe 20:00-120:00. Maybe it's possible to add two sets of values to the Y axis or maybe set Time series maximum 200:00 and minimum 0:00 and create graph according to this. I'm really stupid making graphs and I haven't figured out a clever way yet.
Problem number 5: Graph again. Right now I have to set the series for the graph. I've currently set it to C4:C54 (so 50 rows). I'd like it to move around a bit and by that I mean that if there happens to be C55-th game then the graph would start from C5:C55 and move along (so it'll count 50 last games).
I'm in a benevolent mood so rather than downvoting your question, because it is not really suitable for this forum I'm going to give you some hints and guidance. The numbers below correspond to the problems in your question.
Excel permits more than one window to be used on the same workbook -
so one window can show the data and one the summary.
Find out about absolute and relative cell addressing - its a valuable bit of knowledge for anyone serious about Excel and it will be of use in solving your problem.
Find out about the MATCH function. You can use this to find out which row of your table contains the longest game, shortest game, max MMR, min MMR by matching an element from the summary on the right (cols M onward) against the appropriate column table on the left. The find out about the INDEX function - this can be used to pull the values in the columns for Hero and Date which correspond to a specific row (such as the row containing the longest game, shortest game, etc). Search INDEX MATCH and find out why using these two functions in combination is often preferred to using the VLOOKUP function
Persevere - there are graph options available to do what you want and the only way to really learn is to go through the pain of trying them out, failing and working at it until you succeed.
Set up an area of worksheet to hold the 50*3 table of data for your graphs. Find out about the COUNT function and think how it might be of use in determining which rows of the data table map to the 50 rows of graph data. Then think about how to populate the graph data table using one of the functions mentioned above. Incidentally, C4:C54 is actually 51 rows, not 50.
I am pretty new to Kibana.
I am logging ssh access hits and I want to compare the access hit counts during night time vs day time. How can I get this data? Also, How can I visualize this?
Also, what if I want to compare hits on weekends vs weekdays?
I can only see continuous time-line on X-axis in the visualization tab.
Any help is appreciated.
Hi Your question is very useful & important for time based analysis in Kibana.The answer is based on Kibana 4.1.
For example you want to create visualizations for day vs week:-
1. Click on Visualize tab.
2. Select Line Chart & select from a new search.
3. Select Count in Y-Axis Metric
4. Select Date Range in X-Axis agregation, select date field in Field option, in From field option input the range such as :- now-1W & corresponding mention in To field option as :now-1d
5. Then click Split Lines & select terms and its field to display top N results for that time range.
Hope it answers your query.
You can use hits chart, or trends. Try use trends with 12 hours difference - and that would be day and night for example. Also you can tag your log entries with spec tags depending on time you sent them to server.
I have a table that tracks the dispatching of personnel. The table has the employee name and the date the person went out and the date they returned.
The table has hundreds of entries from 1988 to current.
In Excel I track the cumulative count per day (of the year) of how many people have been sent out, and I also track the number of people out on any given day. The table lists the Month & Day in the first column (every day of the year, including leap days) and the years on the first row. There is data for every date (a zero is entered until the first person is sent out that year, then starts counting up as there are more dispatches, or in the case of the number of people out each day, it will show zero if no one is out that day or if there were, say 5 people out, it would show "5" for that day). I then use the data in Excel to construct a graph that shows the number of dispatches on the y axis and the day of the year on the x axis (along with the current year’s number, the average number and the max over the 27 year history). Currently I just track this manually (I just keep a running count of each and enter it in manually in Excel.) I would like to build a query of my Access data that would return the same information that I could import into my Excel spreadsheet. One query that would show the day & month in the first column and the years along the top row and for each day show a cumulative count for that year of how many people have been sent out. Another query that has the day & month in the first column and the years along the top and a count of how many people were out for that particular day for that particular year. There shouldn't be any gaps (every day has data, even if it is "0"). I would then import those queries into Excel to replace my manual tracking that I am doing now.
I know how to construct the Excel stuff (I have that running already), and how to import info from Access to Excel, what I need to know is how to construct these 2 Access queries.
Any help/ideas on how to construct those 2 queries would be greatly appreciated!
I'd recommend that you migrate this app to a web based solution that uses a real database - SQL Server or MySQL, not Access.
"Desk drawer software" is what I call homegrown apps that someone creates for themselves to perform some small task that eventually become integral to running a business and grow out of hand. Your truck factor is 1: if anything happened to you, no one would know how to do this function. The software may not be backed up or checked into a source code management system. There's no QA. There's no way to migrate new features to production: if you alter the app, then that is what you have.
I'd recommend a web app to mitigate all the risks I've described:
You have to deploy a web app to a server, which takes it off your desktop and puts it in a central place where anyone who's authorized can access it.
Separates database from display issues.
Makes you think about how to archive historical data. Partitioning by year makes sense.
Likely you'll put this in a source code management system like Subversion or Git.
This is a multi-tiered project. Let me give a quick overview. I have attendance data, card/ timestamp punches. I would like to have a pivot table with slicers in Excel. Ideally you'd be able to choose a department / last name / associate number. And also a period of time. Ideally this would be a table with the company period/week. And maybe default to last weeks.
I can get at timecard data in two ways:
(1) generate a CSV that automatically performs the timecard math, to figure out how many hours someone worked and it is smart enough to understand 3rd shift workers. The format of that CSV is:
Last Name, First Name, Personnel Type, Associate Number, Facility, Department, TimeIn, TimeOut, Total Hours
The problem with this method is that I would have to manually append the information to the CSV tables. Or come up with some autoIT script.
(2) Get at the raw data via sql/odbc. This way the math is not done. It is just all of the associates timestamps. I would have to figure up the daily hours myself and figure out a 3rd shift formula too. It is not a set schedule, many people swing shifts and others get called in a lot.
Lastly, I would like to be able to filter the dates by using our company fiscal calendar. I have a spreadsheet that goes from 2000 to 2093. With everyday listed and it's corresponding year/period/week.
Example period info spreadsheet:
date Year Period week WeekTotal Period Total
12/3/2007 2008 1 1 2008.1.1 2008.1
12/4/2007 2008 1 1 2008.1.1 2008.1
I know there is a lot going on here, but what would be the best way to approach this project?
First I have not been able to post any script however the last I tried it I used two options 1. Was a php conversion where the time was numbers ( which makes it easier for calculations)
2. Was in the tables where I deliberately entered the values places the time in different columns or fields for hours, mins, and seconds this meant that while the input is eased I still have to calculate the output in php especially for totals, averages and differences.
Hope it helps a bit