Best way to store replay data - azure

Question

I currently have a game replay file I'm analyzing and was wondering the best way to store the data so I can create complex queries which are fast. For example, every 50 milliseconds the analyzer returns a data structure in which you can access a snapshot of the current round details and current players status in the game such has what weapon hes holding, how much health he has, which players he has currently shot, etc. I want the ability to say: From the start of the replay file to 10000 milliseconds in, what were the player "Micheal" positions. How much damage has the player "Kyle" to other players from 10000ms to 20000ms in. I want the ability to store all the data im analyzing and replay it on a frontend using a API so you can visually replay it.
I can store metadata about the replay into the database such as: (Round 1, StartTime: 10000, EndTime: 30000), (Round 2, StartTime: 31000, EndTime: 37000). I can also store meta data on when a player was killed (Kyle, DeathTime: 31000, KilledBy: Micheal) or when a player was hurt (Kyle, HurtBy: Micheal, Damage: 10, Weapon: x).
To accomplish what I want to do of being able to create complex queries for different situations, do I need a combination of the two? Such as storing the millisecond by millisecond data in like a NoSql database / Document and then also parsing the total file and storing metadata like mentioned in the second paragraph into another database. Is it not feasible to store only the millisecond by millisecond data and then have the ability to create fast queries to parse what I want from it?

Answer 1

Sounds like a cool game that you are working on. The Microsoft Azure Table Storage (a NoSQL database) is very fast and would be perfect for your game. You can use Slazure (I coded Slazure BTW) which has a query language (a custom LINQ provider) included and which also stores the data in the Azure Table Storage. Here is how I would do it if I was using Slazure and the Microsoft Azure Table Storage NoSQL database for your game. I suggest that you would store one row in the PlayersTable table for each event where the PartitionKey is the player name and the RowKey is how many milliseconds into the game round you are - since each table is indexed on the PartitionKey column and sorted by the RowKey column all queries using these columns are very fast indeed. There is also columns created for the weapon used, position on the screen, health, round number, and which player was killed by whom. You could create a table to hold damage data using the same proposed method as below:
using SysSurge.Slazure;
using SysSurge.Slazure.Linq;
using SysSurge.Slazure.Linq.QueryParser;
namespace TableOperations
{
public class PlayerInfo
{
// List of weapons
public enum WeaponList {
Axe, Arrow, Knife, Sling
};
// Update a player with some new data
public void UpdatePlayerData(dynamic playersTable, DateTime gameStartedTime, int round, string playerName, WeaponList weapon, int healthPoints, int xPos, int yPos)
{
// Create an entity in the Players table using the player name as the PartitionKey, the entity is created if it doesn't already exist
var player = playersTable.Entity(playerName);
// Store the time the event was recorded for later as milliseconds since the game started.
// This means there is one row for each stored player event
player.Rowkey = ((DateTime.UtcNow.Ticks - gameStartedTime.Ticks)/10000).ToString("d19");
player.Round = round; // Round number
player.Weapon = (int)weapon; // Weapon carried by player. Example Axe
// Store player X and Y position coordinates on the screen
player.X = xPos;
player.Y = yPos;
// Number of health points, zero means player is dead
player.HealthPoints = healthPoints;
// Save the entity to the Azure Table Service storage
player.Save()
}
// Update a player with some new data
public void PlayerKilled(dynamic playersTable, DateTime gameStartedTime, int round, string playerName, string killedByPlayerName)
{
// Create an entity in the Players table using the player name as the PartitionKey, the entity is created if it doesn't already exist
var player = playersTable.Entity(playerName);
// Store the time the event was recorded for later as milliseconds since the game started.
// This means there is one row for each stored player event
player.Rowkey = ((DateTime.UtcNow.Ticks - gameStartedTime.Ticks)/10000).ToString("d19");
player.Round = round; // Round number
// Number of health points, zero means player is dead
player.HealthPoints = 0;
player.KilledByPlayerName = killedByPlayerName; // Killed by this player, example "Kyle"
// Save the entity to the Azure Table Service storage
player.Save()
}
// Get all the player positions between two time intervals
public System.Linq.IQueriable GetPlayerPositions(dynamic playersTable, string playerName, int fromMilliseconds, int toMilliseconds, int round)
{
return playersTable.Where("PrimaryKey == #0 && RowKey >= #1 && RowKey <= #2 && Round == #3",
playerName, fromMilliseconds.ToString("d19"), toMilliseconds.ToString("d19"), round).Select("new(X, Y)");
}
}
}
First you need to record when the game started and the round number:
var gameStartedTime = DateTime.UtcNow;
var round = 1; // Round #1
, and create a table in the NoQL database:
// Get a reference to the Table Service storage
dynamic storage = new DynStorage("UseDevelopmentStorage=true");
// Get reference to the Players table, it's created if it doesn't already exist
dynamic playersTable = storage.Players;
Now, during the game you can continuously update the player information like so:
UpdatePlayerData(playersTable, gameStartedTime, round, "Micheal", WeaponList.Axe, 45, 12313, 2332);
UpdatePlayerData(playersTable, gameStartedTime, round, "Kyle", WeaponList.Knife, 100, 13343, 2323);
If you need to wait 50 ms between each storage event you can do something like:
System.Threading.Thread.Sleep(50);
, and then store some more player event data:
UpdatePlayerData(playersTable, gameStartedTime, round, "Micheal", WeaponList.Axe, 12, 14555, 1990);
UpdatePlayerData(playersTable, gameStartedTime, round, "Kyle", WeaponList.Sling, 89, 13998, 2001);
When one of the players has died you can call the same method with zero health points and the name of the player that killed him/her:
PlayerKilled(playersTable, gameStartedTime, round, "Micheal", "Kyle");
Now, later in your game analyser you can query for all the positions from the start of the game (0 ms) to 10,000 ms into the game as follows:
// Get a reference to the table storage and the table
dynamic queryableStorage = new QueryableStorage<DynEntity>("UseDevelopmentStorage=true");
QueryableTable<DynEntity> queryablePlayersTable = queryableStorage.PlayersTable;
var playerPositionsQuery = GetPlayerPositions(queryablePlayersTable, "Micheal", 0, 10000, round);
// Cast the query result to a dynamic so that we can get access its dynamic properties
foreach (dynamic player in playerPositionsQuery)
{
// Show player positions in the console
Console.WriteLine("Player position: Name=" + player.PrimaryKey + ", Game time MS " + player.RowKey + ", X-position=" + player.X + ", Y-position=" + player.Y;
}