deleting the entries in Azure Table Storage that Are Certain Days Old Automation - azure

Question

I want to automate to delete One month before entries in Azure Table Storage.
Now in Azure Table Storage I have one year entries. I delete it manually. but for the future, I need to automate this process. I need to delete one month old entries from Azure table storage.

Answer 1

If you have the partitionKey and the rowKey, you can Delete entries directly as the documentation mentions here https://msdn.microsoft.com/en-us/library/dd135727.aspx
, otherwise you will need to select entries first, know their (partitionKey, rowKey), then delete them.

Answer 2

Basically, to delete old entities in Azure Table Storage, you can first query them with Filtering on DateTime Properties, and then delete then in a loop.
And regarding
but for the future, I need to automate this process
If you have an Azure App Service, you can scheduling WebJobs to satisfy your requirement.
Additionally, you also can leverage Function App to implement the delete operations and use Azure Scheduler to configure a schedule to call your api.
Any further concern, please feel free to let me know.

Answer 3

You need to query the whole table, and find the entities whose Timestamp is older than 1 month, and delete all of them. Batch operation may be useful here since it supports deleting multiple entities with same partition key via one request.
Query entities REST API: https://msdn.microsoft.com/en-us/library/azure/dd179421.aspx
Delete entity REST API: https://msdn.microsoft.com/en-us/library/azure/dd135727.aspx
Perform entity group transaction in REST API: https://msdn.microsoft.com/en-us/library/azure/dd894038.aspx
If you're using C#, you can refer to this documentation: https://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-tables/#delete-an-entity

Answer 4

Take a look at the following Powershell solution for querying a storage table. You can incorporate this into some type of automation solution to run on a regular basis.
https://github.com/chriseyre2000/Powershell/tree/master/Azure2

Answer 5

This thread might be a little old, but I recently wrote a library to manage Azure Tables lifecycle.
Source/docs: https://github.com/pflajszer/AzureTablesLifecycleManager
Usage for your use case (and examples provided on the source page):
Using IQueryBuilder
public async Task<DataTransferResponse<T>> DoSomethingWithDataOlderThanAYearUsingQueryBuilder<T>(int option) where T : class, ITableEntity, new()
{
// this will return all the tables since it's an empty query:
var tableQuery =
new QueryBuilder();
// this will return all the data older than 1 year ago:
var dataQuery =
new QueryBuilder()
.AppendCondition(ODataPredefinedFilters.TimestampLessThanOrEqual(DateTime.Now.AddYears(-1)));
var dtr = new DataTransferResponse<T>();
switch (option)
{
case 1:
// this will move all the data that match the above filters to a new table:
var newTableName = "someNewTable";
newTableName.EnsureValidAzureTableName();
dtr = await _api.MoveDataBetweenTablesAsync<T>(tableQuery, dataQuery, newTableName);
break;
case 2:
// ...or delete it permanently:
dtr = await _api.DeleteDataFromTablesAsync<T>(tableQuery, dataQuery);
break;
case 3:
// ...or just fetch the data:
dtr = await _api.GetDataFromTablesAsync<T>(tableQuery, dataQuery);
break;
default:
break;
}
return dtr;
}
Using LINQ:
public async Task<DataTransferResponse<T>> DoSomethingWithDataOlderThanAYearUsingLINQExpression<T>(int option) where T : class, ITableEntity, new()
{
// this query will return all the tables:
Expression<Func<TableItem, bool>> tableQuery = x => true;
// this query will return all data in the above tables that matches the condition (all data older than 1 year ago)
Expression<Func<T, bool>> dataQuery = x => x.Timestamp < DateTime.Now.AddYears(-1);
var dtr = new DataTransferResponse<T>();
switch (option)
{
case 1:
// Moving the data to a new table:
var newTableName = "newTableName";
newTableName.EnsureValidAzureTableName();
dtr = await _api.MoveDataBetweenTablesAsync<T>(tableQuery, dataQuery, newTableName);
break;
case 2:
// this call will delete the data that match the above filters:
dtr = await _api.DeleteDataFromTablesAsync<T>(tableQuery, dataQuery);
break;
case 3:
// ...or just fetch the data:
dtr = await _api.GetDataFromTablesAsync<T>(tableQuery, dataQuery);
break;
default:
break;
}
return dtr;
}