As the title says, is it possible to rename an Azure Storage Table column?
I tried using the portal and through storage explorer with no results
Hmm.. Haven't had to do this before. But if I had to make a guess, it's that columns don't really exist in table storage. There is no schema anyway, you can include whatever fields in the objects. To rename a column, you'd probably have to go through all the objects and rename the field in them.
Related
here is my requirement:
I have an excel with few columns in it and few rows with data
I have uploaded this excel in Azure blob storage
Using ADF I need to read this excel and parse the records in it one by one and perform an action of creating dynamic folders in Azure blob.
This needs to be done for each and every record present in the excel.
Each record in the excel has some information that is going to help me create the folders dynamically.
Could someone help me in choosing the right set of activities or data flow in ADF to do this work?
Thanks in advance!
This is my Excel file as a Source.
I have created folders in Blob storage based on Country column.
I have selected DataFlow activity.
As shown in below screenshot, Go to Optimize tab of Sink configuration.
Now select Partition option as Set Partition.
Partition type as Key.
And Unique value per partition as Country column.
Now run Pipeline.
Expected Output:-
Inside these folders you will get files with corresponding data.
I need to copy file names of excel files that are in my Azure Storage as blobs and then put these names in the SQL Server table using ADF. It can be a file path as a name of a file but the hardest thing is that in the dataset which takes all the files from one specific folder I have to select a sheet name and these sheet names are different for each file, therefore it returns an error. Is there a way to create a collective dataset without indicating the sheet name?
So, if I understand your question correctly you are looking for a way to write all Excel filenames to a SQL Database using ADF.
You can use the generic Get Metadata activity and use a binary dataset as source. Select Child items as an field to retrieve. This will retrieve all files in the folder. Then add a filter to only select the Excel file types.
Hope that this gets you on the right track.
I am trying to copy form Azure Table Storage to Azure CosmosDb sql Api using Azure Data Factory V2.
During copy I want to add a new field (column) to each document by concatenating two of the column values from table. E.g. my table has 2 columns imageId and tenantId and I want to make the id of document in cosmos db like image_tenant_ImageID_TenantID.
For this I am trying to add dynamic content for "Additional Columns" under "Source" in the ADF but couldn't figure out how to do that. Can anyone please help with this?
Seems like I can not do it as part of copy activity. So first I copy from table to temporary cosmos container and then use dataflow to create new column by using "Derived column" activity and copy the results to a new container.
Another way to achieve this is using Change Feed to get your documents and then add this property. Then replace documents.
I want to delete some unnecessary columns which are created but not using currently. Without deleting the table data or table in Microsoft azure storage explorer how can I delete columns manually?
Mildly annoying but you can fix the issue by deleting the entire contents of:
AppData\Roaming\StorageExplorer
that will fix the issue. You'll need to reauthorized the accounts but that's a mild inconvenience. There's likely a file or two within that directory that actually caches that data that is a more surgical approach but the few most obvious candidates didn't work for me so i just deleted the whole directory.
It's not possible to delete columns for all entities in a table, since Azure Storage Table is a schema-less database. In other words, the entities within a table can have different properties respectively. You have to query all the entities, remove the useless properties from them one by one and then replace the modified entities back to the table.
Spinning off of the answer from #frigon I performed the following steps:
Close the storage explorer
Delete \AppData\Roaming\StorageExplorer\Local Storage
Open storage explorer
It kept me logged in, kept all my settings, and only appears to have reset my account filters and table caches.
It looks like the caches are stored in leveldb in that folder, so you may be able to crack that open and find the specific value(s) to drop.
I've found that if you copy the table and paste it into a different storage account or simply rename the table then the new table will not reference the unused columns. However, if you paste it back to the original location or rename the table back to the original name then the unused columns will still be shown, even if you delete the table first.
Strangely, if you create a brand new table with the same name it will only have the default columns. But import the contents of the original table from file and the superfluous columns will also reappear even though there is no reference to those columns in the csv file.
We are using azure table storage and have thousands of tables using the same schema. Now we are looking to add another column to these tables. How do we add another column to our existing tables without deleting the table and re-adding it?
Windows Azure Table Storage doesn't actually have columns. Each entity (e.g. a Row) is simply a set of properties, with no fixed schema. If you're using a strongly-typed class to write to your table, then you just need to add your new property to that class.
For older rows that don't have the new property, the value in your class object will be set to its default value.