azure storage table retrieve rows with correct data types. All values are returned as string instead of their correct type - node.js

Using Node.js with Azure Storage table.
I've created a new table with two fields/keys (CreatedOn and NextRenewalDate) as Int64 values (time since epoch). The entgen correctly identifies the values as Int64 as desired before I save them to the table.
I know Azure documentation says that it stores int64 as string. It has been like that for a loong time.
I have been parsing the values of properties I know are int64 in my app's code after fetching and before using.
I am wondering if that is still the case ? Is there any in built way of getting the correct data type for values when I read them from Azure Storage Tables ?

Only those 8 types are supported by the Table Service Data Model. For more details, please refer to here.
Edm.Binary
Edm.Boolean
Edm.DateTime
Edm.Double
Edm.Guid
Edm.Int32
Edm.Int64
Edm.String
Besides, when we retrieve entities from Azure table, the entity will return as OData JSON format. According to the OData protocol azure table support, Int64 will represent as string. For more details, please refer to here and here

I have not been able to find a built in way to read / convert Azure storage table entities back into json.
So I wrote a quick method, this will work as long as you requested metadata in payload.
const options = {payloadFormat:"application/json;odata=fullmetadata"};
Here is the quick function that might help somebody:
function convertArrayOfEntitiesToJson(arrEntities){
let arrRecords = [];
for(let x=0; x < arrEntities.length; x++){
const entity = arrEntities[x];
let allKeys = Object.keys( entity )
let record = {};
allKeys.forEach(function(key){
if(entity[key]['$'] === "Edm.Int64" )
record[key] = parseInt( entity[key]["_"] );
else if(entity[key]['$'] === "Edm.DateTime" )
record[key] = new Date( entity[key]["_"] );
else if(entity[key]['$'] === "Edm.Boolean" )
record[key] = ( entity[key]["_"] == 'true' );
else if(entity[key]['$'] === "Edm.Boolean" )
record[key] = ( entity[key]["_"] == 'true' );
else // fall back to string
record[key] = entity[key]["_"];
});
arrRecords.push(record);
}
return arrRecords;
}
I'm missing a couple of data types on purpose. I did not have a need for the rest of the types. Feel free to add other data types as needed.

Related

How can i simplify checking if a value exist in Json doc

Here is my scenario, i am parsing via javascript a webpage and then post the result to an restApi to store the json in a db. The code works fine as long as all fields i defined in my script are send. Problem is over time they website might change names for fields and that would cause my code to crash.
Originally i used code like this
const mySchool = new mls.School();
mySchool.highSchoolDistrict = data["HIGH SCHOOL DISTRICT"].trim();
mySchool.elementary = data.ELEMENTARY.trim();
mySchool.elementaryOther = data["ELEMENTARY OTHER"].trim();
mySchool.middleJrHigh = data["MIDDLE/JR HIGH"].trim();
mySchool.middleJrHighOther = data["MIDDLE/JR HIGH OTHER"].trim();
mySchool.highSchool = data["HIGH SCHOOL"].trim();
mySchool.highSchoolOther = data["HIGH SCHOOL OTHER"].trim();
newListing.school = mySchool;
but when the element does not exist it complains about that it can not use trim of undefined. So to fix this i came up with this
if (data["PATIO/PORCH"]) {
newExterior.patioPorch = data["PATIO/PORCH"].trim();
}
this works but i am wondering if there is a more global approach then to go and check each field if it is defined ?
You could leverage a sort of helper function to check first if the item is undefined, and if not, return a trim()-ed version of the string.
var data = Array();
data["HIGH SCHOOL DISTRICT"] = " 123 ";
function trimString(inputStr) {
return (inputStr != undefined && typeof inputStr == "string") ? inputStr.trim() : undefined;
}
console.log(trimString(data["HIGH SCHOOL DISTRICT"]));
console.log(trimString(data["ELEMENTARY OTHER"]));

How to access 'Abbreviation' field of a custom list in NetSuite custom lists

I have a custom list that is used as a matrix option of Inventory item. Its 'Color'. This custom list has an abbreviation column. I am creating a saved search on item and using Color field(join) and trying to access 'Abbreviation' field of color.
Abbreviation on custom list is available on when 'Matrix Option List' is checked.
Can someone please help me achieve this? I tried to do this through script and it seems like we cannot access 'Abbreviation' column through script. I also tried to use script to write a search directly on 'Color' - custom list and get the 'abbreviation' through search columns. It did not work. Is there a way to access 'Abbreviation' from custom lists?
Thanks in Advance
You can access it via suitescript by using the record type "customlist" and the internal id of the list like so:
var rec = nlapiLoadRecord('customlist', 5);
var abbreviation = rec.getLineItemValue('customvalue', 'abbreviation', 1);
nlapiLogExecution('DEBUG', 'abbreviation', abbreviation);
Keep in mind that the third argument of getLineItemValue is the line number, not the internal ID of the item in the list. If you want to find a specifc line item, you may want to use rec.findLineItemValue(group, fldnam, value).
Unfortunately, it doesn't look like this translates to saved searches. The suiteanswer at https://netsuite.custhelp.com/app/answers/detail/a_id/10653 has the following code:
var col = new Array();
col[0] = new nlobjSearchColumn('name');
col[1] = new nlobjSearchColumn('internalid');
var results = nlapiSearchRecord('customlist25', null, null, col);
for ( var i = 0; results != null && i < results.length; i++ )
{
var res = results[i];
var listValue = (res.getValue('name'));
var listID = (res.getValue('internalid'));
nlapiLogExecution('DEBUG', (listValue + ", " + listID));
}
However, whatever part of the application layer translates this into a query doesn't handle the abbreviation field. One thing to keep in mind is that the 'custom list' record is basically a header record, and each individual entry is it's own record that ties to it. You can see some of the underlying structure here, but the takeaway is that you'd need some way to drill-down into the list entries, and the saved search interface doesn't really support it.
I could be wrong, but I don't think there's any way to get it to execute in a saved search as-is. I thought the first part of my answer might help you find a workaround though.
Here is a NetSuite SuiteScript 2.0 search I use to find the internalId for a given abbreviation in a custom list.
/**
* look up the internal id value for an abbreviation in a custom list
* #param string custom_list_name
* #param string abbreviation
* #return int
* */
function lookupNetsuiteCustomListInternalId( custom_list_name, abbreviation ){
var internal_id = -1;
var custom_list_search = search.create({
type: custom_list_name,
columns: [ { name:'internalId' }, { name:'abbreviation' } ]
});
var filters = [];
filters.push(
search.createFilter({
name: 'formulatext',
formula: "{abbreviation}",
operator: search.Operator.IS,
values: abbreviation
})
);
custom_list_search.filters = filters;
var result_set = custom_list_search.run();
var results = result_set.getRange( { start:0, end:1 } );
for( var i in results ){
log.debug( 'found custom list record', results[i] );
internal_id = results[i].getValue( { name:'internalId' } );
}
return internal_id;
}
Currently NetSuite does not allows using join on matrix option field. But as you mentioned, you can use an extra search to get the result, you could first fetch color id from item and then use search.lookupFields as follows
search.lookupFields({ type: MATRIX_COLOR_LIST_ID, id: COLOR_ID, columns: ['abbreviation'] });
Note: Once you have internalid of the color its better to use search.lookupFields rather than creating a new search with search.create.

Sitecore HOWTO: Search item bucket for items with specific values

I have an item bucket with more then 30 000 items inside. What I need is to quickly search items that have particular field set to particular value, or even better is to make something like SELECT WHERE fieldValue IN (1,2,3,4) statement. Are there any ready solutions?
I searched the web and the only thing I found is "Developer's Guide to Item
Buckets and Search" but there is no code examples.
You need something like this. The Bucket item is an IIndexable so it can be searched using Sitecore 7 search API.
This code snippet below can easily be adapted to meet your needs and it's just a question of modifying the where clause.if you need any further help with the sitecore 7 syntax just write a comment on the QuickStart blog post below and I'll get back to you.
var bucketItem = Sitecore.Context.Database.GetItem(bucketPath);
if (bucketItem != null && BucketManager.IsBucket(bucketItem))
{
using (var searchContext = ContentSearchManager.GetIndex(bucketItem as IIndexable).CreateSearchContext())
{
var result = searchContext.GetQueryable<SearchResultItem().Where(x => x.Name == itemName).FirstOrDefault();
if(result != null)
Context.Item = result.GetItem();
}
}
Further reading on my blog post here:
http://coreblimey.azurewebsites.net/sitecore-7-search-quick-start-guide/
Using Sitecore Content Editor:
Go to the bucket item then In search tab, start typing the following (replace fieldname and value with actual field name and value):
custom:fieldname|value
Then hit enter, you see the result of the query, you can multiple queries at once if you want.
Using Sitecore Content Search API:
using Sitecore.ContentSearch;
using Sitecore.ContentSearch.Linq;
using Sitecore.ContentSearch.SearchTypes;
using Sitecore.ContentSearch.Linq.Utilities
ID bucketItemID = "GUID of your bucket item";
ID templateID = "Guid of your item's template under bucket";
string values = "1,2,3,4,5";
using (var context = ContentSearchManager.GetIndex("sitecore_web_index").CreateSearchContext())
{
var predicate = PredicateBuilder.True<SearchResultItem>();
predicate = PredicateBuilder.And(item => item.TemplateId == new ID(templateID)
&& item.Paths.Contains(bucketItemID));
var innerPredicate = PredicateBuilder.False<SearchResultItem>();
foreach(string val in values.Split(','))
{
innerPredicate = PredicateBuilder.False<SearchResultItem>();
innerPredicate = innerPredicate.Or(item => item["FIELDNAME"] == val);
}
predicate = predicate.And(innerPredicate);
var result = predicate.GetResults();
List<Item> ResultsItems = new List<Item>();
foreach (var hit in result.Hits)
{
Item item = hit.Document.GetItem();
if(item !=null)
{
ResultsItems .Add(item);
}
}
}
The following links can give good start with the Search API:
http://www.fusionworkshop.co.uk/news-and-insight/tech-lab/sitecore-7-search-a-quickstart-guide#.VPw8AC4kWnI
https://www.sitecore.net/learn/blogs/technical-blogs/sitecore-7-development-team/posts/2013/06/sitecore-7-poco-explained.aspx
https://www.sitecore.net/learn/blogs/technical-blogs/sitecore-7-development-team/posts/2013/05/sitecore-7-predicate-builder.aspx
Hope this helps!

Linq Invalid Cast Exception Same Object Type

I wrote this query and as my understanding of the business rules has improved I have modified it.
In this most recent iteration I was testing to see if indeed I had some redundancy that could be removed. Let me first give you the query then the error.
public List<ExternalForums> GetAllExternalForums(int extforumBoardId)
{
List<ExternalForums> xtrnlfrm = new List<ExternalForums>();
var query = _forumExternalBoardsRepository.Table
.Where(id => id.Id == extforumBoardId)
.Select(ExtForum => ExtForum.ExternalForums);
foreach (ExternalForums item in query)
{
xtrnlfrm.Add(new ExternalForums { Id = item.Id , ForumName = item.ForumName, ForumUrl = item.ForumUrl });
}
return xtrnlfrm;
}
Now in case it isn't obvious the query select is returning List of ExternalForums. I then loop through said list and add the items to another List of ExternalForums object. This is the redundancy I was expecting to remove.
Precompiler was gtg so I ran through it one time to very everything was kosher before refactoring and ran into a strange error as I began the loop.
Unable to cast object of System.Collections.Generic.HashSet
NamSpcA.NamSpcB.ExternalForums to type NamSpcA.NamSpcB.ExternalForums.
Huh? They are the same object types.
So am I doing something wrong in the way I am projecting my select?
TIA
var query = _forumExternalBoardsRepository.Table
.Where(id => id.Id == extforumBoardId)
.Select(ExtForum => ExtForum.ExternalForums);
This query returns IEnumerable<T> where T is type of ExtForum.ExternalForums property, which I would expect to be another collection, this time of ExternalForum. And the error message matches that, saying you have IEnumerable<HashSet<ExternalForums>>.
If you need that collection of collections to be flattened into one big collection of ExternalForums use SelectMany instead:
var query = _forumExternalBoardsRepository.Table
.Where(id => id.Id == extforumBoardId)
.SelectMany(ExtForum => ExtForum.ExternalForums);

Reconstructing an ODataQueryOptions object and GetInlineCount returning null

In an odata webapi call which returns a PageResult I extract the requestUri from the method parameter, manipulate the filter terms and then construct a new ODataQueryOptions object using the new uri.
(The PageResult methodology is based on this post:
http://www.asp.net/web-api/overview/odata-support-in-aspnet-web-api/supporting-odata-query-options )
Here is the raw inbound uri which includes %24inlinecount=allpages
http://localhost:59459/api/apiOrders/?%24filter=OrderStatusName+eq+'Started'&filterLogic=AND&%24skip=0&%24top=10&%24inlinecount=allpages&_=1376341370337
Everything works fine in terms of the data returned except Request.GetInLineCount returns null.
This 'kills' paging on the client side as the client ui elements don't know the total number of records.
There must be something wrong with how I'm constructing the new ODataQueryOptions object.
Please see my code below. Any help would be appreciated.
I suspect this post may contain some clues https://stackoverflow.com/a/16361875/1433194 but I'm stumped.
public PageResult<OrderVm> Get(ODataQueryOptions<OrderVm> options)
{
var incomingUri = options.Request.RequestUri.AbsoluteUri;
//manipulate the uri here to suit the entity model
//(related to a transformation needed for enumerable type OrderStatusId )
//e.g. the query string may include %24filter=OrderStatusName+eq+'Started'
//I manipulate this to %24filter=OrderStatusId+eq+'Started'
ODataQueryOptions<OrderVm> options2;
var newUri = incomingUri; //pretend it was manipulated as above
//Reconstruct the ODataQueryOptions with the modified Uri
var request = new HttpRequestMessage(HttpMethod.Get, newUri);
//construct a new options object using the new request object
options2 = new ODataQueryOptions<OrderVm>(options.Context, request);
//Extract a queryable from the repository. contents is an IQueryable<Order>
var contents = _unitOfWork.OrderRepository.Get(null, o => o.OrderByDescending(c => c.OrderId), "");
//project it onto the view model to be used in a grid for display purposes
//the following projections etc work fine and do not interfere with GetInlineCount if
//I avoid the step of constructing and using a new options object
var ds = contents.Select(o => new OrderVm
{
OrderId = o.OrderId,
OrderCode = o.OrderCode,
CustomerId = o.CustomerId,
AmountCharged = o.AmountCharged,
CustomerName = o.Customer.FirstName + " " + o.Customer.LastName,
Donation = o.Donation,
OrderDate = o.OrderDate,
OrderStatusId = o.StatusId,
OrderStatusName = ""
});
//note the use of 'options2' here replacing the original 'options'
var settings = new ODataQuerySettings()
{
PageSize = options2.Top != null ? options2.Top.Value : 5
};
//apply the odata transformation
//note the use of 'options2' here replacing the original 'options'
IQueryable results = options2.ApplyTo(ds, settings);
//Update the field containing the string representation of the enum
foreach (OrderVm row in results)
{
row.OrderStatusName = row.OrderStatusId.ToString();
}
//get the total number of records in the result set
//THIS RETURNS NULL WHEN USING the 'options2' object - THIS IS MY PROBLEM
var count = Request.GetInlineCount();
//create the PageResult object
var pr = new PageResult<OrderVm>(
results as IEnumerable<OrderVm>,
Request.GetNextPageLink(),
count
);
return pr;
}
EDIT
So the corrected code should read
//create the PageResult object
var pr = new PageResult<OrderVm>(
results as IEnumerable<OrderVm>,
request.GetNextPageLink(),
request.GetInlineCount();
);
return pr;
EDIT
Avoided the need for a string transformation of the enum in the controller method by applying a Json transformation to the OrderStatusId property (an enum) of the OrderVm class
[JsonConverter(typeof(StringEnumConverter))]
public OrderStatus OrderStatusId { get; set; }
This does away with the foreach loop.
InlineCount would be present only when the client asks for it through the $inlinecount query option.
In your modify uri logic add the query option $inlinecount=allpages if it is not already present.
Also, there is a minor bug in your code. The new ODataQueryOptions you are creating uses a new request where as in the GetInlineCount call, you are using the old Request. They are not the same.
It should be,
var count = request.GetInlineCount(); // use the new request that your created, as that is what you applied the query to.

Resources