I'm using couchDB in the fabric for state database in peers. I have a json data that is stored inside the ledger
JSON DATA
"Company": {
"Associate": {
"entity": {
"name": "foo",
"role": "admin"
},
"access": [
{
"create": "allowed"
}
]
}
}
}
I have to query the data based on access json object, like these have values like "create","readonly","delete" etc..
Currently I tried this but none of the records came up.
{
"selector": {
"Company": {
"Associate": {
"access": {
"$elemMatch": {
"$eq": "create"
}
}
}
}
}
}
How can I query the data's ?
I think you want (I use dot notation for simplicity):
{
"selector": {
"Company.Associate.access": {
"$elemMatch": {
"create": {
"$exists": true
}
}
}
}
}
...or maybe...
{
"selector": {
"Company.Associate.access": {
"$elemMatch": {
"create": "allowed"
}
}
}
}
In troubleshooting this problem we found that swagger.json contained incorrect case for
"type": "String"
in the generated code
"/api/FrameLookUp": {
"post": {
"tags": [
"Frame"
],
"operationId": "FrameLookup",
"consumes": [
"application/json-patch+json",
"application/json",
"text/json",
"application/*+json"
],
"produces": [
"application/json"
],
"parameters": [
{
"in": "header",
"name": "Authorization",
"description": "access token",
"required": true,
"type": "String"
},
{
"in": "body",
"name": "body",
"schema": {
"$ref": "#/definitions/FrameRequest"
}
}
],
"responses": {
"200": {
"description": "Success",
"schema": {
"$ref": "#/definitions/FrameResponse"
}
}
}
}
}
}
I have the following ISchemaFilter
public class SwaggerEnumFilter : ISchemaFilter
{
public void Apply(OpenApiSchema model, SchemaFilterContext context)
{
if (model == null)
throw new ArgumentNullException("model");
if (context == null)
throw new ArgumentNullException("context");
if (context.Type.IsEnum)
model.Extensions.Add(
"x-ms-enum",
new OpenApiObject
{
["name"] = new OpenApiString(context.Type.Name),
["modelAsString"] = new OpenApiBoolean(false)
}
);
}
}
What could be causing this?
It turned out to be that I was using
services.AddSwaggerGen(c =>
{
c.OperationFilter<AuthorizationHeaderParameterOperationFilter>();
and inside the class I had
Schema = new OpenApiSchema() { Type = "String" },
it should have had "string" as lower case.
I want to parse json, based on the following classes:
public class DerModel
{
public string Name { get; set; }
public string Email { get; set; }
}
public class DriverPositiveResultModel
{
public int DriverId { get; set; }
public string DriverName { get; set; }
public string DriverSSN { get; set; }
public string CarrierName { get; set; }
public DerModel DER { get; set; }
}
and the following schema:
{
"properties": {
"CarrierName": {
"type": "string"
},
"DER": {
"properties": {
"Email": {
"type": "string"
},
"Name": {
"type": "string"
}
},
"type": "object"
},
"DriverId": {
"type": "integer"
},
"DriverName": {
"type": "string"
},
"DriverSSN": {
"type": "string"
}
},
"type": "object"
}
but logic allows, that DER can be null. How to set it in the schema?
You need to specify that it can be null:
"type": ["object","null"]
so your code would look like this:
{
"properties": {
"CarrierName": {
"type": "string"
},
"DER": {
"properties": {
"Email": {
"type": "string"
},
"Name": {
"type": "string"
}
},
"type": ["object","null"]
},
"DriverId": {
"type": "integer"
},
"DriverName": {
"type": "string"
},
"DriverSSN": {
"type": "string"
}
},
"type": "object"
}
I have an azure function that calls a Rest API with basic authentication.
The respone form the server is ok but I have a problem with only receiving partial data back from the response - some fields are coming back as null.
I am using httpclient to make the rest API call. Here is my code:
using (var client = new HttpClient())
{
var webUrl = "https://api.livechatinc.com/chats/XXXXXX";
client.BaseAddress = new Uri(webUrl);
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
var user = "XXXXXXXXXX";
var password = "XXXXXXXXX";
var base64String = Convert.ToBase64String(Encoding.ASCII.GetBytes($"{user}:{password}"));
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", base64String);
HttpResponseMessage response = await client.GetAsync(webUrl);
response.EnsureSuccessStatusCode();
//Only getting partial information
string jsonString = await response.Content.ReadAsStringAsync();
//Deserialise JSON
Chat responseData = JsonConvert.DeserializeObject<Chat>(jsonString);
}
When I run my azure function debugger I can prove that I get null fields for the URL data that I am expecting.
Notice that responseData field shows a null value for ChatStartUrl after desrialisation.
This is because the field is not being returned from the response content.
I have made sure it is not my data contract and JSON.net deserialisation. My data contract is :
public class Chat
{
[JsonProperty("type")]
public string Type { get; set; }
[JsonProperty("id")]
public string Id { get; set; }
[JsonProperty("tickets")]
public Ticket[] Tickets { get; set; }
[JsonProperty("visitor_name")]
public string VisitorName { get; set; }
[JsonProperty("visitor_id")]
public string VisitorId { get; set; }
[JsonProperty("visitor_ip")]
public string VisitorIp { get; set; }
[JsonProperty("visitor")]
public Visitor Visitor { get; set; }
[JsonProperty("agents")]
public Agent[] Agents { get; set; }
[JsonProperty("supervisors")]
public object[] Supervisors { get; set; }
[JsonProperty("rate")]
public string Rate { get; set; }
[JsonProperty("duration")]
public long Duration { get; set; }
[JsonProperty("chat_start_url")]
public Uri ChatStartUrl { get; set; }
[JsonProperty("referrer")]
public Uri Referrer { get; set; }
[JsonProperty("group")]
public long[] Group { get; set; }
[JsonProperty("started")]
public string Started { get; set; }
[JsonProperty("custom_variables")]
public object[] CustomVariables { get; set; }
[JsonProperty("pending")]
public bool Pending { get; set; }
[JsonProperty("tags")]
public object[] Tags { get; set; }
[JsonProperty("timezone")]
public string Timezone { get; set; }
[JsonProperty("greeting")]
public Greeting Greeting { get; set; }
[JsonProperty("messages")]
public Event[] Messages { get; set; }
/* [JsonProperty("prechat_survey")]
public PrechatSurvey[] PrechatSurvey { get; set; }*/
[JsonProperty("events")]
public Event[] Events { get; set; }
[JsonProperty("engagement")]
public string Engagement { get; set; }
[JsonProperty("started_timestamp")]
public long StartedTimestamp { get; set; }
[JsonProperty("ended_timestamp")]
public long EndedTimestamp { get; set; }
[JsonProperty("ended")]
public string Ended { get; set; }
}
public class Agent
{
[JsonProperty("display_name")]
public string DisplayName { get; set; }
[JsonProperty("email")]
public string Email { get; set; }
[JsonProperty("ip")]
public string Ip { get; set; }
}
public class Event
{
[JsonProperty("author_name", NullValueHandling = NullValueHandling.Ignore)]
public string AuthorName { get; set; }
[JsonProperty("text")]
public string Text { get; set; }
[JsonProperty("message_json", NullValueHandling = NullValueHandling.Ignore)]
public string MessageJson { get; set; }
[JsonProperty("date")]
public string Date { get; set; }
[JsonProperty("timestamp")]
public long Timestamp { get; set; }
[JsonProperty("agent_id", NullValueHandling = NullValueHandling.Ignore)]
public string AgentId { get; set; }
[JsonProperty("user_type")]
public string UserType { get; set; }
[JsonProperty("type")]
public string Type { get; set; }
[JsonProperty("welcome_message", NullValueHandling = NullValueHandling.Ignore)]
public bool? WelcomeMessage { get; set; }
[JsonProperty("event_type", NullValueHandling = NullValueHandling.Ignore)]
public string EventType { get; set; }
}
public class Greeting
{
[JsonProperty("id")]
public long Id { get; set; }
[JsonProperty("name")]
public string Name { get; set; }
}
public class Ticket
{
[JsonProperty("id")]
public string Id { get; set; }
}
public class Visitor
{
[JsonProperty("id")]
public string Id { get; set; }
[JsonProperty("name")]
public string Name { get; set; }
[JsonProperty("email")]
public string Email { get; set; }
[JsonProperty("ip")]
public string Ip { get; set; }
[JsonProperty("city")]
public string City { get; set; }
[JsonProperty("region")]
public string Region { get; set; }
[JsonProperty("country")]
public string Country { get; set; }
[JsonProperty("country_code")]
public string CountryCode { get; set; }
[JsonProperty("timezone")]
public string Timezone { get; set; }
[JsonProperty("user_agent")]
public string UserAgent { get; set; }
}
What's interesting is that the rest api works fine in Postman and I can get all data back.
This is what I get from the Postman response
{
"type": "chat",
"id": "PPTE7XI3AQ",
"tickets": [
{
"id": "DJWKT"
}
],
"visitor_name": "Rupinder Kaur",
"visitor_id": "S1552009602.e395581450",
"visitor_ip": "49.183.59.243",
"visitor": {
"id": "S1552009602.e395581450",
"name": "Rupinder Kaur",
"email": "rupinder#creativa.com.au",
"ip": "49.183.59.243",
"city": "Abbotsford",
"region": "Victoria",
"country": "Australia",
"country_code": "AU",
"timezone": "Australia/Victoria",
"user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36"
},
"agents": [
{
"display_name": "David",
"email": "david.sender#wiise.com",
"ip": "203.30.93.14"
}
],
"supervisors": [],
"rate": "not_rated",
"duration": 357,
"chat_start_url": "https://wiise.com/au/explore/detailed-accounting/?utm_source=googlewiise&utm_medium=paidsearch&utm_campaign=Wiise&gclid=EAIaIQobChMI3ZrLuqG14QIVBR2PCh0VIAb7EAAYASAAEgKkWPD_BwE",
"referrer": "https://www.google.com/",
"group": [
0
],
"started": "Thu, 04/04/19 12:25:43 pm",
"custom_variables": [],
"pending": false,
"tags": [],
"timezone": "Australia/Sydney",
"greeting": {
"id": 7991,
"name": "Invite after 360 seconds"
},
"messages": [
{
"author_name": "David",
"text": "Hi there :) \n\nThank you for checking out our website, how can I help?",
"message_json": "",
"date": "Thu, 04/04/19 12:25:43 pm",
"timestamp": 1554341143,
"agent_id": "david.sender#wiise.com",
"user_type": "agent",
"type": "message",
"welcome_message": true
},
{
"author_name": "Rupinder Kaur",
"text": "hi there",
"message_json": "",
"date": "Thu, 04/04/19 12:25:54 pm",
"timestamp": 1554341154,
"user_type": "visitor",
"type": "message"
},
{
"author_name": "Rupinder Kaur",
"text": "I would like to trial it",
"message_json": "",
"date": "Thu, 04/04/19 12:26:01 pm",
"timestamp": 1554341161,
"user_type": "visitor",
"type": "message"
},
{
"author_name": "Rupinder Kaur",
"text": "I am mainly looking for cashflow reports",
"message_json": "",
"date": "Thu, 04/04/19 12:26:25 pm",
"timestamp": 1554341185,
"user_type": "visitor",
"type": "message"
}
],
"prechat_survey": [
{
"key": "Name:",
"value": "xxxxxx",
"id": "2001"
},
{
"key": "E-mail:",
"value": "xxxxxx",
"id": "2002"
}
],
"events": [
{
"author_name": "David",
"text": "Hi there :) \n\nThank you for checking out our website, how can I help?",
"message_json": "",
"date": "Thu, 04/04/19 12:25:43 pm",
"timestamp": 1554341143,
"agent_id": "david.sender#wiise.com",
"user_type": "agent",
"type": "message",
"welcome_message": true
},
{
"author_name": "XXXXXX",
"text": "XXXXXXXXXX",
"message_json": "",
"date": "Thu, 04/04/19 12:25:54 pm",
"timestamp": 1554341154,
"user_type": "visitor",
"type": "message"
},
{
"author_name": "XXXXXX",
"text": "XXXXXXXXXX",
"message_json": "",
"date": "Thu, 04/04/19 12:26:01 pm",
"timestamp": 1554341161,
"user_type": "visitor",
"type": "message"
},
{
"author_name": "XXXXXX",
"text": "XXXXXXXXXX",
"message_json": "",
"date": "Thu, 04/04/19 12:26:25 pm",
"timestamp": 1554341185,
"user_type": "visitor",
"type": "message"
},
{
"text": "Rupinder Kaur left the chat.",
"date": "Thu, 04/04/19 12:31:40 pm",
"timestamp": 1554341500,
"type": "event",
"event_type": "closed",
"user_type": "visitor"
}
],
"engagement": "auto_invite",
"started_timestamp": 1554341143,
"ended_timestamp": 1554341500,
"ended": "Thu, 04/04/19 12:31:40 pm"}
I have tried deserialisaing the contents returned from Postman and I get all the data with no null fields
//This works
var testJSON = System.IO.File.ReadAllText(#"C:\temp\ChatSchema.json");
Chat responseData = JsonConvert.DeserializeObject<Chat>(testJSON);
So I suspect that the Content.ReadAsStringAsync function is not owrking as expected.
I tried the following and still had the same issue:
var buffer = await response.Content.ReadAsByteArrayAsync();
var responseString = Encoding.UTF8.GetString(buffer, 0, buffer.Length);
string content = responseString;
Chat des = JsonConvert.DeserializeObject<Chat>(content);
To read in the raw text and then de-serialise it to the model class. I also had to change the serialisation routine to save as 'UTF-8'.
You could refer to the code as below:
var stream = response.Content.ReadAsStreamAsync().Result;
StreamReader reader = new StreamReader(stream);
string text = reader.ReadToEnd();
string content = text;
string des = JsonConvert.DeserializeObject<Chat>(content);
Just solved my problem !
I had to add a custom header since LiveChat required it:
client.DefaultRequestHeaders.Add("X-API-Version", "2");
Now I am getting all the info including Uri data coming through !
Thanks for all responses.
I have been trying to follow the steps in this document https://learn.microsoft.com/en-us/azure/data-factory/v1/data-factory-use-custom-activities
to create a custom activity, but it throw run timed out error in output dataset:
Pipeline:
{
"name": "ADFTutorialPipelineCustom",
"properties": {
"description": "Use custom activity",
"activities": [
{
"type": "DotNetActivity",
"typeProperties": {
"assemblyName": "MyDotNetActivity.dll",
"entryPoint": "MyDotNetActivityNS.MyDotNetActivity",
"packageLinkedService": "Destination-BlobStorage",
"packageFile": "blobcontainer/MyDotNetActivity.zip",
"extendedProperties": {
"SliceStart": "$$Text.Format('{0:yyyyMMddHH-mm}', Time.AddMinutes(SliceStart, 0))"
}
},
"outputs": [
{
"name": "OutputDataset"
}
],
"policy": {
"timeout": "00:30:00",
"concurrency": 1,
"retry": 3
},
"scheduler": {
"frequency": "Minute",
"interval": 30
},
"name": "MyDotNetActivity",
"linkedServiceName": "AzureBatchAccount"
}
],
"start": "2018-01-09T06:00:00Z",
"end": "2018-01-11T07:00:00Z",
"isPaused": false,
"hubName": "test2017dec11_hub",
"pipelineMode": "Scheduled"
}
Output Dataset:
{
"name": "OutputDataset",
"properties": {
"published": false,
"type": "AzureBlob",
"linkedServiceName": "Destination-BlobStorage",
"typeProperties": {
"fileName": "{slice}.txt",
"folderPath": "blobcontainer",
"partitionedBy": [
{
"name": "slice",
"value": {
"type": "DateTime",
"date": "SliceStart",
"format": "yyyy-MM-dd-HH"
}
}
]
},
"availability": {
"frequency": "Minute",
"interval": 30
}
}
Activity code:
public class MyDotNetActivity : IDotNetActivity
{
public IDictionary<string, string> Execute(IEnumerable<LinkedService> linkedServices, IEnumerable<Dataset> datasets, Activity activity, IActivityLogger logger)
{
logger.Write("Activity start");
AzureStorageLinkedService outputLinkedService;
Dataset outputDataset = datasets.Single(dataset => dataset.Name == activity.Outputs.Single().Name);
outputLinkedService = linkedServices.First(
linkedService =>
linkedService.Name ==
outputDataset.Properties.LinkedServiceName).Properties.TypeProperties
as AzureStorageLinkedService;
string connectionString = outputLinkedService.ConnectionString;
string folderPath = GetFolderPath(outputDataset);
UploadFileToBlob(connectionString, folderPath, GetFileName(outputDataset), logger);
return new Dictionary<string, string>();
}
private static string GetFolderPath(Dataset dataArtifact)
{
if (dataArtifact == null || dataArtifact.Properties == null)
{
return null;
}
AzureBlobDataset blobDataset = dataArtifact.Properties.TypeProperties as AzureBlobDataset;
if (blobDataset == null)
{
return null;
}
return blobDataset.FolderPath;
}
public void UploadFileToBlob(string blobConnectionString, string blobFolderPath, string fileName, IActivityLogger logger)
{
logger.Write("connecting to the blob..");
var outputStorageAccount = CloudStorageAccount.Parse(blobConnectionString);
string output = string.Empty;
output += "test blob storage";
var outputBlobUri = new Uri(outputStorageAccount.BlobEndpoint, blobFolderPath + "/" + fileName);
var outputBlob = new CloudBlockBlob(outputBlobUri, outputStorageAccount.Credentials);
logger.Write("uploading to the blob URI: {0}", outputBlobUri.ToString());
outputBlob.UploadText(output);
logger.Write("upload succeeded");
}
}
Please advise how to solve this problem, thanks.
From what you've showing, your activity policy timeout is set to 30 minutes. To change this, edit existing activity policy timeout in your pipeline and set it to a bigger value. i.e.
"policy": {
"timeout": "02:00:00"
}
To do this, go to your Data Factory and click Author and Deploy action: http://prntscr.com/hxjr9n then click Pipelines http://prntscr.com/hxjrie and edit JSON of your relevant one as I wrote.