We are getting the below exception while reading data using JsonTextReader
Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
JsonTextReader jsonReader - parameter
while (hasRecords(jsonReader, JsonToken.StartObject, null, null)) //Row
{
...
//it's ok to read this all into memory - it's just one row's worth of data
JArray values = (JArray)JToken.ReadFrom(jsonReader);
Also, including the code for HttpPost implementation for better clarity
HttpClientHandler handler = new HttpClientHandler() { Credentials = taskProfileInfo.Credential };
HttpClient httpClient = new HttpClient(handler) { Timeout = TimeSpan.FromSeconds(taskProfileInfo.CommandTimeout) };
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
HttpResponseMessage response;
HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, url);
request.Content = new StringContent(postBody, Encoding.UTF8, "application/json");
response = await httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
response.EnsureSuccessStatusCode();
//using (var responseStream = await response.Content.ReadAsStreamAsync())
//{
// using (var reader = new StreamReader(responseStream))
// {
// responseFromAPI = reader.ReadToEnd();
// }
//}
return new JsonTextReader(new StreamReader(await response.Content.ReadAsStreamAsync()));
Appreciate if any one can help us ..
Edit: Please note that we are able to debug it locally and works fine. Only problem when we run this as Worker Role in Azure Cloud service.
I finally addressed this issue. Just to close this (might help someone) -
After doing remote debugging we found the below inner exception :
{"The request was aborted: The request was canceled."}
And, the root cause for this issue is that we did set the timeout to less than what the actual read (JsonTextReader) operation would take. The below line of code which sets the time out :
HttpClient httpClient = new HttpClient(handler) { Timeout = TimeSpan.FromSeconds(taskProfileInfo.CommandTimeout) };
So, the FIX is to increase time out value so that request will not be cancelled while reading data.
I fixed increasing the seconds in the web config, it's an alternative:
sessionState timeout="50"
Related
We are needing to write some software, that receives events one at a time, and we need to ingress them into ADX. We are struggling to understand how the Kusto Client is meant to be utilized.
public void SaveEvent(Object event)
{
var _kcsb = new KustoConnectionStringBuilder(Uri).WithAadApplicationKeyAuthentication(
applicationClientId: "{}",
applicationKey: "{}",
authority: TenantId);
using var ingestClient = KustoIngestFactory.CreateQueuedIngestClient(_kcsb);
//// Create your custom implementation of IRetryPolicy, which will affect how the ingest client handles retrying on transient failures
IRetryPolicy retryPolicy = new NoRetry();
//// This line sets the retry policy on the ingest client that will be enforced on every ingest call from here on
((IKustoQueuedIngestClient)ingestClient).QueueOptions.QueueRequestOptions.RetryPolicy = retryPolicy;
var ingestProperties = new KustoIngestionProperties(DatabaseName, TableName)
{
Format = DataSourceFormat.json,
IngestionMapping = new IngestionMapping { IngestionMappingKind = Kusto.Data.Ingestion.IngestionMappingKind.Json, IngestionMappingReference = MappingName }
};
// Build the stream
var stream = new MemoryStream();
using var streamWriter = new StreamWriter(stream: stream, encoding: Encoding.UTF8, bufferSize: 4096, leaveOpen: true);
using var jsonWriter = new JsonTextWriter(streamWriter);
packet.Id = DateTime.UtcNow.Ticks;
var serializer = new JsonSerializer();
serializer.Serialize(jsonWriter, event);
streamWriter.Flush();
stream.Seek(0, SeekOrigin.Begin);
// Tell the client to ingest this
await ingestClient.IngestFromStreamAsync(data, ingestProperties);
}
Now I have several concerns with this. We are calling this function 300 to 500 times a second. I believe the custom client has built in batching, but do we not then need to use a singleton instance of the custom client?
Next thing is that I am creating a steam per event and then calling ingerss. This feels wrong? is there no way I can setup the custom client etc, and then just enqueue each event into the custom client as we receiver them?
I have an Azure Durable Function that sends some HTTP request via VNet. Ones in a while the following method throws a SocketException:
public async Task<(HttpStatusCode statusCode, string Content)> PostAsync(string uri ,string json)
{
if (string.IsNullOrEmpty(uri))
{
throw new ArgumentException("message", nameof(uri));
}
var client = new HttpClient
{
BaseAddress = new Uri(uri)
};
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
var content = new StringContent(json, Encoding.UTF8, "application/json");
HttpResponseMessage response = await client.PostAsync("", content);
string responseContent = await response.Content.ReadAsStringAsync();
return (response.StatusCode, responseContent);
}
The requested address is not valid in its context
Is it a problem of VNet or how should I approach this problem?
This error is thrown at the TCP layer. A quick search shows that it most often happens when trying to bind to a IP address which isn't valid, like 0.0.0.0. I would start by looking at the URI value you're using to determine whether it corresponds to something you expect.
Here is the code:
private AccessTokenInfo GetToken()
{
WebRequest webRequest = WebRequest.Create("https://oxford-speech.cloudapp.net/token/issueToken");
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(_requestDetails);
webRequest.ContentLength = bytes.Length;
try
{
using (Stream outputStream = webRequest.GetRequestStream())
{
outputStream.Write(bytes, 0, bytes.Length);
}
// ...
I have got the exception:
the underlying connection was closed could not establish trust relationship
How can I fit it ?
I hope I'm not missing something here...
The URL you're using isn't the one that generates tokens for the Text-to-Speech API as documented here. (The "Oxford" that's referenced in your URL refers to the Project Oxford which Cognitive Services was formerly known as.)
Also, WebRequest is deprecated. Use the System.Net.Http package instead.
The code to invoke the new REST endpoint then would look something like:
using (var client = new HttpClient())
using (var request = new HttpRequestMessage(HttpMethod.Post, "https://api.cognitive.microsoft.com/sts/v1.0/issueToken"))
{
request.Headers.Add("Ocp-Apim-Subscription-Key", "YOUR-KEY-HERE");
var response = await client.SendAsync(req);
var token = await response.Content.ReadAsStringAsync();
}
Finally, there are several client libraries that may get you around from writing any code to hit the REST services at all.
I'm a beginner when it comes to HTTP connections.
Currently i'm working with SAAJ api to have my soap based client, where to handle timeouts i ended up using URLStreamHandler for HTTP connection properties with the endpoints.
Problem is that this timeout works for my windows based system, however it isn't working for the Linux server it is going to go live on.
below is the code for fetching endpoint with set properties. It is a HTTP POST connection.
URL endpoint = new URL (null, url, new URLStreamHandler () {
protected URLConnection openConnection (URL url) throws IOException {
// The url is the parent of this stream handler, so must create clone
URL clone = new URL (url.toString ());
HttpURLConnection connection = (HttpURLConnection) clone.openConnection();
connection.setRequestProperty("Content-Type",
"text/xml");
connection.setRequestProperty("Accept",
"application/soap+xml, text/*");
// If we cast to HttpURLConnection, we can set redirects
// connection.setInstanceFollowRedirects (false);
connection.setDoOutput(true);
connection.setConnectTimeout(3 * 1000);
connection.setReadTimeout(3 * 1000);
return connection;
for the SAAJ API part, below is the implementation, pretty basic one
SOAPConnectionFactory soapConnectionFactory = SOAPConnectionFactory
.newInstance();
soapConnection = soapConnectionFactory.createConnection();
is = new ByteArrayInputStream(command.getBytes());
SOAPMessage request = MessageFactory.newInstance(
SOAPConstants.SOAP_1_1_PROTOCOL).createMessage(
new MimeHeaders(), is);
MimeHeaders headers = request.getMimeHeaders();
headers.addHeader("Content-Type", "text/xml");
request.saveChanges();
ByteArrayOutputStream out = new ByteArrayOutputStream();
request.writeTo(out);
soapResponse = soapConnection.call(request, endpoint);
Is it that system properties would affect connect or read timeout. If so please let me know what could cause this behavior.
I want to show message to end user when blob is taking so much time for uploading and downloading. I found useful blog here.
Simply linear retry policy
public static RetryPolicy LinearRetry(int retryCount, TimeSpan intervalBetweenRetries)
{
return () =>
{
return (int currentRetryCount, Exception lastException, out TimeSpan retryInterval) =>
{
// Do custom work here
// Set backoff
retryInterval = intervalBetweenRetries;
// Decide if we should retry, return bool
return currentRetryCount < retryCount;
};
};
}
But here I didn't get how to send response to user back while retrying. Is this right way or anything else. Please suggest
OperationContext class in Storage Client Library has an event called Retrying that you can consume and send message back to the client.
For example, I created a simple console application which tries to create a blob container. When I ran this application, I deliberately turned off Internet access so that I can simulate a situation where operation would be retried. Then in this event consumer, I simply write something back to console. You could simply raise another event from there that would send a message back to your client.
var requestOptions = new BlobRequestOptions()
{
RetryPolicy = new ExponentialRetry(),
};
var operationContext = new OperationContext();
operationContext.Retrying += (sender, args) =>
{
Console.WriteLine("I'm retrying ....");
};
var cloudStorageAccount = new CloudStorageAccount(new StorageCredentials(accountName, accountKey), true);
var blobClient = cloudStorageAccount.CreateCloudBlobClient();
var container = blobClient.GetContainerReference("test");
container.CreateIfNotExists(requestOptions, operationContext);