I have a global requestfilter from where i want to log all http traffic using log4net - company restriction. Problem is that the InputStream is always lenght = 0. The soap envelope is desezerialized correctly and execution of service is succesfull, but inputstream is unavailable after first serialization. Is this wrong approach, if i want to log all ingoing and outgoing http traffic? What should i do to accomplish this? I do not want to log the deserialized requestDto.
this.RequestFilters.Add((httpReq, httpResp, requestDto) =>
{
LogManager.LogFactory.GetLogger(this.GetType()).Info(httpReq.GetRawBody());
});
Error seems to occur in type ServiceStack.WebHost.Endpoints.Support.SoapHandler in method, where using statement closes stream without buffering it:
protected static Message GetRequestMessage(Stream inputStream, MessageVersion msgVersion)
{
using (var sr = new StreamReader(inputStream))
{
var requestXml = sr.ReadToEnd();
var doc = new XmlDocument();
doc.LoadXml(requestXml);
var msg = Message.CreateMessage(new XmlNodeReader(doc), int.MaxValue,
msgVersion);
return msg;
}
}
When i try to access GetRawBody on type ServiceStack.WebHost.Endpoints.Extensions the following logic is executed:
public string GetRawBody()
{
if (bufferedStream != null)
{
return bufferedStream.ToArray().FromUtf8Bytes();
}
using (var reader = new StreamReader(InputStream))
{
return reader.ReadToEnd();
}
}
Here I would expect the inpustream to be buffered, since it Inputstreaem is no longer available (lenght = 0).
Isn't this a bug?
Have you seen Request Logger or IRequiresRequestStream. These might help provide some insight on logging requests. Also, I don't believe InputStream would have a length on GET requests.
Related
I am exploring Microsoft Computer Vision's Read API (asyncBatchAnalyze) for extracting text from images. I found some sample code on Microsoft site to extract text from images asynchronously.It works in following way:
1) Submit image to asyncBatchAnalyze API.
2) This API accepts the request and returns a URI.
3) We need to poll this URI to get the extracted data.
Is there any way in which we can trigger some notification (like publishing an notification in AWS SQS or similar service) when asyncBatchAnalyze is done with image analysis?
public class MicrosoftOCRAsyncReadText {
private static final String SUBSCRIPTION_KEY = “key”;
private static final String ENDPOINT = "https://computervision.cognitiveservices.azure.com";
private static final String URI_BASE = ENDPOINT + "/vision/v2.1/read/core/asyncBatchAnalyze";
public static void main(String[] args) {
CloseableHttpClient httpTextClient = HttpClientBuilder.create().build();
CloseableHttpClient httpResultClient = HttpClientBuilder.create().build();;
try {
URIBuilder builder = new URIBuilder(URI_BASE);
URI uri = builder.build();
HttpPost request = new HttpPost(uri);
request.setHeader("Content-Type", "application/octet-stream");
request.setHeader("Ocp-Apim-Subscription-Key", SUBSCRIPTION_KEY);
String image = "/Users/xxxxx/Documents/img1.jpg";
File file = new File(image);
FileEntity reqEntity = new FileEntity(file);
request.setEntity(reqEntity);
HttpResponse response = httpTextClient.execute(request);
if (response.getStatusLine().getStatusCode() != 202) {
HttpEntity entity = response.getEntity();
String jsonString = EntityUtils.toString(entity);
JSONObject json = new JSONObject(jsonString);
System.out.println("Error:\n");
System.out.println(json.toString(2));
return;
}
String operationLocation = null;
Header[] responseHeaders = response.getAllHeaders();
for (Header header : responseHeaders) {
if (header.getName().equals("Operation-Location")) {
operationLocation = header.getValue();
break;
}
}
if (operationLocation == null) {
System.out.println("\nError retrieving Operation-Location.\nExiting.");
System.exit(1);
}
/* Wait for asyncBatchAnalyze to complete. In place of this wait, can we trigger any notification from Computer Vision when the extract text operation is complete?
*/
Thread.sleep(5000);
// Call the second REST API method and get the response.
HttpGet resultRequest = new HttpGet(operationLocation);
resultRequest.setHeader("Ocp-Apim-Subscription-Key", SUBSCRIPTION_KEY);
HttpResponse resultResponse = httpResultClient.execute(resultRequest);
HttpEntity responseEntity = resultResponse.getEntity();
if (responseEntity != null) {
String jsonString = EntityUtils.toString(responseEntity);
JSONObject json = new JSONObject(jsonString);
System.out.println(json.toString(2));
}
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
}
There is no notification / webhook mechanism on those asynchronous operations.
The only thing that I can see right know is to change the implementation you mentioned by using a while condition which is checking regularly if the result is there or not (and a mechanism to cancel waiting - based on maximum waiting time or number of retries).
See sample in Microsoft docs here, especially this part:
// If the first REST API method completes successfully, the second
// REST API method retrieves the text written in the image.
//
// Note: The response may not be immediately available. Text
// recognition is an asynchronous operation that can take a variable
// amount of time depending on the length of the text.
// You may need to wait or retry this operation.
//
// This example checks once per second for ten seconds.
string contentString;
int i = 0;
do
{
System.Threading.Thread.Sleep(1000);
response = await client.GetAsync(operationLocation);
contentString = await response.Content.ReadAsStringAsync();
++i;
}
while (i < 10 && contentString.IndexOf("\"status\":\"Succeeded\"") == -1);
if (i == 10 && contentString.IndexOf("\"status\":\"Succeeded\"") == -1)
{
Console.WriteLine("\nTimeout error.\n");
return;
}
// Display the JSON response.
Console.WriteLine("\nResponse:\n\n{0}\n",
JToken.Parse(contentString).ToString());
I'm trying to use the c# library to download a file from an FTP. The code we are using is straight forward.
static void Main(string[] args)
{
Connect(true, true, true);
}
private static void Connect(bool keepAlive, bool useBinary, bool usePassive)
{
string RemoteFtpPath = "ftp://ftp.xxxx.ac.uk/incoming/testExtractCSVcoursesContacts.csv";
const string Username = "anonymous";
const string Password = "anonymous#xxxx.ac.uk";
var request = (FtpWebRequest)WebRequest.Create(new Uri(RemoteFtpPath));
request.Method = WebRequestMethods.Ftp.DownloadFile;
request.KeepAlive = keepAlive;
request.UsePassive = usePassive;
request.UseBinary = useBinary;
request.Credentials = new NetworkCredential(Username, Password);
request.Timeout = 30000;
try
{
var response = (FtpWebResponse)request.GetResponse();
var responseStream = response.GetResponseStream();
var reader = new StreamReader(responseStream);
var fileString = reader.ReadToEnd();
Console.WriteLine(
$"Success! keepAlive={keepAlive}, useBinary={useBinary}, usePassive={usePassive} Length={fileString.Length}");
reader.Close();
response.Close();
}
catch (Exception e)
{
Console.WriteLine(
$"Failed! keepAlive={keepAlive}, useBinary={useBinary}, usePassive={usePassive}, message={e.Message}");
}
}
`
we also tried to set passive = true with identical results.
When we run it, using wireshark we are getting : Wireshark log c#
Now we tried the same with Python and it's working just fine:
import urllib.request
data = urllib.request.urlretrieve('path')
print(data)
the wireshark log looks quite different:
So tried different things, but not able to sort this out.
Some ftp servers don't support OPTS UTF8 but still transmit file names in UTF8. (Note that 'OPTs UTF8' is NOT required by the FTP Internationalization Standard, although supporting UTF8 file names is.) The .NET Ftp classes will use the default code page if they don't get an OK response to OPTS UTF8... It's unfortunate that MS didn't provide some way to use UTF8 anyway, since this leaves you unable to transmit international file names to and from otherwise UTF8-compliant servers.
The issue is sorted after using a different library as FtpWebRequest doesn't support it
In our company we are using IBM MQ server for message queueing. For the same queue we are putting both String and Object type data message with the help of methods WriteObject and WriteString.
The challenge is occured when it comes to consumption(read message) of queue. Since the Get data can be both Object or String we need to decide which method to use ReadString or ReadObject on the MQMessage instance. I made workaround like firstly using ReadObject method, if an exception occured then try it with ReadString.
I did not like this workaround, is there any way to identify message data type after calling MQQueue instance's Get method?
Here below you can find my workaround:
public object GetMessage(string queueName) {
MQQueueManager queueManager = new MQQueueManager("queueManagerName", "channel", "connection");
MQGetMessageOptions queueGetMessageOptions = new MQGetMessageOptions();
queueGetMessageOptions.Options = MQC.MQGMO_WAIT + MQC.MQGMO_FAIL_IF_QUIESCING + MQC.MQPMO_SYNCPOINT;
try {
queueRead = queueManager.AccessQueue(queueName, MQC.MQOO_INPUT_AS_Q_DEF + MQC.MQOO_FAIL_IF_QUIESCING);
queueMessage = new MQMessage();
queueMessage.Format = MQC.MQFMT_STRING;
queueRead.Get(queueMessage, queueGetMessageOptions);
try {
var readObject = queueMessage.ReadObject();
return readObject;
} catch (SerializationException) { } // if message in queue not a object
queueMessage.DataOffset = 0;
var stringMsg = queueMessage.ReadString(queueMessage.MessageLength);
return stringMsg;
} catch (MQException exp) {
if (exp.ReasonCode != 2033) {
log.ErrorFormat("MQException: ResonCode: {0}, {1}", exp.ReasonCode, exp.Message);
}
}
return "";
}
What a horrible design.
First off, why are you putting 2 different message types into the same queue? Bad, very bad idea. You should be using 2 different queues. What, does someone think queues are scarce or rare? You should be handing out queues like candy.
Secondly, if you really need to go with this design then you should read the MQ Knowledge Center on MQMD structure. It contains a field called 'Message Type'. Most applications use 'Message Type' to contain either 'MQMT_DATAGRAM' or 'MQMT_REQUEST' but you can set your own values starting with 'MQMT_APPL_FIRST'.
So, define a couple of constants:
public const int MY_MSG_OBJECT = MQC.MQMT_APPL_FIRST + 1;
public const int MY_MSG_STRING = MQC.MQMT_APPL_FIRST + 2;
Hence, the sending application putting a string message would do:
MQMessage sendmsg = new MQMessage();
sendmsg.Format = MQC.MQFMT_STRING;
sendmsg.MessageType = MY_MSG_STRING;
sendmsg.WriteString("This is a test message");
queue.Put(sendmsg, pmo);
and the sending application putting an object message would do:
MQMessage sendmsg = new MQMessage();
sendmsg.Format = MQC.MQFMT_NONE;
sendmsg.MessageType = MY_MSG_OBJECT;
sendmsg.WriteObject(someObject);
queue.Put(sendmsg, pmo);
The receiving application would do:
MQMessage rcvmsg = new MQMessage();
queue.Get(rcvmsg, gmo);
// Check the Message Type
if (rcvmsg.MessageType == MY_MSG_STRING)
{
readString = queue.ReadString();
}
else if (rcvmsg.MessageType == MY_MSG_OBJECT)
{
readObject = queue.ReadObject();
}
else
{
System.Console.Out.WriteLine("Error: Unknown message type.");
}
When I declare a temporary reply queue to be exclusive (e.g. anonymous queue (exclusive=true, autodelete=true) in rpc-pattern), the response message cannot be posted to the specified reply queue (e.g. message.replyTo="amq.gen-Jg_tv8QYxtEQhq0tF30vAA") because RabbitMqProducer.PublishMessage() tries to redeclare the queue with different parameters (exclusive=false), which understandably results in an error.
Unfortunately, the erroneous call to channel.RegisterQueue(queueName) in RabbitMqProducer.PublishMessage() seems to nack the request message in the incoming queue so that, when ServiceStack.Messaging.MessageHandler.DefaultInExceptionHandler tries to acknowlege the request message (to remove it from the incoming queue), the message just stays on top of the incoming queue and gets processed all over again. This procedure repeats indefinitely and results in one dlq-message per iteration which slowly fills up the dlq.
I am wondering,
if ServiceStack handles the case, when ServiceStack.RabbitMq.RabbitMqProducer cannot declare the response queue, correctly
if ServiceStack.RabbitMq.RabbitMqProducer muss always declare the response queue before publishing the response
if it wouldn't be best to have some configuration flag to omit all exchange and queue declaration calls (outside of the first initialization). The RabbitMqProducer would just assume every queue/exchange to be properly set up and just publish the message.
(At the moment our client just declares its response queue to be exclusive=false and everything works fine. But I'd really like to use rabbitmq's built-in temporary queues.)
MQ-Client Code, requires simple "SayHello" service:
const string INQ_QUEUE_NAME = "mq:SayHello.inq";
const string EXCHANGE_NAME="mx.servicestack";
var factory = new ConnectionFactory() { HostName = "192.168.179.110" };
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
// Create temporary queue and setup bindings
// this works (because "mq:tmp:" stops RabbitMqProducer from redeclaring response queue)
string responseQueueName = "mq:tmp:SayHello_" + Guid.NewGuid().ToString() + ".inq";
channel.QueueDeclare(responseQueueName, false, false, true, null);
// this does NOT work (RabbitMqProducer tries to declare queue again => error):
//string responseQueueName = Guid.NewGuid().ToString() + ".inq";
//channel.QueueDeclare(responseQueueName, false, false, true, null);
// this does NOT work either (RabbitMqProducer tries to declare queue again => error)
//var responseQueueName = channel.QueueDeclare().QueueName;
// publish simple SayHello-Request to standard servicestack exchange ("mx.servicestack") with routing key "mq:SayHello.inq":
var props = channel.CreateBasicProperties();
props.ReplyTo = responseQueueName;
channel.BasicPublish(EXCHANGE_NAME, INQ_QUEUE_NAME, props, Encoding.UTF8.GetBytes("{\"ToName\": \"Chris\"}"));
// consume response from response queue
var consumer = new QueueingBasicConsumer(channel);
channel.BasicConsume(responseQueueName, true, consumer);
var ea = (BasicDeliverEventArgs)consumer.Queue.Dequeue();
// print result: should be "Hello, Chris!"
Console.WriteLine(Encoding.UTF8.GetString(ea.Body));
}
}
Everything seems to work fine when RabbitMqProducer does not try to declare the queues, like that:
public void PublishMessage(string exchange, string routingKey, IBasicProperties basicProperties, byte[] body)
{
const bool MustDeclareQueue = false; // new config parameter??
try
{
if (MustDeclareQueue && !Queues.Contains(routingKey))
{
Channel.RegisterQueueByName(routingKey);
Queues = new HashSet<string>(Queues) { routingKey };
}
Channel.BasicPublish(exchange, routingKey, basicProperties, body);
}
catch (OperationInterruptedException ex)
{
if (ex.Is404())
{
Channel.RegisterExchangeByName(exchange);
Channel.BasicPublish(exchange, routingKey, basicProperties, body);
}
throw;
}
}
The issue got adressed in servicestack's version v4.0.32 (fixed in this commit).
The RabbitMqProducer no longer tries to redeclare temporary queues and instead assumes that the reply queue already exist (which solves my problem.)
(The underlying cause of the infinite loop (wrong error handling while publishing response message) probably still exists.)
Edit: Example
The following basic mq-client (which does not use ServiceStackmq client and instead depends directly on rabbitmq's .net-library; it uses ServiceStack.Text for serialization though) can perform generic RPCs:
public class MqClient : IDisposable
{
ConnectionFactory factory = new ConnectionFactory()
{
HostName = "192.168.97.201",
UserName = "guest",
Password = "guest",
//VirtualHost = "test",
Port = AmqpTcpEndpoint.UseDefaultPort,
};
private IConnection connection;
private string exchangeName;
public MqClient(string defaultExchange)
{
this.exchangeName = defaultExchange;
this.connection = factory.CreateConnection();
}
public TResponse RpcCall<TResponse>(IReturn<TResponse> reqDto, string exchange = null)
{
using (var channel = connection.CreateModel())
{
string inq_queue_name = string.Format("mq:{0}.inq", reqDto.GetType().Name);
string responseQueueName = channel.QueueDeclare().QueueName;
var props = channel.CreateBasicProperties();
props.ReplyTo = responseQueueName;
var message = ServiceStack.Text.JsonSerializer.SerializeToString(reqDto);
channel.BasicPublish(exchange ?? this.exchangeName, inq_queue_name, props, UTF8Encoding.UTF8.GetBytes(message));
var consumer = new QueueingBasicConsumer(channel);
channel.BasicConsume(responseQueueName, true, consumer);
var ea = (BasicDeliverEventArgs)consumer.Queue.Dequeue();
//channel.BasicAck(ea.DeliveryTag, false);
string response = UTF8Encoding.UTF8.GetString(ea.Body);
string responseType = ea.BasicProperties.Type;
Console.WriteLine(" [x] New Message of Type '{1}' Received:{2}{0}", response, responseType, Environment.NewLine);
return ServiceStack.Text.JsonSerializer.DeserializeFromString<TResponse>(response);
}
}
~MqClient()
{
this.Dispose();
}
public void Dispose()
{
if (connection != null)
{
this.connection.Dispose();
this.connection = null;
}
}
}
Key points:
client declares anonymous queue (=with empty queue name) channel.QueueDeclare()
server generates queue and returns queue name (amq.gen*)
client adds queue name to message properties (props.ReplyTo = responseQueueName;)
ServiceStack automatically sends response to temporary queue
client picks up response and deserializes
It can be used like that:
using (var mqClient = new MqClient("mx.servicestack"))
{
var pingResponse = mqClient.RpcCall<PingResponse>(new Ping { });
}
Important: You've got to use servicestack version 4.0.32+.
I want to call Api function (1st) . from 2nd Api function using HttpClient. But I always get 404 Error.
1st Api Function (EndPoint : http : // localhost : xxxxx /api/Test/)
public HttpResponseMessage Put(int id, int accountId, byte[] content)
[...]
2nd Api function
public HttpResponseMessage Put(int id, int aid, byte[] filecontent)
{
WebRequestHandler handler = new WebRequestHandler()
{
AllowAutoRedirect = false,
UseProxy = false
};
using (HttpClient client = new HttpClient(handler))
{
client.BaseAddress = new Uri("http://localhost:xxxxx/");
// Add an Accept header for JSON format.
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
var param = new object[6];
param[0] = id;
param[1] = "/";
param[2] = "?aid=";
param[3] = aid;
param[4] = "&content=";
param[5] = filecontent;
using (HttpResponseMessage response = client.PutAsJsonAsync("api/Test/", param).Result)
{
return response.EnsureSuccessStatusCode();
}
}
}
So My question is that. Can I post Method Parameter as an object array from HttpClient as I did ? I don't want to Pass model as method parameter.
What is the wrong in my code ?
Unable to get any response , after change code to
return client.PutAsJsonAsync(uri, filecontent)
.ContinueWith<HttpResponseMessage>
(
task => task.Result.EnsureSuccessStatusCode()
);
OR
return client.PutAsJsonAsync(uri, filecontent)
.ContinueWith
(
task => task.Result.EnsureSuccessStatusCode()
);
As you probably found out, no you can't. When you call PostAsJsonAsync, the code will convert the parameter to JSON and send it in the request body. Your parameter is a JSON array which will look something like the array below:
[1,"/","?aid",345,"&content=","aGVsbG8gd29ybGQ="]
Which isn't what the first function is expecting (at least that's what I imagine, since you haven't showed the route info). There are a couple of problems here:
By default, parameters of type byte[] (reference types) are passed in the body of the request, not in the URI (unless you explicitly tag the parameter with the [FromUri] attribute).
The other parameters (again, based on my guess about your route) need to be part of the URI, not the body.
The code would look something like this:
var uri = "api/Test/" + id + "/?aid=" + aid;
using (HttpResponseMessage response = client.PutAsJsonAsync(uri, filecontent).Result)
{
return response.EnsureSuccessStatusCode();
}
Now, there's another potential issue with the code above. It's waiting on the network response (that's what happens when you access the .Result property in the Task<HttpResponseMessage> returned by PostAsJsonAsync. Depending on the environment, the worse that can happen is that it may deadlock (waiting on a thread in which the network response will arrive). In the best case this thread will be blocked for the duration of the network call, which is also bad. Consider using the asynchronous mode (awaiting the result, returning a Task<T> in your action) instead, like in the example below
public async Task<HttpResponseMessage> Put(int id, int aid, byte[] filecontent)
{
// ...
var uri = "api/Test/" + id + "/?aid=" + aid;
HttpResponseMessage response = await client.PutAsJsonAsync(uri, filecontent);
return response.EnsureSuccessStatusCode();
}
Or without the async / await keywords:
public Task<HttpResponseMessage> Put(int id, int aid, byte[] filecontent)
{
// ...
var uri = "api/Test/" + id + "/?aid=" + aid;
return client.PutAsJsonAsync(uri, filecontent).ContinueWith<HttpResponseMessage>(
task => task.Result.EnsureSuccessStatusCode());
}