I am launching requests to a web service using Groovy/XmlSlurper:
new XmlSlurper().parse("http://en.wikipedia.org/w/api.php?action=...")
I would like to set the user agent to let the website know what project the requests are from.
How to set the user agent?
XMLSlurper's constructors do not have anything like this.
The setProperty method sounds interesting but Google does not lead to anything related to UA.
I would rather avoid changing the code to abandon XmlSlurper in favor of another library.
Looks like Groovy uses the JVM's default SAX parser to connect to the input stream and perform the initial parsing. I saw this answer on SO that might help - all you need to do is set this before your call to the slurper to make it happen:
System.setProperty("http.agent", "my-agent-name");
P.S. This is just a guess - hope it helps.
I see p.3, btw there is also Groovy Http Builder, it uses JsonSlurper for JSON results, but it's much more flexible for making http requests - http://groovy.codehaus.org/HTTP+Builder
Related
I need to set up an application in Azure and make communicate 2 functions (one written in C# and one written in JavaScript).
The C# fragment consists in analyzing a XML feed, get the data and save in objects then finally send them to the other JavaScript function by parameter.
I did read that we could establish communication between both functions using HTTP calls but is it possible to do it with parameters ?
If not, would have any suggestions in order to achieve something like this properly? I'm getting started with Azure and i don't have enough visibility to know what is recommened in such a situation
Thank you for your advices
Yes, this is absolutely possible. How you do this is up to you. If you look at the default HTTP trigger templates, you can see that they take parameters (for example, as query string parameters). You can find more examples in the HTTP and webhook recipes documentation.
You can use other trigger types for cross-function communication as well. Take a look at this documentation for related best practices: https://learn.microsoft.com/en-us/azure/azure-functions/functions-best-practices#cross-function-communication
I'm fairly new to programming and this question is about making sure I get the HTTP protocol correctly. My issue is that when I read about HTTP request/response, it looks like it needs to be in a very specific format with a status code, HTTP version number, headers, a blank line followed by the body.
However, after creating a web app with nodejs/express, I never once had to actually write code that made an HTTP response in this format (I'm assuming, although I don't know for sure that other frameworks like ruby on rails or python/Django are the same). In the express app, I just set up the route handlers to render the appropriate pages, when a request was made to that route.
Is this because express is actually putting the response in the correct HTTP format behind the scenes? In other words, if I looked at the expressJS code, would there be something in that code that actually makes an HTTP response in the HTTP format?
My confusion is that, it seems like the HTTP request/response format is so important but somehow I never had to write any code dealing with it for a node/express application. Maybe this is the entire point of a framework like express... to take out the details so that developers can deal with business logic. And if that is correct, does anyone ever write web apps without a framework to do this. Would you then be responsible for writing code that puts the server's response into the exact HTTP format?
I'm fairly new to programming and this question is about making sure I get the HTTP protocol correctly. My issue is that when I read about HTTP request/response, it looks like it needs to be in a very specific format with a status code, HTTP version number, headers, a blank line followed by the body.
Just to give you an idea, there are probably hundreds of specifications that have something to do with the HTTP protocol. They deal with not only the protocol itself, but also with the data format/encoding for everything you send including headers and all the various content types you can send, authentication schemes, caching, status codes, URL decoding, etc.... You can see some of the specifications involved just by looking here: https://www.w3.org/Protocols/.
Now a simple request and a simple text response could get away with only knowing a few of these specifications, but life is not always that simple.
Is this because express is actually putting the response in the correct HTTP format behind the scenes? In other words, if I looked at the expressJS code, would there be something in that code that actually makes an HTTP response in the HTTP format?
Yes, there would. A combination of Express and the HTTP library that is built into node.js handle all the details of the specification for you. That's the advantage of using a library/framework. They even handle different versions of the protocol and feedback from thousands of other developers have helped them to clean up edge case bugs. A good library/framework allows you to still control any detail about the response (headers, content types, status codes, etc..) without making you have to go through the detail work of actually creating the exact response. This is a good thing. It lets you write code faster and lets you ride on the shoulders of others who have already figured out minutiae details that have nothing to do with the logic of your app.
In fact, one could say the same about the TCP protocol below the HTTP protocol. No regular app developer wants to write their own TCP stack. Instead, you just want a working TCP stack that you can use that's already been tuned and debugged for you.
However, after creating a web app with nodejs/express, I never once had to actually write code that made an HTTP response in this format (I'm assuming, although I don't know for sure that other frameworks like ruby on rails or python/Django are the same). In the express app, I just set up the route handlers to render the appropriate pages, when a request was made to that route.
Yes, this is a good thing. The framework did the detail work for you. You just call res.setHeader(), res.status(), res.cookie(), res.send(), res.json(), etc... and Express makes the entire response for you.
And if that is correct, does anyone ever write web apps without a framework to do this. Would you then be responsible for writing code that puts the server's response into the exact HTTP format?
If you didn't use a framework or library of any kind and were programming at the raw TCP level, then yes you would be responsible for all the details of the HTTP protocol. But, hardly anybody other than library developers ever does this because frankly it's just a waste of time. Every single platform has at least one open source library that does this already and even if you were working on a brand new platform, you could go get an open source body of code and port it to your platform much quicker than you could write all this yourself.
Keep in mind that one of the HUGE advantages of node.js is that there's an enormous body of open source code (mostly in NPM and Github) already prepackaged to work with node.js. And, because node.js is server-side where code memory isn't usually tight and where code just comes from the local hard disk at server init time, there's little downside to grabbing a working and tested package that does what you already need, even if you're only going to use 5% of the functionality in the package. Or, worst case, clone an existing repository and modify it to perfectly suit your needs.
Is this because express is actually putting the response in the
correct HTTP format behind the scenes?
Yes, exactly, HTTP is so ubiquitous that almost all programming languages / frameworks handle the actual writing and parsing of HTTP behind the scenes.
Does anyone ever write web apps without a framework to do this. Would
you then be responsible for writing code that puts the server's
response into the exact HTTP format?
Never (unless you're writing code that needs very low level tweaking of HTTP code or something)
Due to some limitations about the web services I am proxying, I have to inject some JS code so that it allows the iframe to access the parent window and perform some actions.
I have built a proxy system with node-http-proxy which works pretty nicely. However I have spent unmeasurable hours trying to modify the content (on my own, using harmon as well, etc) that is being sent to the user without any success. I have found some articles and even some questions here but all of them are outdated and are not useful anymore.
I was wondering if someone can give me an actual example about how to do this, because I am unable to do it and maybe it is just that it is impossible to do at this point?
I haven't tried harmon, but I did try cheerio and it works.
However, I used http-mitm-proxy and not node-http-proxy.
If you are using http-mitm-proxy, you need to return a promise in the response handler. Otherwise, the proxy continues to send the original response without picking up your changes.
I have recently written another proxy at:
https://github.com/noeltimothy/noelsproxy
I'm going to add response handling to this soon. This one uses a callback mechanism, which means it wont return the response until the caller signals it to.
You should be able to use 'cheerio' and alter the content in JQuery style.
I am tring to do the following:
I want a SIP User Agent to perform the following steps on receiving an inbound call (call set up request).
1) Read the caller ID from the SIP request and Log the details to file
2) Drop the call (terminate the call without picking up the call)
I have not been able to find a high level api that will let me script this interaction. I have taken a look at Jain but it seems to be a very low level API and I imagine will require a lot of work to get the above interaction coded up and working. Can anyone suggest an apropriate API to implement the above.
NOTE: I have tried ROXEO.com and their CCXML based apps are great but their pricing is aimed at big companies, so Voxeo is not an option.
There are quite a few open source SIP stacks around two examples of many are pjsip and sipsorcery (as a disclaimer I do some dev work on the latter). It will all depend on your language and prefeences as to which one suits. There are also lots of SIP tools around that may be a more efficient approach for you such as SIPp.
Apart from those options and given your very simple requirements you could probably get away with 20 or 30 lines of code that listens on a UDP socket, parses the incoming INVITE to extract the From header and then sends back a rejection response by changing the top line of the request to make it a response and sending it back to where it came from.
If you're using C, try eXosip, you could easily whatever you want.
Here
It's clear that Jain SIP could be quite painful (actually all the configuration but the API otherwise is quite high-level, to manipulate messages) , but you can take the jain-sip-presence-proxy and removes almost everything from their INVITE handler and build your own message
if you're using java, you can use peers which provides a high level api in package net.sourceforge.peers.sip.core.useragent. The entry point is UserAgent class, take a look at gui package if you want to see how it is used. Traces are in log files so you can track calls.
ivrworx but it can handle one scenarion at a time only
Asterisk pbx can act as a simple sip client, and do just that, however if you wante to integrate something in your own solution, take a look at: http://sipsimpleclient.org/projects/sipsimpleclient/wiki/SipMiddlewareApi
I’m looking for tools for interactively inspecting HTTP servers by manually constructing requests (and viewing responses), under GNU/Linux. Something that would let me quickly specify standard header fields, make a form request body, etc. (netcat doesn’t really excel at this.)
Any suggestions?
Maybe simply cURL is what you need?
A Python script with urllib2 would seem appropriate. You can manipulate headers at will. Of course you have access to all the request/response fields too. A tutorial can be found here.
If all you are looking to do is to make manual HTTP requests to interact with web services for testing / reference purposes, then there is a really nice add-on for Firefox that does just that. It is called Poster.