I am using 3scale to manage the front end of any API. I want to include the API in the header of a request instead of the URL string (seems a little neater)
At the moment it gets defined as such.
set req.http.X-3scale-app_id = regsub(req.url,".*[&?](app_id.[^&]*).*","\1");
set req.http.X-3scale-app_key = regsub(req.url,".*[&?](app_key.[^&]*).*","\1");
set req.http.X-3scale-user_key = regsub(req.url,".*[&?](user_key.[^&]*).*","\1");
set req.http.X-3scale-user_id = regsub(req.url,".*[&?](user_id.[^&]*).*","\1");
if (req.http.X-3scale-app_id != req.url) {
set req.http.X-3scale-authrep = req.http.X-3scale-authrep + "&" + req.http.X-3scale-app_id;
}
if (req.http.X-3scale-app_key != req.url ) {
set req.http.X-3scale-authrep = req.http.X-3scale-authrep + "&" + req.http.X-3scale-app_key;
}
if (req.http.X-3scale-user_id != req.url) {
set req.http.X-3scale-authrep = req.http.X-3scale-authrep + "&" + req.http.X-3scale-user_id;
}
if (req.http.X-3scale-user_key != req.url) {
set req.http.X-3scale-authrep = req.http.X-3scale-authrep + "&" + req.http.X-3scale-user_key;
}
unset req.http.X-3scale-app_id;
unset req.http.X-3scale-app_key;
unset req.http.X-3scale-user_id;
unset req.http.X-3scale-user_key;
What I would like to do is take the key from the header because i think it seems better, so I could do something like...
set req.http.X-3scale-app_key = req.http.x-appid
However I can not get this to work! I am missing something
You can try to use get method from vmod header with something like this
set req.X-3scale-app_id = header.get(req.http.SomeOtherHeader,".*");
On client side you should set SomeOtherHeader in plaintext
But why you not set this X-3scale-app_id directly on client side?
Related
I want to invalidate a request in Varnish from a java backend with HTTP headers.
till now I am able to achieve cache arequest which does not have query param in it.
Let's say I have a request: localhost:8090/api/data/abc?fields=test,test1
what headers do I need to set in this case for varnish to cache it.
I am able to ban a request which is like : localhost:8090/api/data/abc
by using this headers for this request:
request: localhost:8090/api/data/abc
headers:
responseHeaders.set("x-host", "localhost:8080");
responseHeaders.set("x-url", "/api/data/abc");
VCL code
You can use the VCL example from the banning tutorial on the Varnish Developer Portal, which looks like this:
vcl 4.1;
acl purge {
"localhost";
"192.168.55.0"/24;
}
sub vcl_recv {
if (req.method == "BAN") {
if (!client.ip ~ purge) {
return (synth(405));
}
if (!req.http.x-invalidate-pattern) {
return (purge);
}
ban("obj.http.x-url ~ " + req.http.x-invalidate-pattern
+ " && obj.http.x-host == " + req.http.host);
return (synth(200,"Ban added"));
}
}
sub vcl_backend_response {
set beresp.http.x-url = bereq.url;
set beresp.http.x-host = bereq.http.host;
}
sub vcl_deliver {
unset resp.http.x-url;
unset resp.http.x-host;
}
Keep in mind that you need to add your backend definition and customize the values of the ACL.
Removing objects that match a specific query string pattern
Let's say the cache contains 4 objects identified by the following URL:
/?a=1
/?a=2
/?a=3
/?a=4
Imagine we want to remove the first 3. Assuming the value of the a query string parameter is a number, we can create the following HTTP request:
curl -XBAN -H"x-invalidate-pattern: ^/\?a=[1-3]+" http://localhost
As a result /?a=1, /?a=2 and /?a=3 will be removed from the cache, whereas /?a=4 is still stored in the cache.
Conclusion
The query string parameter is part of the URL. As long as you can match it in a regular expression, you can remove specific objects from the cache.
The file-name of any image is appearing like
/de-de/medias/sys_master/images/images/h9c/h5f/8796178743326/8796178743326.jpg in the url.
Instead of 8796178743326.jpg there should be file-name.jpg
I have already set media.legacy.prettyURL=true
8796178743326 is the PK of the image.
Any help!
With the prettyURL, if there is no realfilename value in media instance then URL will end with PK instead real file name.
/medias/sys_master/images/images/h9c/h5f/8796178743326/8796178743326.jpg
If you really want the file name in the URL then you have to edit respective media from the backoffice/impex and assign value to the realFileName attribute.
Have a look into assembleLegacyURL method of LocalMediaWebURLStrategy class
String realFileName = this.getRealFileNameForMedia(mediaSource);
if (realFileName == null) {
basePath = mediaSource.getLocation().substring(0, lastDotIdx);
lastDotIndexForRealFileName = StringUtils.lastIndexOf(basePath, '/');
String fileName = basePath.substring(lastDotIndexForRealFileName + 1);
sb.append(basePath).append("/").append(fileName).append('.').append(fileExtension);
} else {
basePath = location.substring(0, lastDotIdx);
lastDotIndexForRealFileName = realFileName.lastIndexOf(46);
if (lastDotIndexForRealFileName != -1) {
realFileName = realFileName.substring(0, lastDotIndexForRealFileName);
}
sb.append(basePath).append("/").append(realFileName).append('.').append(fileExtension);
}
I'm using LokiJS and it's fine for most parts. Except that I can't seem to delete a record from the database.
The find gets me the right record to delete, but neither remove() not findAndRemove() remove it after saveDatabase. What am I doing wrong?
(Note: I've tried saveDatabase outside the loadDatabase(...) callback - no change. whenever I re-run it, the record I thought I deleted is still there)
Here's the code: dbName is a valid path to the file
var db = new loki(dbName);
db.loadDatabase({}, function (e) {
var orders = db.getCollection("orders");
console.log(orders.count() + " orders found... iterating");
orders.find().forEach((v, i) => {
console.log(i + "." + v.key + " " + v.places.display + " " + v.theme + " " + moment(v.date).format("DD MMM YYYY HH:mm"));
});
}
let dKey = "some key that is valid";
if (!dKey) {
console.log("no key specified, returning");
return;
}
console.log("deleting specific record " + dKey);
let dObj = orders.find({ key: dKey });
if (!dObj || dObj.length == 0) {
console.log("no such record, returning");
return;
}
console.log(dObj[0]);
orders.remove(dObj[0]);
db.saveDatabase((e) => {
console.log("save complete");
});
}
});
Try: orders.chain().find({ key: dKey }).remove()
I faced the same issue while working with lokijs, so putting it here so that it can help anyone else looking for the same issue in future.
I was trying to lookup a record in db with a particular $loki value, but I was getting an empty ResultSet. Then I tried this.
data.chain().find({"$loki":{$aeq:serial}}).remove().data().
Here, data is my db instance and serial is the $loki value I was looking for and passing to find().
It seems like lokijs stores them as strings thats why $aeq worked and I was able to find and delete the required record
I have a WSDL which has multiple operations. For each op i want a template .xml with its response and request.
I know how to do this manually in soapUI but I would like to generate them using a groovy script.
I googled a lot already, but seems I'm the only one who is looking for this.
My service has 16 Operations, so to do this manual would be too much time. Since the service gets updates every 2 months, an automation using a test step would be perfect.
I managed to do it for the requests already:
right-click on ´services´ in left tree, ´Generate Test Suite´, ´Single Test Case with one Request for each Operation´
then I loop through those Test Step Requests and store them on my disk.
import com.eviware.soapui.impl.wsdl.teststeps.*
for( testCase in testRunner.testCase.testSuite.getTestCaseList() )
{
for( testStep in testCase.getTestStepList() )
{
if( testStep instanceof WsdlTestRequestStep )
{
log.info "operation name: " +testStep.getName()
// create file name
Date startTime = new Date();
def cur_Time = startTime.getMonth() + "_" + startTime.getDate();
cur_Time = cur_Time + "_" + startTime.getHours() + startTime.getMinutes() +startTime.getSeconds()
def fileName = testStep.getName() + "_" + cur_Time
def inputFileRequest = new File("T:\\"+ "Request_" + fileName+".txt")
def inputFileResponse = new File("T:\\"+ "Response_" + fileName+".txt")
// write request to file
inputFileRequest.write(testStep.getProperty("request").value)
}
}
}
But I havent figured out a way to do this also for the resposes.
If i use getProperty("reponse") it's null of course.
Any hint? :)
and the winner is, I figured it out myself:
map = context.testCase.testSuite.project.interfaces["services"].operations
for (entry in map)
{
opName = entry.getKey()
inputFileRequest = new File("T:\\" + opName + "Request.xml")
inputFileResponse = new File("T:\\" + opName + "Response.xml")
inputFileRequest.write(entry.getValue().createRequest(true))
inputFileResponse.write(entry.getValue().createResponse(true))
}
This is great , even I am also working on the same. As of now I am taking xml request from a folder but I just want to get the Request from WSDL itself and want to get it's parameter.
try{
//Hitting the WSDLs one by one
wsdlList.each
{
wsdl ->
wsdlToHit=wsdl
log.info("WSDL To Hit :" + wsdlToHit)
// Creating an interface
log.info("Before Interface Creation")
iface= WsdlInterfaceFactory.importWsdl( project,wsdl, false )[0]
//iface= WsdlInterfaceFactory.importWsdl( project,WSDLFile, false )[0]
log.info("After Interface Creation")
if(Operation == "xyz")
{
requestXML= requestXML1
responseActual= responseActual1
expectedActual=expectedActual1
}
if(Operation == "abc")
{
requestXML= requestXML2
responseActual= responseActual2
expectedActual=expectedActual2
}
requestXML.each
{
request1 ->
def wsdlReqDir=request1
log.info("RequestLocation : " + wsdlReqDir)
File fl = new File(wsdlReqDir)
File[] wsdlDirFiles = fl.listFiles()
log.info("XML Files in Request Folder : " + wsdlDirFiles)
if(wsdlDirFiles.size()>0)
{
wsdlDirFiles.each
{
wsdlFile->
log.info("Request XML file to Send :" + wsdlFile)
//Calling the function to hit the service
sendRequest(wsdlFile,iface,Operation,Report_File_LOC,requestXML,responseActual,propData)
reportFilewriter.flush()
}
}
}
//removing Interface created
removeInterface(wsdl)
log.info("Removed iface : " + wsdl)
reportFilewriter.flush()
}
Thanks,
Hanumant
I'm using Varnish 2.0.6 and I'm having trouble with finding good documentation to write the vcl_hash function.
I need to remove a few parameters from the URL of my API before caching. In particular a userid that is passed to track analytics but not to change the results.
URL: /api/browse?node=123&userid=3432432564363
I wrote this but it's not cleat to me if the vcl_hash function needs to end with 'hash' or 'return(hash)' or NOTHING and if I need to handle all the cases or just my special case. It's not clear to me if I'm overwriting method or I'm extending it.
I have:
sub vcl_hash {
if (req.url ~ "^/api/browse") {
set req.hash += regsuball(req.url,"&userid=([A-z0-9]+)","");
}
hash;
}
Is it missing something?
I tested a few things, and this one seems to work:
sub vcl_hash {
if (req.url ~ "^/api/browse") {
set req.hash += regsuball(req.url,"&userid=([A-z0-9]+)","");
} else {
set req.hash += req.url;
}
set req.hash += req.http.host;
hash;
}
So it looks like you also have to handle the default case when you rewrite vcl_hash.
The following is a general solution that works for me (starting from varnish v4), to remove several unwanted parameters.
The list of parameters can be extended easily, as long as the value-regex matches: The value regex matches all URL-safe characters, so it should match for all URL-encoded parameters.
sub vcl_hash {
# conditional replacement is faster then evaluating regexes all the time
if (req.method == "GET" || req.method == "HEAD") {
hash_data(regsuball(req.url, "(userid|sid|uid)=[%.-_~A-z0-9]+&?", ""));
}
else {
hash_data(req.url);
}
hash_data(req.http.host);
return (lookup);
}