ESI includes in JSON response - varnish

I'm struggling getting ESI includes to be replaced in a JSON context.
The following working correctly. Resulting in the ESI tag replaced by the actual contents.
<esi:include src="http://domain.com/esipath/" />
And this is not working. The ESI tag isn't replaced.
{
"test": "<esi:include src="http://domain.com/esipath/" />"
}
I have found some resources on google and a possible solution seems to set esi_syntax to 0x3 for varnish to skip checking for valid XML.
This doesn't seem to work on Varnish 4. When I start the deamon with the runtime param I get the following error.
Error: Unknown parameter "esi_syntax".
Is this setting deprecated in Varnish 4?
Any ideas to get this working?

indeed that parameter is depricated in varnish4. What you have to do is to activate the feature ESI_DISABLE_XML_CHECK.
For me I edit /etc/default/varnish (debian based system) and add
-p feature=+esi_disable_xml_check
to the DEAMON_OPTS parameters, for example:
DAEMON_OPTS="-a :6081 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s malloc,4GB \
-p feature=+esi_disable_xml_check"

On this diff we can see the removal of esi_syntax, and the new esi_disable_xml_check feature coming in for 0x3 old value.
I also found a reference on esi_disable_xml_check on the man.

Related

Varnish update from version 3 to 5 - parameters expiry_sleep and queue_max

In our company we want update Varnish 3.0 to 5.2 version. Some of parameters in DAEMON_OPTS (/etc/init.d/varnish) has been renamed but I can't find what happened with parameters:
expiry_sleep and queue_max
Please, how can I replace these parameters? Thank you very much! :)
DAEMON_OPTS="-a ${ADDRESS} \
-p connect_timeout=2\
-p sess_timeout=10\
-p syslog_cli_traffic=off\
-p expiry_sleep=0.01\
-p queue_max=300"
It should be like this:
DAEMON_OPTS="-a ${ADDRESS} \
-p connect_timeout=2\
-p sess_timeout=10\
-p syslog_cli_traffic=off\
-p timeout_linger=0.01\
-p thread_queue_limit=300"
Please consider reading the threading parameters as they have changed throughout the years.
varnishadm param.show -l | less

Reindex Solr in crafter delivey version 3

I am trying to reindex solr in one of our crafter delivery node with curl command
curl "http://{hostname}:{port}/api/1/target/deploy/{environment}/{siteName}" -X POST -H "Content-Type: text/json" -d '{ "reprocess_all_files": true }'
And changing the curl command URL as per our configuration but I am getting an error "{"message":"Content type 'text/json;charset=UTF-8' not supported"}"
Any suggestion will be much appreciated.
The content type seems to be wrong, it should be application/json, not text/json. The documentation seems to have an error.

curl command - could not resolve xn--x-5gn/post on Ubuntu

I am using curl command to call rest api. I want to post data and my curl command looks like:
curl –x POST -u 'username:PW' -k -H "Content-Type:application/json" -d '{"json-input":{"handler":"getContent","image":true,"video":false,"text":false,"source":"1","lage":"testlage1"}}' -i http://localhost:8080/com.knime.enterprise.server/rest/v4/jobs/3fd2ca61-c173-4160-a20d-45c387f65f64
I am getting following message:
curl: (6) Could not resolve host: xn--x-5gn curl: (6) Could not
resolve host: POST
The letter before X is wrong. It is supposed to be an ascii minus ('-', ascii code 0x2d / 45) and not the unicode dash character (U+2013) as used in the question.
curl will treat all options that don't start with a minus as a URL, which makes it convert the dash-X string to a IDN hostname and try it. It then continues to try the "POST" host name as that follows the dash-X... None of those host names can be resolved, which is the curl error messages you see.
Then finally: don't use -X POST when you do a post with -d (or with -F)! Just remove the -X POST entirely and things will work better.
check the codepage settings of the terminal software in use and compare these to the host settings
in our case we saw the same strange hostname error returned to a simple
curl -v http://{hostname}:{port}
we discovered the problem was the dash character; the codepage specified in Putty was not consistent with the codepage used at the host so curl treated the -v expression as the hostname.

How do I load JSON attributes pass via knife through chef recipes?

I have a web server that basically accepts input for a knife ec2 create server command.
In my knife command, I pass in
-j '{"branch":"clone"}'
From documentation I thought I would be able to do this in my recipes that are being loaded:
##branch = node['branch']
...
git clone git#github.com/blah -b #{##branch}
However, the git clones are failing because ##branch doesn't return. Or atleast the chef logs say begin output of result:
git clone git#github.com/blah -b
~Git error showing options and appropriate input~
How do I load in my passed JSON attribute? What is the correct way?
Edit:
I manually put in the knife command, and the JSON Attributes seem to load using Ruby hash instead of JSON
JSON Attributes: {"branch"=>"Air"}
Last Edit:
My command:
sudo knife ec2 server create -x ubuntu -i ~/.ssh/key.pem -I ami-0eca6e67 -d ubuntu-12.04 -j '{ "branch" : "clone" }' -Z us-east-1a -s subnet-6827ec00 -f c1.medium -g sg-bc9d86d0 -r 'role[role]'
Answer below.
So it seems if I use a bootstrap template that already exists, -d ubuntu-12.04, knife ec2 does NOT load -j json-attributes.
Get rid of the -d option. Json loads appropriately.

Troubleshoot varnish and dual server configuration

I have Varnish set up and working with 2 server instances.
I've made changes to the default.vcl file and updated my changes as follows on both my servers:
$NOW = fdfdf;
sudo varnishadm -T xx.xx.xx.xx: -S /etc/varnish/secret vcl.load reload$NOW /etc/varnish/default.vcl && sudo varnishadm -T xx.xx.xx.xx: -S /etc/varnish/secret vcl.use reload$NOW"
One server reflects the new change and works fine, but the other still seems to be using the old configuration.
Does anyone have idea why this might be or how to troubleshoot?
Thanks,
If you execute each command manually, does it also work? My best guesses now are:
you're prompted for your sudo password on one system, not on the other (thus it hangs)?
you're pointing to a secret-file with incorrect permissions?
somehow your first varnishadm didn't return "true" and thus the vcl.use didn't trigger in the second part
So, best thing to do: execute each command manually and check the responses you get.

Resources