CSP script-src strict-dynamic with hash is blocking host domain - node.js

I have a react (create-react-app, not ejected) front end, node/express back end with the following csp config:
app.use(
helmet.contentSecurityPolicy({
directives: {
'script-src': [
"'self'",
"'sha256-47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU='",
"'sha256-10e801rrdN2Gq8YctvySwnSlugHJX+Xjgx1mhmij72w='",
"'sha256-e89fobGAetuB/6VgXYgfYEJo7toSqmridYOdrJoE6LU='",
"'strict-dynamic'",
'http:',
'https:',
"'unsafe-inline'",
],
'object-src': ["'none'"],
'base-uri': ["'none'"],
},
})
)
The hashes you see there were provided in previous csp errors. I located the three scripts, and used a hash tool and verified that the hashes in the above directives belong to those three scripts.
When I publish to heroku with this configuration, I test the url with https://csp-evaluator.withgoogle.com/ and it says everything is A-ok.
But my site will not render, and in chrome devtools I get the following CSP error, "Content Security Policy of your site blocks some resources":
As you can see, it's blocking resources from the host url, not 3rd party.
The scripts from the errors look like this, in index.html:
<script src="/static/js/runtime-main.11477cd6.js"></script>
<script src="/static/js/7.e1d80075.chunk.js"></script>
<script src="/static/js/main.c7392c1f.chunk.js"></script>
I visited the link provided with the error, "See how to set a strict CSP" and followed their instructions (to the letter) for hash based CSP, and only when that did not work, I added 'self', 'http:', 'https:', and 'unsafe-inline'. I think those are ok to leave there, they don't seem to be causing any problem. I even tried adding the actual domain, but nothing changed.
I have set the INLINE_RUNTIME_CHUNK=false environment variable, needed for CRA apps (in heroku config as well) which fixed some previous (different) problems, but now this is happening.
I thought maybe the problem was that the hash was made from the path to the script, literally src="/path/file.js" and not the actual script in that file, so I hashed those three scripts and added those hashes to the directives. Still not working.
Additionally, I noticed that with nonces, the examples show the script tag with an added nonce property, like <script src="something.js" nonce="noncerandom"></script> so, I tried to use 'integrity' with the hashes of the scripts, (not the file paths.) Now, there are no errors whatsoever, but also, nothing is rendering. There is nothing inside the root <div>s.
I give up! Please help. I have read at least 50 related questions on stackoverflow, followed every link recommended in the comments & answers, re-written and re-deployed 106 times according to heroku. (None of the problems happen on localhost, so I have to republish every time.)

Turns out, the script-src hashes AND the <script> tag integrity values must both be added, even when using middleware like helmet. I got the wrong impression that you didn't need to bother updating <script> tags unless you were setting your headers in a <meta> tag, but in fact, they are required in any case.
That means you have to manually replace all those hashes after every front end build, as those scripts will change. Not a big deal imo, just an extra manual chore during development. There's probably a package for that somewhere.
Here's what my helmet config ended up looking like:
app.use(
helmet.contentSecurityPolicy({
directives: {
'img-src': [
"'self'",
'data:',
'flagcdn.com',
'upload.wikimedia.org',
'openweathermap.org',
'hereapi.com',
'js.api.here.com',
],
'default-src': ["'none'"],
'script-src': [
"'sha256-gpDxdDuBGxxl88r6aymWROliaETfsyODwU6dpFZyIUU='",
"'sha256-33dcmxHc726AphEOtauUa39NPzHtsEPzEAX8PKd8NU0='",
"'sha256-vqol01UCQbQtIbFsadt22MWtP/EzXBhlXJVTdE3Z0Nk='",
"'sha256-vnJfeIr7hNIEwFqAV/GfKdJDn1SeGTUHl87WXU7cOxA='",
"'strict-dynamic'",
],
'object-src': ["'none'"],
'base-uri': ["'none'"],
'connect-src': [
'sheltered-scrubland-08732.herokuapp.com',
'https://*.here.com:*',
'https://*.hereapi.com:*',
'blob:',
],
'worker-src': ["'self'", 'blob:'],
'manifest-src': ['https://sheltered-scrubland-08732.herokuapp.com'],
// 'require-trusted-types-for': [`'script'`], // cannot use. 'script' value requires further specifications which are a mystery to solve some other time.
},
})
)
And here's the <script> tags in index.html:
<script
src="/static/js/runtime-main.11477cd6.js"
integrity="sha256-gpDxdDuBGxxl88r6aymWROliaETfsyODwU6dpFZyIUU="
></script>
<script
src="/static/js/7.e1d80075.chunk.js"
integrity="sha256-33dcmxHc726AphEOtauUa39NPzHtsEPzEAX8PKd8NU0="
></script>
<script
src="/static/js/main.c7392c1f.chunk.js"
integrity="sha256-vqol01UCQbQtIbFsadt22MWtP/EzXBhlXJVTdE3Z0Nk="
></script>
To get those hashes, I copied and pasted the actual source code from runtime-main.11477cd6.js etc., into https://report-uri.com/home/hash. This is something I never read about ANYwhere, but it did not make sense to me to make a hash out of <script> tags with file paths. What would be the point of hashing a file path??? And in more than one article, they actually suggested using the hashes provided in the devtools csp error messages, which I discovered were made from the file path script tags, which is probably why nothing was working at first. smh.

Related

MathJax TypeSetting

I'm using MathJax for my personal blog (hosted on Github using Jekyll).
I notice that MathJax equations flicker when I refresh the page, the font was originally relatively small, and it looks thin, and less than half a second later, it would refresh and become much bolder.
I think I like the thin font style and smaller equations (that look much better inline with text) than the bolder version, so I try to configure it but failed. This is the documentation I'm looking at right now: http://docs.mathjax.org/en/latest/options/output-processors/HTML-CSS.html#configure-html-css
Here is my set up that's not working:
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
tex2jax: {
inlineMath: [['$','$'], ['\\(','\\)']],
displayMath: [['$$','$$']],
processEscapes: true,
skipTags: ["script","noscript","style","textarea"],
preferredFont: "TeX",
scale: 90,
EqnChunkFactor: 1,
EqnChunk: 1,
EqnChunkDelay: 10
}
});
</script>
<script
type="text/javascript"
charset="utf-8"
src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<script
type="text/javascript"
charset="utf-8"
src="https://vincenttam.github.io/javascripts/MathJaxLocal.js">
</script>
On the side note, I also have this annoying error message MathJaxLocal.js:1 Uncaught ReferenceError: MathJax is not defined.
Would really appreciate if someone answers this question!!
You're asking two fairly different questions but let me wrap them together anyway.
flickering
The "flickering" is (probably) the PreviewHTML output )docs). This may be surprising but comes from the fact that the combined configuration file you're loading (TeX-AMS-MML_HTMLorMML) MathJax will run the PreviewHTML output first, then the HTML-CSS output (cf. the combined config docs and the fastpreview extension docs.
You can use the PreviewHTML output like any other output manually but keep in mind that it is a far less complete (but faster) output processor which does not require webfonts (but uses whatever Times-like fonts the system has).
So following the configuration docs, something like
MathJax.Hub.Config({
messageStyle: "none",
extensions: ["tex2jax.js"],
jax: ["input/TeX", "output/PreviewHTML"],
tex2jax: {
inlineMath: [ ['$','$'], ["\\(","\\)"] ],
displayMath: [ ['$$','$$'], ["\\[","\\]"] ],
processEscapes: true
skipTags: ["script","noscript","style","textarea"]
}
TeX: {
// whatever is in MathJaxlocal.js
}
});
MathJax not defined
The error is due to the fact that all scripts on the page are loaded asynchronously. Very likely, MathJaxlocal.js will load and execute before MathJax.js (since it's on the same domain).
You'll need to ensure that the configuration is loaded before MathJax is. Luckily, MathJax can do that for you cf the docs.
Put your configuration in MathJaxLocal.js and then only load
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=https://vincenttam.github.io/javascripts/MathJaxLocal.js"></script>
See the link on how to add other configuration files, including combined ones from the CDN - which would make sense if you want to go back to a full output processor as the combined configurations are packaged more efficiently.

gulp-inject: missing end tag for start tag when using pug

I have a rather specific problem: I am trying to build a complex stack using Sails.js with machinepack-sailsgulpify.
In order to inject assets into my templates I use gulp-inject plugin, as the machinepack suggests. The problem is that for anything other than html and ejs the injector doesn't work. It simply doesn't change anything. No errors, nothing.
My task looks like this:
gulp.task('sails-linker-gulp:devViews', function() {
return gulp.src('views/**/*.?(html|ejs|jade|pug|haml|slim|dust)') // Read templates
.pipe(
plugins.inject(
gulp.src(require('../pipeline').jsFilesToInject, {read: false}), // Link the javaScript
{
starttag: generateScriptStartTag,
endtag: generateScriptEndTag,
ignorePath: '.tmp/public',
transform: (filepath, file, i, length) => {
return `script(src="${filepath}")`;
}
}
)
)
.pipe(
plugins.inject(
gulp.src(require('../pipeline').cssFilesToInject, {read: false}), // Link the styles
{
starttag: generateStyleStartTag,
endtag: generateStyleEndTag,
ignorePath: '.tmp/public'
}
)
)
.pipe(
plugins.inject(
gulp.src( ['.tmp/public/jst.js'], {read: false}), // Link the JST Templates
{
starttag: generateTemplateStartTag,
endtag: generateTemplateEndTag,
ignorePath: '.tmp/public'
}
)
)
.pipe(gulp.dest('views/'))// Write modified files...
Don't worry about the generateScriptStartTag and such functions, they are just there for control and I am 1000% sure they work correctly, tested a lot. They generate the tags kind of like this:
//- SCRIPTS
//- SCRIPTS END
depending on the template language.
Adding custom transform function did not work. If I use ejs or html or really anything that resembles html syntax it works fine.
Now, about Sails: I can NOT add a gulp task to compile the template before injecting because Sails renders templates on request in development, it doesn't actually pre-compile them into any directory. And honestly: why should I? The injection is just adding lines to my .jade/.pug files in views, the files are there already, so I don't see why there's a problem there. Can someone advise?
UPDATE:
Rather frustrating inspection of the code revealed that the 'matches' property when running the inject function of has length 0 and when inspecting the content of the stream in node inspector, I did not see the comments, they were stripped away, despite the fact that they are clearly there in the file.
UPDATE #2:
It appears that I was wrong about ejs. ONLY HTML files are getting processed. Also it works OK when it doesn't detect the injection comments. However if it does the end event simply never emits for that file and nothing gets injected. This is true for ALL templating engines, only static HTML files have injection working fine.
UPDATE #3:
After another 5 hours of debugging I found the problem, however my understanding of streams isn't good enough to get me any closer to the solution. The problem is that in inject function of the plugin there's a loop that doesn't quit properly, and while it perfectly injects the required tags into the stream, it then runs that loop again on the same stream (with injected tags), and throws an error.
Why that error never showed up in any console I don't know but there you go. Can someone please help? I am completely lost with this... Is it a bug in the plugin?
I had to figure this out on my own.
It is a bug in gulp-inject. The regex that this plugin generates to test against the injection tags does not match the whole line, it's simply matches the first occurrence. This means that if I have my tags set like so:
//SCRIPTS
//SCRIPTS END
The regex will match the starttag: //SCRIPTS twice:
And the end tag will only be matched once. This was causing the second faulty loop with the error for missing end tag.
A workaround is to avoid repeating start of tags.
//SCRIPTS
//END SCRIPTS
That's not a solution, however. A solution would be to alter the regex so that it only allows whitespace and newline characters in order to match the tag, when using an indent-based template language.
Something like this would work: /\/\/-\s*SCRIPTS(?=\s*\n|$)/ig
Can't believe nobody has stumbled upon this until now, seems like it would be a more common problem...

How to use Magento2 with RequireJS?

I'm just migrating from Magento 1.x to Magento 2.x. I found that Magento2 uses RequireJS for handling JavaScript files. So I learnt what RequireJS is, and how to use it.
I found that most of the examples uses data-main="main" to define the configuration file.
In Magento2's default_head_blocks.xml file, I found the script tag like this:
<script src="requirejs/require.js"/>
Here they did not specify any data-main.
These are my questions:
How Magento2/RequireJS knows which JS should be loaded for configuration? (I found requirejs-config.js for this in multiple places)
By default Magento2 loads lots of JS (more than 20), how can I limit them?
I could not find enough documentation on this.
The best place to get all your answers for Magento 2 JS development is the Magento 2 docs, it really is a useful resource on this. http://devdocs.magento.com/guides/v2.0/javascript-dev-guide/javascript/js_init.html explains in detail about component initialisation.
To answer your two questions above -
Q.1. How Magento2/RequireJS knows which JS should be loaded for
configuration? (I found requirejs-config.js for this in multiple
places)
In each Magento 2 module there is a requirejs-config.js file to load all that modules configuration. i.e.
var config = {
map: {
'*': {
compareItems: 'Magento_Catalog/js/compare',
compareList: 'Magento_Catalog/js/list',
relatedProducts: 'Magento_Catalog/js/related-products',
upsellProducts: 'Magento_Catalog/js/upsell-products',
productListToolbarForm: 'Magento_Catalog/js/product/list/toolbar',
catalogGallery: 'Magento_Catalog/js/gallery',
priceBox: 'Magento_Catalog/js/price-box',
priceOptionDate: 'Magento_Catalog/js/price-option-date',
priceOptionFile: 'Magento_Catalog/js/price-option-file',
priceOptions: 'Magento_Catalog/js/price-options',
priceUtils: 'Magento_Catalog/js/price-utils',
catalogAddToCart: 'Magento_Catalog/js/catalog-add-to-cart'
}
}
};
This is telling requirejs where all the required JavaScript files are located.
There are multiple way to tell Magento when to use your JS file -
data-mage-init on a HTML element. e.g. <div class="block upsell" data-mage-init="{"upsellProducts":{}}" data-limit="0" data-shuffle="0">
script tag on the page e.g
<script type="text/x-magento-init">
{
"[data-role=tocart-form], .form.map.checkout": {
"catalogAddToCart": {}
}
}
</script>
within a JS file e.g. $('.yourSelector').yourPlugin();
Q.2. By default Magento2 loads lot's of JS (more than 20), how can I
limit them?
The sheer number of JS files that are loaded as a result of multiple modules is one of the downsides, however, with the correct usage of full page caching with a reverse proxy like Varnish the performance reduction is negligible, even in a development server.

Why would 'data:' be a script-src for CSP?

I've seen the following Content Security Violation in my server's logs. When would there be a 'data:' type script-src? Isn't 'data:' only for e.g. base64 encoded images?
CSP violation!
{ 'csp-report':
{ 'blocked-uri': 'data:',
'document-uri': 'https://certsimple.com/blog/domain-validated-ssl',
'original-policy': longPolicyGoesHere,
referrer: '',
'violated-directive': 'script-src https://example.com https://use.typekit.net \'unsafe-inline\' https://js.stripe.com \'unsafe-eval\' https://platform.twitter.com https://cdn.mxpnl.com https://syndication.twitter.com' } }
data: is for base64 encoded, embedded data. While the most popular usage is to encode images into stylesheets to reduce the number of requests, that is not the only use. The URI scheme can be used for scripts like the following:
<script src="data:application/javascript;charset=utf-8;base64,YWxlcnQoJ1hTUycpOw=="></script>
Also available on jsfiddle.
The report you're seeing is legitimate, something is attempting to inject arbitrary javascript into your page using the Data URI scheme to obfuscate what it's doing. While this could be reflective of an issue in your application, it's more likely going to be a rogue browser extension that's either malicious and trying to do sneaky things or benign and very badly coded.

How does the text! plugin use the baseUrl?

I'm having an issue getting the text! plugin to work in my requirejs site. It's always including lib/ in the request url, however all of the other files (not using text!) are being successfully found and loaded. Here is my directory structure:
WebContent/
|--backbone/
|--Bunch of folders and files
|--config/
|--config.js
|--lib/
|--jquery.js
|--text.js
|--require.js
|--index.html
my index.html file is:
<body>
<div id="siteLayoutContainer"></div>
<script data-main='config/config' src="lib/require.js"></script>
</body>
The config file is:
requirejs.config({
baseUrl: './',
paths: {
jquery: 'lib/jquery.js',
backbone: 'lib/backbone.js',
text: 'lib/text',
application: 'backbone/application'
},
text: {
env: 'xhr'
}
});
require(['application'], function(App) {
App.start();
});
I'm using the text! plugin like so:
define([
'jquery',
'text!backbone/templates/SomeTemplate.html'
], function(jQuery, NotFoundHtml) {
//Some code here
}
So, in the above script, the url being used for the template is:
http://localhost/lib/backbone/templates/SomeTemplate.html
and I am expecting it to be:
http://localhost/backbone/templates/SomeTemplate.html
I've tried the following:
Moving the text.js and require.js files out into the WebContent
directory but I get the same results. Also something interesting is
if I put a space after text! and then the path, that works fine and
doesn't include the lib/ directory in the request to get the html
template. However the optimizer includes the space and can't find the
template.
Not defining a baseUrl - same results.
Moved the
require config.js content into index.html in it's own script tag that runs
before the require.js script tag - same results.
Getting rid of the the text options in the config file
Oh yeah, forgot I've also tried 'text!../backbone/templates/SomeTemplate.html - same results
So I'm stuck and can't figure out what I'm missing. I'm obviously not understanding how the text! plugin uses the baseUrl or how it determines the url it's going to use to fetch the defined file.
After your edits to your question, it now contains all the information to diagnose the problem. As you guessed in one of your comments, the issue is indeed that this path:
backbone: 'lib/backbone.js',
is throwing off the resolution of the template you give to the text plugin. When the text plugin loads what you give to it, it takes the path after the ! symbol and treats it as if it were a module name, and it goes through the module resolution process. The way module resolution works is that it checks if there is a prefix that matches any of the keys in paths and will change the prefix with the value associated with the key, which gives the result you obtained. One way to fix the issue would be to add this to your paths configuration:
"backbone/templates": "backbone/templates"
This will make it so that anything you request under backbone/templates won't get messed up by the backbone path.
Note: it is preferable to avoid putting extensions in module names so you should remove it from the values you have for jQuery and Backbone.

Resources