I have some additional processing, and would like to generate some javascript files based on existing content.
Therefore, I need my code to be called after the site compilation and generation, so I can create new files in the output.
How to do that in nanoc? Thank you!
You can use Ruby code in Rules with the postprocess method:
postprocess do
items.each do |item|
if item.identifier.to_s.end_with?(".js")
# Do stuff
end
end
end
Related
We have a Kiba pipeline where we need to do some task after the job has ended, no matter if there were errors or not (the whole pipeline doesn't fail, we just have couple of validation errors or similar).
This is what the documentation says:
:warning: Post-processors won't get called if an error occurred before them.
https://github.com/thbar/kiba/wiki/Implementing-pre-and-post-processors
Would this be recommended way to do this:
Kiba.run(
Kiba.parse do
source(...)
transform(...)
destination(....)
post_process do
# we cannot do it here, because it won't get called
end
end
)
# is this the location to do it?
Job.some_special_cleanup_task
Thanks!
PS what does it mean:
Post-processors won't get called if an error occurred before them.
Does this mean if the error occurred and wasn't rescued from?
Indeed post_process will not be called in case of errors, as documented and as you pointed out!
At this point, the best solution is to use a form of ensure statement:
A common way to structure that is:
module ETL
module Job
module_function
def setup(config)
Kiba.parse do
source(...)
transform(...)
destination(....)
end
end
def some_special_cleanup_task
# ...
end
def run(job)
Kiba.run(job)
ensure
Job.some_special_cleanup_task
end
end
end
Doing so allows to keep the code for the always-run task close to the ETL job, which is nice.
If your task is instead very independent from the job, and you want to encourage reuse between jobs, you can also create a generic block-form component:
module ETL
module Middlewares
module CommonTask
module_function
def with_common_task
yield
ensure
do_the_common_task
end
end
end
end
Which you would use like this:
ETL::Middlewares::CommonTask.with_common_task do
Kiba.run(...)
end
This second form is used by Kiba Pro FileLock, for instance.
In the future, Kiba ETL will introduce a form of middleware to make this even easier.
Hope this helps, and please mark the question as answered if it properly solves your problem!
I use sub-flows extensively and would like to have an option to generate a separate ATE flow file per sub-flow (e.g. on V93K a separate .tf file). Is this possible? Perhaps something like this, wih the default being the current behavior:
import :my_subflow, generate_standalone: true
The reason this is valuable is that we use a test program assembler that can merge multiple flow files together given a sequence of flow files. This saves on test program load time for debug sessions.
thx
There is no switch like that currently, but note that you can generate sub-flows directly:
origen p program/_my_subflow.rb
The leading "_" in the name means that they will be not be generated as standalone flows if you point the program generator to the directory (origen p program/), but if you explicitly ask it to generate that file then it will.
During my Grunt tasks, add a unique string to the end of my filenames. I have tried grunt-contrib-copy and grunt-filerev. Neither have been able to do what I need them to...
Currently my LESS files are automatically compiled on 'save' in Sublime Text 3 (so this does not yet occur in my grunt tasks). Then, I open my terminal and run 'grunt', which concatenates (combines) my JS files. After this is done, then grunt should rename 'dist/css/main.css' and 'dist/js/main.js' with a "version" at the end of the filename.
I have tried:
grunt-contrib-copy ('clean:expired' deletes the concatenated JS before grunt-contrib-copy' can rename the file)
grunt-filerev ('This only worked on the CSS files for some reason, and it inserted the version number BEFORE the '.css'. Not sure why it didn't work on the JS files.')
Here's my Gruntfile.js
So, to be clear, I am not asking for "code review" I simply need to know how I can incorporate a "rename" process so that when the tasks are complete, I will have 'dist/css/main.css12345 & dist/js/main.js12345' with no 'dist/css/main.css' or 'dist/js/main.js' left in their respective directories.
Thanks in advance for any help!
UPDATE: After experimenting with this, I ended up using grunt-contrib-rename and it works great! I beleieve the same results can be achieved via grunt-contrib-copy, in fact I know it does the same thing. So either will work. As far as support for regex, not sure if both support it, so may be something else worth looking into before choosing one of these plugins :)
Your rename:dist looks like it should do what you want, you just need to move clean:dist to be the first task that runs (so it deletes things from the prior build rather than the current build). The order of tasks is defined by the array on this last line:
grunt.registerTask('default', ['jshint:dev', 'concat:dist', 'less:dist', 'csslint:dist', 'uglify:dist', 'cssmin:dist', 'clean:dist', 'rename:dist']);
That said, I'm not sure why you want this behavior. The more common thing to do is to insert a hash of the file into the filename before the file extension.
The difference between a hash and a timestamp is that the hash value will always be the same so long as the file contents don't change - so if you only change one file, the compiled output for just that file will be different and thus browsers only need to re-downloaded that one file while using cached versions of every other file.
The difference between putting this number before the file extension and after the extension is that a lot of tools (like your IDE) have behavior that changes based on the extension.
For this more standard goal, there are tons of ways to accomplish it but one of the more common is to combine grunt-filerev with grunt-usemin which will create properly named files and also update your HTML file(s) to reference these new file names
I'm not sure to understand completely what end you want, but if you add a var timestamp = new Date().getTime(); at the beginning of your gruntfile and concatenate to your dest param that should do the job.
dest: 'dist/js/main.min.js' + timestamp
Is it what your looking for?
Is it possible to load the step definitions I have defined into the calabash-android console?
I would like to be able to use them when navigating the app within the console.
Thanks
No from the console you can not run a single step definition.
But you can start execution of a test at a specific line appending parameter to the call to start your test
:<linenumber>
This will start execution of your feature file from that specific line and it will run from there to the end of the file.
So while it is not what you are looking for at least it is something.
Did you try step('<step_name>') method?
To be honest I'm not sure if this will work. I know it's working insinde Ruby methods and step definitions - I wanted to post a comment but I can't with 28 points of reputation ;)
You can also try making ruby methods with code from within the step definition:
Then /^I do something$/ do
some code
goes here
end
def do_something
some code
goes here
# same code as in step definition
end
or just use step method:
def do_something
step('I do something')
end
and then call it in a calabash console (I prefer using binding.pry inside some script rather than calling "pure" calabash-console - it makes me sure that I will have all needed methods included).
I'm trying to automate my work of converting PDF to png file with scons. The tool used for my conversion is convert from ImageMagick.
Here's the raw command line:
convert input.pdf temp/temp.png
convert temp/*.png -append output.png
The first command will generate one PNG file for each page in PDF file, so the target of the first command is a dynamic file list.
Here's the SConstruct file I'm working on:
convert = Builder(action=[
Delete("${TARGET.dir}"),
Mkdir("${TARGET.dir}"),
"convert $SOURCE $TARGET"])
combine = Builder(action="convert $SOURCE -append $TARGET")
env = Environment(BUILDERS={"Convert": convert, "Combine": combine})
pdf = env.PDF("input.tex")
pngs = env.Convert("temp/temp.png", pdf) # I don't know how to specify target in this line
png = env.Combine('output.png', pngs)
Default(png)
The code pngs = env.Convert("temp/temp.png", pdf) actually is wrong since the target is multiple files that I don't know how many before env.Convert is executed, so the final output.png only contains the first page of the PDF file.
Any hint is appreciated.
UPDATE:
I just found that I can use command convert input.pdf -append output.png to avoid the two-step conversion.
Still I'm curious how to handle the scenario when the intermediate temporary file list is unknown beforehand and requires a dynamic target list.
If you want to know how to do the original (convert and combine) situation you proposed, I would suggest creating a builder with a SCons Emitter. The emitter allows you to modify the list of source and target files. This works nicely for generated files that dont exist with a clean build.
As you mentioned, the convert step will generate multiple targets, the trick is you need to be able to "calculate" those targets in the emitter based on the source. For example, recently I created a wsdl2java builder and was able to do some simple wsdl parsing in the emitter to calculate all of the target java files to be generated (the source being the wsdl).
Here is a general idea of what the build scripts should look like:
def convert_emitter(source, target, env):
# both and source and target will be a list of nodes
# in this case, the target will be empty, and you need
# to calculate all of the generated targets based on the
# source pdf file. You will need to open the source file
# with standard python code. All of the targets will be
# removed when cleaned (scons -c)
target = [] # fill in accordingly
return (target, source)
# Optionally, you could supply a function for the action
# which would have the same signature as the emitter
convert = env.Builder(emitter=convert_emitter,
action=[
Delete("temp"),
Mkdir("temp"),
"convert $SOURCE $TARGET"])
env.Append(BUILDERS={'Convert' : convert})
combine = env.Builder(action=convert_action, emitter=combine_emitter)
env.Append(BUILDERS={'Combine' : combine})
pdf = env.PDF('input.tex')
# You can omit the target in this call, as it will be filled-in by the emitter
pngs = env.Convert(source=pdf)
png = env.Combine(target='output.png', source=pngs)
Depending on what qualifies as "dynamic" for you, I believe the correct answer is: not possible.
As long as the source on which you would like to "dynamically" compute a target set is present when SCons is run, #Brady's solution should work fine. However, if the source in question itself is the target of some other command, it will not work. This is a fundamental limitation of SCons, as it makes the assumption that the set of build targets can be statically determined from the base set of input (non-intermediate) sources. It runs through and computes a build/target/dependency graph in one sweep, then executes it in the next. It has no ability to run through some known portion of the build graph, stop to introspect some intermediate targets to dynamically compute the rest of the build graph, and then continue. I'd frankly love for this ability in the work that I do with SCons, but I'm afraid this is just a fundamental limitation.
The best you can do is set the build up so that on the first run, it stops at the construction of the PDF (if no PDF target exists when the build script is executed). Once the PDF has been built, you can rerun the build and set things up so the rest of the build steps execute based on the PDF built from the last run. This more or less works decently... except for one problem. If the PDF ends up changing (and producing some new pages for instance), you'll actually have to rerun the build twice in order to capture the changes to the PDF, since any page counts (etc) will be based on the old version of the PDF.
I'd love for someone to prove me wrong here, but such is the way of things.
Looking at this, there's no requirement for the individual temp/*png to be kept - if there was, you shouldn't be putting them in a temp directory, and in any case you'd have to do quite a bit of work if you wanted to work out which pages to generate.
So it looks more sensible to do this as one step, this So you'd have something like this
png = env.Convert('output.png', 'input.pdf')
where the action function for convert was something like this:
Delete('temp'),
Mkdir('temp'),
'convert $SOURCE temp/$TARGET',
'for i in temp/*png; do convert $TARGET temp/$i',
Delete('temp')
Though frankly you might do better with writing that whole thing as a single callable script to make sure you got the page sorting correct.