I'm trying to store whole the output of my build, this includes some empty folders. These aren't included by the artefact mechanism in teamcity:
What doesn't work:
OAR\=> OAR.zip
OAR->OAR.zip
OAR
Inside of OAR i have a folder structure that needs to be stored. I know i could put a placeholder file in each but that is not the answer i'm after. Otherwise ill have to zip it myself?
Unfortunately TeamCity, by design, searches for files and uploads them as artifacts which means that empty folders are never included. Given the open and very old issue in the TeamCity tracker I doubt they are going to fix it any time soon.
I would recommend zipping the folder yourself, that is the approach we have taken. How you implement that depends on the build technology you are using. For example, if you are building using Nant you could add the zip task to your build, there are similar options for MSBuild and Ant.
If you don't want to rely on the build performing the zip I would recommend installing 7zip on your build agents and using the command line to perform the zip. Just remember if you want 7zip to include empty directories use * as the wildcard rather than *. * like so:
7z a -r OAR.zip *
Technically you could use powershell to do the zipping, which would be better than having to install something on your agents. I haven't tried this option myself.
Apologies for not linking all my references above. Apparently, and understandably so, I need at least 10 reputation to post more than 2 links.
Related
Is it possible to create dirs(folders) on cx_freeze build output, cause i include(include_files) many databases files and i want these to be in specific folder etc. I can take them easily from my folders.....
"include_files": ["databases/nations.txt","databases/newafrica.txt",
"databases/neweeurope.txt","databases/neweurope.txt","databases/newmeast.txt","graph.py",
"databases/newnamerica.txt","databases/plates.txt",
"databases/ACN/rigidA.txt","databases/ACN/rigidB.txt",
"databases/ACN/rigidC.txt","databases/ACN/rigidD.txt","databases/ACN/flexibleA.txt",
"databases/ACN/flexibleB.txt","databases/ACN/flexibleC.txt",
"databases/ACN/flexibleD.txt","alternates.xlsx",
but this will just copy all of them in exe build dir and its a mess.
Thanks in advance.
There are several ways you can go around solving the problem.
Method 1 - Using include_files
Rather than ask for each individual text file you could just put the file name in the setup script and leave out the individual text files. In your case it would be like this:
"include_files": ["databases"]
This would copy the entire databases folder with everything in it into you build folder.
Absolute file paths work as well.
If you are going to use the installer feature (bdist_msi) this is the method to use.
You can copy sub folders only using "include_files": ["databases/ACN"]
Method 2 - Manually
Ok it's rather un-pythonic but the one way to do it is to copy it manually into the build folder.
Method 3 - Using the os module
Much the same as method two it would copy the folder into your build folder but instead of coping it manually it would use Python. You also have the option of using additional Python features as well.
Hope I was helpful.
I'm trying to set up a build system involving a code generator. The exact files generated are unknown until after the generator is run, but I'd like to be able to run further build steps by pattern matching (run some program on all files with some extension). Is this possible?
Some of the answers here involving code generation seem to assume that the output is known or a listing of generated files is created. This isn't impossible in my case, but I'd like to avoid it since it makes things more complicated.
https://bitbucket.org/scons/scons/wiki/DynamicSourceGenerator seems to indicate that it's possible to add additional targets during Builder actions, but while I could get the build to run and list the generated files, any build steps introduced don't run.
https://bitbucket.org/scons/scons/wiki/NonDeterministicDependencies uses Scanners to add build steps. I put a glob(...) in a scanner, and it succeeds in detecting the generated files, but the files are inexplicably deleted before it actually runs the dependent step.
Is this use case possible? And why is SCons deleting my generated files?
A toy example
source (the file referenced in SConscript)
An example generator, constructs 3 files (not easily known to the build system) and puts them in the argument folder
echo "echo 1" > $1/gen1.txt
echo "echo 2" > $1/gen2.txt
echo "echo 3" > $1/gen3.txt
SConstruct
Just sets up a variant_dir
SConscript('SConscript', variant_dir='build')
SConscript
The goal is for it to:
"Compile" the generator (in this toy example, just copies a file called 'source' and adds execute permissions
Run the "compiled" generator ('source' is a script that generates files)
Perform some operation on each of those generated files by extension. This example just runs the "compile" copy operation on them (for simplicity).
env = Environment()
env.Append(BUILDERS = {'ExampleCompiler' :
Builder(action=[Copy('$TARGET', '$SOURCE'),
Chmod('$TARGET', 0755)])})
generator = env.ExampleCompiler('generator', 'source')
env.Append(BUILDERS = {'GeneratorRun' :
Builder(action=[Mkdir('$TARGET'),
'$SOURCE $TARGET'])})
generated_dir = env.GeneratorRun(Dir('generated'), generator)
Everything's fine up to here, where all the targets are explicitly known to the build system ahead of time.
Attempting to use this block of code to glob over the generated files causes SCons to delete (!!) the generated files:
for generated in generated_dir[0].glob('*.txt'):
generated_run = env.ExampleCompiler(generated.abspath + '.sh', generated)
Attempting to use an action to update the build tree results in additional actions not being run:
def generated_scanner(target, source, env):
for generated in source[0].glob('*.txt'):
print "scanned " + generated.abspath
generated_target = env.ExampleCompiler(generated.abspath + '.sh', generated)
Alias('TopLevelAlias', generated_target)
env.Append(BUILDERS = {'GeneratedOperation' :
Builder(action=[generated_scanner])})
dummy = env.GeneratedOperation(generated_dir[0].File('#dummy'), generated_dir)
Alias('TopLevelAlias', dummy)
The Alias operations are suggested in above dynamic source generator guide, but don't seem to do anything. The prints do execute and indicate that the action gets run.
Running some build pattern on special file extensions is possible with SCons. For C/CPP files this is the preferred scheme, for example:
env = Environment()
env.Program('main', Glob('*.cpp'))
The main task of SCons, as a build system, is to do the minimum amount of work such that all your targets are up-to-date. This makes things complicated for the use case you've described above, because it's not clear how you can reach a "stable" situation where no generated files are added and all targets are built.
You're probably better off by using a simple Python script directly...I really don't see how using SCons (or any other build system for that matter) is mission-critical in this case.
Edit:
At some point you have to tell SCons about the created files (*.txt in your example above), and for tracking all dependencies properly, the list of *.txt files has to be complete. This the task of the Emitter within SCons, which is responsible for returning the list of resulting target and source files for a Builder call. Note, that these files don't have to exist physically during the "parse" phase of SCons. Please also have a look at my answer to Scons: create late targets , which goes into some more detail.
Once you have a proper Emitter in place (see also https://bitbucket.org/scons/scons/wiki/ToolsForFools , "Using Emitters") you should be able to use the Glob('*.txt') call, which will detect and track your created files automatically.
Finally, on our page "Talks and Slides" ( https://bitbucket.org/scons/scons/wiki/TalksAndSlides ) you can find my talk from the PyCon FR.2014, "Why SCons is Not Slow", which explains shortly how SCons works internally. This might be helpful in understanding this problem better and coming up with a full solution.
I'm using autotools to package my software and compile. The problem I'm having is that during the installation process, I'm creating a folder in /etc/myapp and in that folder I'm placing several files that I need. In addition, when my software is running, I'm generating files and storing them in that same location (/etc/myapp). When I execute "sudo make uninstall", all of the files that were initially installed by using "sudo make install" are removed from /etc/myapp. However, the files that are generated by the software and store in that same spot are not removed and now I have left over files.
Where in the Makefile.am files would I specify to remove the entire /etc/myapp folder during the uninstall process?
FYI, I'm using Ubuntu 12.
Thank you
D
In your Makefile.am, you can create a target named uninstall-hook which will delete your generated files:
uninstall-hook:
rm -f $(DESTDIR)$(datadir)/my_generated_file $(DESTDIR)$(datadir)/my_other_generated_file
Or even:
uninstall-hook:
rm -rf $(DESTDIR)$(datadir)
Your question involves more than just tooling; it involves policy. What type of files are you creating? If they can be lost over a reboot, they (probably) belong under /var/run/myapp. If they are data files, they probably belong in $(prefix)/share/myapp. I say "probably" because it depends on the platform, and the user. Most debian users will want such files in the locations I mention above, but they may have reasons for putting them elsewhere. The important point is that it is the user's choice, not yours. As the package maintainer, you don't get to choose /etc/myapp. You need to configure your package to make those choices available to the user. The typical way to do that is to use automake's _DATA primaries and the 'sysconfdir', 'sharedstatedir', 'localstatedir', and 'pkgdatadir' family of variables.
Let there be:
There are different repositories repoA, repoB and repoC each respecting the same directory layout principles, which are to be merged onto a third repoM's working directory (the "master" project).
repoM has an atypical setup (--work-dir and --git-dir are sepparate). repo[A-C] are cloned as bare, and they are set as core.bare = false and core.worktree=<--work-dir-of-repoM>.
The requirements:
I need to always have an overview over the history of all files in repoM's work-dir, which could have stemmed from repo[A-C]. With this approach, I lose all that information.
Alternative:
I've been thinking about using git-subtree instead (git version 1.7.11.2, so it's already built-in), leaving repo[A-C] bare, and then
git pull -s subtree, or
git subtree ...
With the subtree pull strategy, I lose the history on a merge conflict (git blame says so).
I've never used subtree before, but from my understanding it's not possible to merge files from repo[A-C] into repoM's work-dir, those files must be put into a subdirectory of repo[A-C]. This is definitely not what I need. Why? Because of the following ...
Problem statement:
You have different git repositories each containing different sets of files, usually configuration files and some shell scripts. You want to put everything in the $HOME (which is <--work-dir-of-repoM>) directory from all those repositories. You should be able to see at all time where each file comes from, edit, commit and push changes to each one's origin. You've guessed it, it something like vundle, but generalized for any kind of configuration of any program, not just vim bundles. If a conflict occures, one should be able to track down which two authors of the same file need to get in touch with each other and make up a deal (if one needs to be made).
This is for an open-source project I'm trying to get a prototype working, so any help is highly appreciated. Also ideas about already existing projects which do this in a similar manner are highly appreciated.
Note: the "master directory" does not necessarily have to be $HOME, I've used it as a possible hint on the kind of problem this could solve.
Why not simply use Git Submodules in your "master project"?
Does anyone have any example of how to use modification reader task?
Ok, I use this over XML:
<modificationReader>
<filename>mods.xml</filename>
<path>path/to/my/file/</path>
</modificationReader>
then, what? How do I get the information in "mods.xml" and use it?
Thanks
This appears to be used with the modificationWriter task which writes the modifications to a file (in the artifact directory by default).
http://build.sharpdevelop.net/ccnet/doc/CCNET/Modification%20Writer%20Task.html
If you're just trying to read in the modifications in to a different projects' buildLog, the above - with a path to the first project - should be sufficient.
Are you trying to do something different?
CruiseControl.NET: Build subproject obtained by SVN